Cost Effective Embedded Action Recognition Using 3D CNN

Abstract

The recent 3D convolutional neural network (3D-CNN) is a promising candidate for solving the action recognition problem by providing the attractive algorithm-level performance. However, it is almost impractical to apply the advanced 3D-CNN architecture to the resource-limited real-time embedded system due to the excessive amount of computational costs. In this thesis, we present several optimization schemes that can relax the complexity of 3D-CNN processing without sacri cing the recognition accuracy. More precisely, we rst develop a number of 3D-CNN architectures for exploiting the trade-o between the network complexity and the recognition performance. Evaluating the current con dential level, then, the proposed method dynamically changes the network structure to be used for the inference processing of the next clip. In addition, we present in this work a systematic way for managing the network sequence for minimizing the computing overheads while supporting the acceptable algorithm-level performance. Compared to the previous work, as a result, the proposed scheme drastically reduces the processing latency as well as the energy consumption by selecting the most simplest 3D-CNN architecture at the run time, allowing the cost effective action recognition for embedded edges.