Yulan Zhao, Hyo Jong Lee

KIPS Transactions on Computer and Communication Systems, Vol. 11, No. 3, pp. 99-104, Mar. 2022
https://doi.org/10.3745/KTCCS.2022.11.3.99,   PDF Download:   Full Text
Keywords: Two-Stream Network, Teacher-Student Architecture, CNN, optical flow, Action Recognition
Abstract

Convolutional neural network (CNN) generally uses two-stream architecture RGB and optical flow stream for its action recognition function. RGB frames stream display appearance and optical flow stream interprets its action. However, the standard method of using optical flow is costly in its computational time and latency associated with increased action recognition. The purpose of the study was to evaluate a novel way to create a two sub-networks in neural networks. The optical flow sub-network was assigned as a teacher and the RGB frames as a student. In the training stage, the optical flow sub-network extracts features through the teacher sub-network and transmits the information to student sub-network for baseline training. In the test stage, only student sub-network was operational with decreased in latency without computing optical flow. Experimental results shows that our network fed only by RGB stream gets a competitive accuracy of 54.5% on HMDB51, which is 1.5 times better than that on R3D-18.

Statistics
Cite this article