Over the past few years, various tasks involving videos such as classification, description, summarization and question answering have received a lot of attention. Current models for these tasks compute an encoding of the video by treating it as a sequence of images and going over every image in the sequence, which becomes computationally expensive for longer videos. In this paper, we focus on the task of video classification and aim to reduce the computational cost by using the idea of distillation. Specifically, we propose a Teacher-Student network wherein the teacher looks at all the frames in the video but the student looks at only a small fraction of the frames in the video. The idea is to then train the student to minimize  (i)  the difference between the final representation computed by the student and the teacher and/or (ii) the difference between the distributions predicted by the teacher and the student. This smaller student network which involves fewer computations but still learns to mimic the teacher can then be employed at inference time for video classification. We experiment with the YouTube-8M dataset and show  that the proposed student network can reduce the inference time by upto 30% with a negligent drop in the performance.
Teacher-Student framework for efficient video classification using fewer frames  The paper proposes an idea to distill from a full video classification model a small model that only receives smaller number of frames. The authors present a teacher-student network to solve video classification problem, proposing serial and parallel training algorithms aimed at reducing computational costs.