Developing conditional generative models for text-to-video synthesis is an extremely challenging yet an important topic of research in machine learning. In this work, we address this problem by introducing Text-Filter conditioning Generative Adversarial Network (TFGAN), a GAN model with novel conditioning scheme that aids improving the text-video associations. With a combination of this conditioning scheme and a deep GAN architecture, TFGAN generates photo-realistic videos from text on very challenging real-world video datasets. In addition, we construct a benchmark synthetic dataset of moving shapes to systematically evaluate our conditioning scheme. Extensive experiments demonstrate that TFGAN significantly outperforms the existing approaches, and can also generate videos of novel categories not seen during training.

An effective text-conditioning GAN framework for generating videos from text This paper presents a GAN-based method for video generation conditioned on text description, with a new conditioning method that generates convolution filters from the encoded text, and uses them for a convolution in the discriminator. This paper proposes conditional GAN models for text-to-video synthesis: developing text-feature-conditioned CNN filters and constructing moving-shape dataset with improved performance on video/image generation.