A novel adversarial inference framework for video prediction with action control
Document Type
Conference Proceeding
Publication Date
10-1-2019
Abstract
The ability of predicting future frames in video sequences, known as video prediction, is an appealing yet challenging task in computer vision. This task requires an in-depth representation of video sequences and a deep understanding of real-word causal rules. Existing approaches often result in blur predictions and lack the ability of action control. To tackle these problems, we propose a framework, called VPGAN, which employs an adversarial inference model and a cycle-consistency loss function to empower the framework to obtain more accurate predictions. In addition, we incorporate a conformal mapping network structure into VPGAN to enable action control for generating desirable future frames. In this way, VPGAN is able to produce fake videos of an object moving along a specific direction. Experimental results show that a combination of VPGAN with some pre-trained image segmentation models outperforms existing stochastic video prediction methods.
Identifier
85082500921 (Scopus)
ISBN
[9781728150239]
Publication Title
Proceedings 2019 International Conference on Computer Vision Workshop Iccvw 2019
External Full Text Location
https://doi.org/10.1109/ICCVW.2019.00101
First Page
768
Last Page
772
Recommended Citation
Hu, Zhihang and Wang, Jason, "A novel adversarial inference framework for video prediction with action control" (2019). Faculty Publications. 7280.
https://digitalcommons.njit.edu/fac_pubs/7280
