Document Type


Date of Award


Degree Name

Doctor of Philosophy in Computing Sciences - (Ph.D.)


Computer Science

First Advisor

Jason T. L. Wang

Second Advisor

Ioannis Koutis

Third Advisor

Jing Li

Fourth Advisor

Katherine Grace Herbert

Fifth Advisor

Eric Liguori


Deep Learning has spanned a variety of applications in computer vision as well as computational astronomy. These two aspects obtained similar data structure, therefore, their solutions can be transferable between each other. This dissertation look into two video-related tasks in computer vision and propose a novel problem in computational astronomy.

Specifically, acquiring an in-depth understanding of videos has been a cornerstone problem in computer vision. This problem has been studied by various researchers from different perspectives, among which video prediction has attracted much attention. Video prediction aims to generate the pixels of future frames given a sequence of context frames. In practice, unlabeled video sequences can be gathered autonomously from a sensor or recording device. A machine capable of predicting future events using these video sequences in an unsupervised manner will gain extensive and deep knowledge about its physical environment and surroundings. However, despite its appealing prospects, accurate video prediction remains an open problem. The major challenge is the inherent uncertainty in the dynamics of the world. A typical example is that the future trajectory of a ball hitting the ground is inherently random.

This dissertation proposes new generative adversarial networks (GANs) for stochastic video prediction. The proposed framework, dubbed Video-Prediction-GAN(VPGAN), employs an adversarial inference model and a cycle-consistency loss function to empower the framework to obtain more accurate predictions. In addition, a conformal mapping network structure is incorporated into VPGAN to enable action control for generating desirable future frames. In this way, VPGAN can produce fake videos of an object moving along a specific direction. Experimental results based on different datasets demonstrate the good performance of VPGAN and its superiority over existing methods. Other contributions of this dissertation include the development of a Three-Dimensional atrous convolutional long short-term memory network for background subtraction used in video processing and an extension of VPGAN for synthesizing vector magnetograms of active regions in different solar cycles. The dissertation concludes by pointing out some directions of future research in applying deep learning to computer vision and astronomy.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.