Exploring Transfer Learning to Reduce Training Overhead of HPC Data in Machine Learning
Document Type
Conference Proceeding
Publication Date
8-1-2019
Abstract
Nowadays, scientific simulations on high-performance computing (HPC) systems can generate large amounts of data (in the scale of terabytes or petabytes) per run. When this huge amount of HPC data is processed by machine learning applications, the training overhead will be significant. Typically, the training process for a neural network can take several hours to complete, if not longer. When machine learning is applied to HPC scientific data, the training time can take several days or even weeks. Transfer learning, an optimization usually used to save training time or achieve better performance, has potential for reducing this large training overhead. In this paper, we apply transfer learning to a machine learning HPC application. We find that transfer learning can reduce training time without, in most cases, significantly increasing the error. This indicates transfer learning can be very useful for working with HPC datasets in machine learning applications.
Identifier
85073220149 (Scopus)
ISBN
[9781728144092]
Publication Title
2019 IEEE International Conference on Networking Architecture and Storage Nas 2019 Proceedings
External Full Text Location
https://doi.org/10.1109/NAS.2019.8834723
Grant
CCF-1718297
Fund Ref
National Science Foundation
Recommended Citation
Liu, Tong; Alibhai, Shakeel; Wang, Jinzhen; Liu, Qing; He, Xubin; and Wu, Chentao, "Exploring Transfer Learning to Reduce Training Overhead of HPC Data in Machine Learning" (2019). Faculty Publications. 7432.
https://digitalcommons.njit.edu/fac_pubs/7432
