MetaStream: Live Volumetric Content Capture, Creation, Delivery, and Rendering in Real Time
Document Type
Conference Proceeding
Publication Date
1-1-2023
Abstract
While recent work explored streaming volumetric content on-demand, there is little effort on live volumetric video streaming that bears the potential of bringing more exciting applications than its on-demand counterpart. To fill this critical gap, in this paper, we propose MetaStream, which is, to the best of our knowledge, the first practical live volumetric content capture, creation, delivery, and rendering system for immersive applications such as virtual, augmented, and mixed reality. To address the key challenge of the stringent latency requirement for processing and streaming a huge amount of 3D data, MetaStream integrates several innovations into a holistic system, including dynamic camera calibration, edge-assisted object segmentation, cross-camera redundant point removal, and foveated volumetric content rendering. We implement a prototype of MetaStream using commodity devices and extensively evaluate its performance. Our results demonstrate that MetaStream achieves low-latency live volumetric video streaming at close to 30 frames per second on WiFi networks. Compared to state-of-the-art systems, MetaStream reduces end-to-end latency by up to 31.7% while improving visual quality by up to 12.5%.
Identifier
85191429448 (Scopus)
ISBN
[9781450399906]
Publication Title
Proceedings of the Annual International Conference on Mobile Computing and Networking MOBICOM
External Full Text Location
https://doi.org/10.1145/3570361.3592530
ISSN
15435679
First Page
425
Last Page
439
Grant
2147821
Fund Ref
Recommended Citation
Guan, Yongjie; Hou, Xueyu; Wu, Nan; Han, Bo; and Han, Tao, "MetaStream: Live Volumetric Content Capture, Creation, Delivery, and Rendering in Real Time" (2023). Faculty Publications. 2081.
https://digitalcommons.njit.edu/fac_pubs/2081
