Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision Users
Document Type
Conference Proceeding
Publication Date
10-27-2024
Abstract
Virtual reality (VR), inherently reliant on spatial interaction, poses signifcant accessibility barriers for individuals who are blind or have low vision (BLV). Traditional audio descriptions (AD) typically provide a verbal explanation of visual elements in 2D or fat video media, facilitating access for BLV audiences but failing to convey the complex spatial information essential in VR. This shortfall is especially pronounced in musical performances, where understanding the spatial arrangement of the stage setup and movements of performers is crucial. To overcome these limitations, we have developed two AD approaches-Spatial AD for a dance performance and View-dependent AD for an instrumental performance-within VR-based 360° environments. Spatial AD employs spatial audio technology to align descriptions with corresponding visuals, dynamically adjusting to follow the visuals, such as the movements of performers in the dance performance. Meanwhile, View-dependent AD adapts descriptions based on the orientation of the VR headset, activating when particular visuals enter the central view of the camera, ensuring that the description aligns with the user's attention directed to a particular location within the VR environment. These methods are designed as enhancements to traditional AD, aiming to improve spatial orientation and immersive experiences for BLV audiences. This demonstration showcases the potential of these AD approaches to improve interaction and engagement, furthering the development of inclusive virtual environments.
Identifier
85211458577 (Scopus)
ISBN
[9798400706776]
Publication Title
ASSETS 2024 - Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility
External Full Text Location
https://doi.org/10.1145/3663548.3688492
Recommended Citation
Dang, Khang and Lee, Sooyeon, "Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision Users" (2024). Faculty Publications. 128.
https://digitalcommons.njit.edu/fac_pubs/128
