Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
Document Type
Conference Proceeding
Publication Date
8-14-2022
Abstract
Recent years have witnessed remarkable success achieved by graph neural networks (GNNs) in many real-world applications such as recommendation and drug discovery. Despite the success, oversmoothing has been identified as one of the key issues which limit the performance of deep GNNs. It indicates that the learned node representations are highly indistinguishable due to the stacked aggregators. In this paper, we propose a new perspective to look at the performance degradation of deep GNNs, i.e., feature overcorrelation. Through empirical and theoretical study on this matter, we demonstrate the existence of feature overcorrelation in deeper GNNs and reveal potential reasons leading to this issue. To reduce the feature correlation, we propose a general framework DeCorr which can encourage GNNs to encode less redundant information. Extensive experiments have demonstrated that DeCorr can help enable deeper GNNs and is complementary to existing techniques tackling the oversmoothing issue.
Identifier
85132171748 (Scopus)
ISBN
[9781450393850]
Publication Title
Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
External Full Text Location
https://doi.org/10.1145/3534678.3539445
First Page
709
Last Page
719
Recommended Citation
Jin, Wei; Liu, Xiaorui; Ma, Yao; Aggarwal, Charu; and Tang, Jiliang, "Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective" (2022). Faculty Publications. 2732.
https://digitalcommons.njit.edu/fac_pubs/2732
