A Momentum-Accelerated Hessian-Vector-Based Latent Factor Analysis Model

Document Type

Article

Publication Date

3-1-2023

Abstract

Service-oriented applications commonly involve high-dimensional and sparse (HiDS) interactions among users and service-related entities, e.g., user-item interactions from a personalized recommendation services system. How to perform precise and efficient representation learning on such HiDS interactions data is a hot yet thorny issue. An efficient approach to it is latent factor analysis (LFA), which commonly depends on large-scale non-convex optimization. Hence, it is vital to implement an LFA model able to approximate second-order stationary points efficiently for enhancing its representation learning ability. However, existing second-order LFA models suffer from high computational cost, which significantly reduces its practicability. To address this issue, this paper presents a Momentum-accelerated Hessian-vector algorithm (MH) for precise and efficient LFA on HiDS data. Its main ideas are two-fold: a) adopting the principle of a Hessian-vector-product-based method to utilize the second-order information without manipulating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning scheme for accelerating its convergence rate to a stationary point. Experimental results on nine industrial datasets demonstrate that compared with state-of-the-art LFA models, an MH-based LFA model achieves gains in both accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with the algorithms, e.g., a second-order algorithm, which implicitly rely on gradients.

Identifier

85131756449 (Scopus)

Publication Title

IEEE Transactions on Services Computing

External Full Text Location

https://doi.org/10.1109/TSC.2022.3177316

e-ISSN

19391374

First Page

830

Last Page

844

Issue

2

Volume

16

Grant

CAAIXSJLJJ-2021-035A

Fund Ref

National Natural Science Foundation of China

This document is currently not available here.

Share

COinS