Spark-based large-scale matrix inversion for big data processing

Document Type

Conference Proceeding

Publication Date

9-6-2016

Abstract

Matrix inversion is a fundamental operation to solve linear equations for many computational applications. However, it is a challenging task to invert large-scale matrices of extremely high order (several thousands), which are common in most of web-scale systems like social networks and recommendation systems. In this paper, we present a LU decomposition based block-recursive algorithm for large-scale matrix inversion, and its well-designed implementation with optimized data structure, reduction of space complexity and effective matrix multiplication on the Spark parallel computing platform. The experimental evaluation results show that the proposed algorithm is efficient to invert large-scale matrices on a cluster composed of commodity servers and scalable to invert even larger matrices. The proposed algorithm and implementation will be a solid base to build a high-performance linear algebra library on Spark for big data processing.

Identifier

84988807075 (Scopus)

ISBN

[9781467399555]

Publication Title

Proceedings IEEE INFOCOM

External Full Text Location

https://doi.org/10.1109/INFCOMW.2016.7562171

ISSN

0743166X

First Page

718

Last Page

723

Volume

2016-September

This document is currently not available here.

Share

COinS