Straggler-Resilient Differentially Private Decentralized Learning

Document Type

Article

Publication Date

1-1-2024

Abstract

We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency - comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme, is identified and empirically validated for logistic regression on a real-world dataset and for image classification using the MNIST and CIFAR-10 datasets.

Identifier

85194060373 (Scopus)

Publication Title

IEEE Journal on Selected Areas in Information Theory

External Full Text Location

https://doi.org/10.1109/JSAIT.2024.3400995

e-ISSN

26418770

First Page

407

Last Page

423

Volume

5

This document is currently not available here.

Share

COinS