Header menu link for other important links
X
Coded sequential matrix multiplication for straggler mitigation
, Erfan Hosseini, Ashish Khisti
Published in Neural information processing systems foundation
2020
Volume: 2020-December
   
Abstract
In this work, we consider a sequence of J matrix multiplication jobs which needs to be distributed by a master across multiple worker nodes. For i ? {1, 2, . . ., J}, job-i begins in round-i and has to be completed by round-(i + T). Previous works consider only the special case of T = 0 and focus on coding across workers. We propose here two schemes with T > 0, which feature coding across workers as well as the dimension of time. Our first scheme is a modification of the polynomial coding scheme introduced by Yu et al. and places no assumptions on the straggler model. Exploitation of the temporal dimension helps the scheme handle a larger set of straggler patterns than the polynomial coding scheme, for a given computational load per worker per round. The second scheme assumes a particular straggler model to further improve performance (in terms of encoding/decoding complexity). We develop theoretical results establishing (i) optimality of our proposed schemes for a certain class of straggler patterns and (ii) improved performance for the case of i.i.d. stragglers. These are further validated by experiments, where we implement our schemes to train neural networks. © 2020 Neural information processing systems foundation. All rights reserved.
About the journal
JournalAdvances in Neural Information Processing Systems
PublisherNeural information processing systems foundation
ISSN10495258