Document Type
Conference Paper
Publication Date
2026
DOI
10.1145/3806645.3816236
Publication Title
HPDC '26: Proceedings of the 2026 35th International Symposium on High-Performance Parallel and Distributed Computing
Pages
737-744
Conference Name
HPDC '26: 35th International Symposium on High-Performance Parallel and Distributed Computing, July 13-16, 2026, Cleveland, Ohio, U.S.A.
Abstract
Asynchronous iterative methods tolerate straggling processors by allowing workers to proceed with stale data, but at a cost: the iterates become inconsistent, potentially degrading convergence. We investigate whether convergence accelerators such as Anderson acceleration compensate for this degradation. We experimentally study three fixed-point iterations: the Jacobi method for sparse linear systems, value iteration for the Bellman equation, and the Hartree–Fock self-consistent field (SCF) iteration. The experiments are conducted using a high-performance execution framework, Ray, which abstracts the complexity of distributed systems and enables code parallelization and fault injection with minimal changes.
We establish two main results. First, straggler tolerance is universal: asynchronous execution provides wall-clock speedups of 2.9 × (Jacobi), 7.7 × (VI), and 16.9 × (SCF) over synchronous execution with a 100 ms-delayed worker, independent of whether acceleration is used. Second, we identify two distinct mechanisms, iterate-level corruption (Jacobi) and evaluation-level perturbation (VI, SCF), and show that Anderson must be applied at every worker return under the former but tolerates a range of extrapolation intervals under the latter.
Rights
© 2026 Copyright held by the owner/authors.
This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.
Original Publication Citation
Coleman, E., & Sosonkina, M. (2026). Fault tolerance of accelerated asynchronous fixed-point iterations on flexible computing infrastructure. In Sanmukh Kuppannagari, Mehmet Koyuturk, Alfredo Goldman, & Dimitrios S. Nikolopoulos (Eds.), HPDC '26: Proceedings of the 2026 35th International Symposium on High-Performance Parallel and Distributed Computing (pp. 737-744). Association for Computing Machinery. https://doi.org/10.1145/3806645.3816236
Repository Citation
Coleman, Evan and Sosonkina, Masha, "Fault Tolerance of Accelerated Asynchronous Fixed-Point Iterations on Flexible Computing Infrastructure" (2026). Electrical & Computer Engineering Faculty Publications. 608.
https://digitalcommons.odu.edu/ece_fac_pubs/608