Age-of-Gradient Updates for Federated Learning over Random Access Channels

Kurzfassung

This paper addresses the problem of federated learning (FL) over a random access channel. In this setting, a group of remote users train a centralized deep neural network model using stochastic gradient descent with support from a parameter server. Model updates are transmitted using a contention-based slotted ALOHA protocol. A natural tension arises: while learning accelerates as more users transmit over the channel, increased transmissions lead to a higher likelihood of packet collisions. To address this trade-off, we propose the Age-of-Gradient (AoG) policy, which optimizes user participation and communication efficiency. AoG integrates gradient sparsification, dynamic transmission probabilities based on gradient freshness, and error feedback to mitigate the effects of packet collisions and gradient compression. Inspired by the age of information concept in communication theory, AoG quantifies the freshness of local gradients to guide transmission decisions. Numerical simulations demonstrate the superior performance of AoG compared to baseline policies, showcasing its scalability and effectiveness in resource-constrained FL scenarios.