Reinforcement Learning algorithms for IoT communications over uncoordinated access channels

Kurzfassung

With the Internet of Things (IoT), we intend any system of multiple devices able to collect and transfer information with any IT device, like smartphones or computers, through the Internet connectivity. This thesis focuses on one of its applications which is massive Remote Monitoring Systems, where thousands of devices send time stamped updates over a wireless channel to a common receiver. We focus on the underlying - link layer - communication protocol of such scenario. A simple yet powerful random access protocol, the slotted-ALOHA, is used as benchmark model to organize the channel access strategy. The overall objective to keep an up-to-date perception at the receiver in terms of the freshness of data, described by the Age of Information (AoI), a novel metric which tracks the timeliness of a specific process with respect to the receiver. Under the paradigm of Reinforcement Learning, the proposed algorithm - AoI-Q-ALOHA method - tackles a Multi-Agent setting in a decentralized fashion, where each user is independently learning an AoI-based channel access policy. To let the agents adapt to a shared channel, a binary success/collision feedback distributed by the receiver is used as the main global incentive during the learning process. Without any assumption on the channel population, team behaviour is encouraged through a tailored individual reward designation, based on the current value of AoI. We compare the performance in terms of the mean channel AoI, Throughput and Fairness index, to the standard slotted-ALOHA protocol, and to the threshold-ALOHA policy, a benchmark protocol that resorts to a central optimization of the access parameters. Interesting insights on the distributed setup are derived, as well an exhaustive surveyon the properties and robustness of the algorithm.