Reinforcement learning-aided NOMA random access: an AoI-based timeliness perspective

Abstract

Age-of-Information (AoI) is a critical metric for assessing the freshness of data in Internet of Things (IoT) networks, particularly in non-orthogonal multiple access (NOMA) networks. This study investigates the AoI performance in a NOMA network, where multiple IoT devices transmit data to a common gateway in a grant-free, random manner. Specifically, we consider a framed setup with multiple time slots and employ the Q-learning algorithm to define, in a distributed manner, the time slots and power levels at which each IoT device transmits within a frame. In the proposed AoI-QL-NOMA scheme, the Q-learning reward is adapted to minimize the average AoI of the network, requiring only a single feedback bit per time slot on a per-frame basis. Our results show that AoI-QL-NOMA significantly improves AoI performance compared to several recently proposed schemes, without significantly reducing network throughput.