Enhancing Routing in Satellite Constellations with Software-Defined Networking and Reinforcement Learning

Abstract

The emergence of interconnected satellite mega-constellations has highlighted the need for optimized routing and network management solutions to enable reliable broadband connectivity. Given the intended integration into terrestrial networks in 5G and beyond architectures, the demand for efficient routing to support more users and higher traffic volumes becomes increasingly important. To achieve higher system throughput while maintaining quality of service requirements and ensuring constellation autonomy, tailored routing solutions are required. Dynamic network characteristics as well as on-board limitations, such as the processing power, have to be accommodated in the design. In this thesis, novel routing protocols and traffic engineering algorithms are proposed to address these challenges. A load-balancing in-orbit solution based on distributed software-defined networking is presented to enable flexible and adaptive routing strategies. The performance of the protocol is evaluated in different scenarios using a system-level simulator specifically developed to enable fine-grained analyses of performance indicators. The viability and validity of the approach is underlined by a demonstration in a testbed based on space-qualified hardware. In addition, traffic engineering algorithms can be employed to further improve load-balancing performance. By proactively avoiding potential bottlenecks, efficient routing configurations can be found which enable higher throughput. This optimization, described by the multi-commodity flow problem, is NP-hard. Given the limited on-board processing power, complex rule-based approaches are impractical. Conventional optimization techniques are not well-suited due to their computational complexity and limited adaptability. Therefore, deep reinforcement learning approaches are derived from first principles for the given scenario. Underlying traffic patterns are learned directly from the environment. A novel soft actor-critic method, which operates on sets of candidate paths, is proposed. Its learned traffic engineering capabilities surpass the performance of comparable approaches. Overall, the proposed routing protocol and data-driven load-balancing methods extend the state of the art and provide valuable insights into network optimization in satellite constellations.