Deep Reinforcement Learning-Based Computing-Aware Routing Optimization Method for Low Earth Orbit Satellite Networks

CAI Jiahui; ZHOU Jiaen; ZHAO Yafei; PENG Mugen

doi:10.11959/j.issn.1000-0801.DXKX250758

您当前的位置：

首页 >

文章列表页 >

Deep Reinforcement Learning-Based Computing-Aware Routing Optimization Method for Low Earth Orbit Satellite Networks

更新时间：2026-05-19

- Deep Reinforcement Learning-Based Computing-Aware Routing Optimization Method for Low Earth Orbit Satellite Networks
- Telecommunications Science (2026)
- 作者机构：
  
  北京邮电大学网络与交换技术全国重点实验室，北京，100876
- 作者简介：
- 基金信息：
  
  Stability Support Funding Program of National Defense Key Laboratory(WDZC20265290408)
- DOI：10.11959/j.issn.1000-0801.DXKX250758
  CLC： TN927
- Received：30 December 2025，
  
  Revised：2026-05-07，
  
  Accepted：18 May 2026，
- 稿件说明：
移动端阅览
CAI Jiahui, ZHOU Jiaen, ZHAO Yafei, et al. Deep Reinforcement Learning-Based Computing-Aware Routing Optimization Method for Low Earth Orbit Satellite Networks[J/OL]. Telecommunications Science, 2026.
DOI：

CAI Jiahui, ZHOU Jiaen, ZHAO Yafei, et al. Deep Reinforcement Learning-Based Computing-Aware Routing Optimization Method for Low Earth Orbit Satellite Networks[J/OL]. Telecommunications Science, 2026. DOI： 10.11959/j.issn.1000-0801.DXKX250758.

摘要

针对低轨卫星网络中拓扑动态、链路时变及星上算力异构等挑战，本文提出了一种基于传算协同优化的双重深度Q网络（Computing–Transmission Double Deep Q-Network

CTDDQN）算力路由方法。该方法在软件定义网络架构下由控制器获取全局拓扑与资源信息，并在控制器侧进行集中式传算联合决策，构建以端到端时延最小化为目标的传输–计算联合优化框架，并结合K-hop前瞻机制实现路由与计算决策的协同。仿真结果表明，在不同星座规模、链路带宽与算力配置下，与多种基线算法相比，所提方法可使平均端到端时延降低18.9%以上，验证了该方法在高动态低轨卫星网络中的有效性与可扩展性。

Abstract

To address the computing-aware routing problem in Low-Earth-Orbit (LEO) satellite networks

which faces highly dynamic topology

time-varying inter-satellite links

and heterogeneous onboard computing capabilities

this paper proposes a Computing–Transmission Double Deep Q-Network (CTDDQN)-based computing-aware routing method. Under a Software-Defined Networking (SDN) architecture

the controller acquires global topology and resource information and performs centralized joint transmission–computing decision-making. Based on this

we formulate an integrated transmission-computing optimization framework that minimizes end-to-end delay

and further incorporate a K-hop lookahead mechanism to coordinate routing and onboard computing decisions. Simulation results under different constellation scales

link bandwidths

and onboard computing configurations show that

compared with multiple baseline methods

the proposed method reduces the average end-to-end delay by over 18.9%

demonstrating its effectiveness and scalability in highly dynamic LEO satellite networks.

关键词

Keywords

references

王志勤 , 杜滢 , 沈霞 , 等 . 面向6G典型场景的无线系统研究 [J ] . 中兴通讯技术 , 2024 , 30 ( 4 ): 65 - 68 .

WANG Z Q , DU Y , SHEN X , et al . Research on wireless systems for typical 6G scenarios [J ] . ZTE Technology Journal , 2024 , 30 ( 4 ): 65 - 68 .

赵亚飞 , 周家恩 , 王鑫洋 . 面向卫星通信的6G雾计算网络技术研究与展望 [J ] . 无线电通信技术 , 2023 , 49 ( 5 ): 834 - 841 .

ZHAO Y , ZHOU J E , WANG X Y , et al . Research and Prospect of 6G Fog Computing Network for Satellite Communication [J ] . Radio Communications Technology , 2023 , 49 ( 5 ): 834 - 841 .

李佳奇 , 陈全 , 杨磊 . 低轨巨型星座路由技术研究现状及展望 [J ] . 空间电子技术 , 2025 , 22 ( 2 ): 1 - 12 .

LI J Q , CHEN Q , YANG L . Routing technologies in LEO mega-constellations: A survey and outlook [J ] . Space Electronic Technology , 2025 , 22 ( 2 ): 1 - 12 .

TIRMIZI S B R , CHEN Y , LAKSHMINARAYANA S , et al . Hybrid Satellite–Terrestrial Networks toward 6G: Key Technologies and Open Issues [J ] . Sensors , 2022 , 22 ( 21 ): 8544 .

3GPP. System architecture for the 5G System (5GS): TS 23.501 [S ] . 2024 .

KODHELI O , LAGUNAS E , MATURO N , et al . Satellite communications in the new space era: A survey and future challenges [J ] . IEEE Communications Surveys & Tutorials , 2020 , 23 ( 1 ): 70 - 109 .

ZHU X , JIANG C . Integrated satellite-terrestrial networks toward 6G: Architectures, applications, and challenges [J ] . IEEE Internet of Things Journal , 2021 , 9 ( 1 ): 437 - 461 .

LIU C , FENG W , TAO X , et al . MEC-empowered non-terrestrial network for 6G wide-area time-sensitive internet of things [J ] . Engineering , 2022 , 8 : 96 - 107 .

WANG R , KISHK M A , ALOUINI M S . Stochastic geometry-based low latency routing in massive LEO satellite networks [J ] . IEEE Transactions on Aerospace and Electronic Systems , 2022 , 58 ( 5 ): 3881 - 3894 .

HAN Z , ZHAO G , XING Y , et al . Dynamic routing for software-defined LEO satellite networks based on ISL attributes [C ] // 2021 IEEE Global Communications Conference (GLOBECOM) . Madrid, Spain : IEEE , 2021 : 1 - 6 .

HAN Z , XU C , ZHAO G , et al . Time-varying topology model for dynamic routing in LEO satellite constellation networks [J ] . IEEE Transactions on Vehicular Technology , 2022 , 72 ( 3 ): 3440 - 3454 .

PAGE P S , BHARGAO K S , BAVISKAR H V , et al . Distributed probabilistic congestion control in LEO satellite networks [C ] // 2023 15th International Conference on COMmunication Systems & NETworkS (COMSNETS) . Bangalore, India : IEEE , 2023 : 335 - 339 .

SORET B , LEYVA-MAYORGA I , LOZANO-CUADRA F , et al . Q-learning for distributed routing in LEO satellite constellations [C ] // 2024 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) . Stockholm, Sweden : IEEE , 2024 : 208 - 213 .

徐晖 , 陈山枝 , 艾明 . 面向6G的星地融合网络架构 [J ] . 中兴通讯技术 , 2023 , 29 ( 5 ): 9 - 15 .

XU H , CHEN S Z , AI M . Architecture of satellite-terrestrial integrated networks for 6G [J ] . ZTE Technology Journal , 2023 , 29 ( 5 ): 9 - 15 .

CAO H , WANG H , WU T , et al . Task Offloading Strategy in Satellite Edge Computing Based on Matching Game [C ] // Proceedings of the 2023 12th International Conference on Networks, Communication and Computing . Osaka, Japan : ACM , 2023 : 91 - 98 .

LI Y , ZHU S , XIONG T , et al . Joint Task Offloading and Power Allocation for Satellite Edge Computing Networks [J ] . Sensors , 2025 , 25 ( 9 ): 2892 .

YANG J , SHAH A A , PEZAROS D . Priority-Aware Task Offloading in UAV-Assisted Satellite MEC Networks [C ] // 2025 IEEE International Conference on Communications Workshops (ICC Workshops) . Montreal, Canada : IEEE , 2025 : 640 - 645 .

WANG Y , FENG C , SUN J . Cost-Efficient Computation Offloading and Service Chain Caching in LEO Satellite Networks [J ] . arXiv preprint arXiv: 2311.07872 , 2023 .

LAN W , CHEN K , CAO J , et al . Security-sensitive task offloading in integrated satellite-terrestrial networks [J ] . IEEE Transactions on Mobile Computing , 2024 , 24 ( 3 ): 2220 - 2233 .

CAO J , ZHANG S , CHEN Q , et al . Computing-aware routing for leo satellite networks: A transmission and computation integration approach [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 12 ): 16607 - 16623 .

LI Y , ZHANG Q , YAO H , et al . Stigmergy and hierarchical learning for routing optimization in multi-domain collaborative satellite networks [J ] . IEEE Journal on Selected Areas in Communications , 2024 , 42 ( 5 ): 1188 - 1203 .

孔梦燕 . 基于深度强化学习的低轨卫星网络算力路由研究 [D ] . 北京 : 中国电子科技集团公司电子科学研究院 , 2025 .

KONG M Y . Computing-aware routing in LEO satellite networks based on deep reinforcement learning [D ] . Beijing : Electronic Science Research Institute of China Electronics Technology Group Corporation , 2025 .

ZHONG L , LI Y , GE M F , et al . Joint Task Offloading and Resource Allocation for LEO Satellite-Based Mobile Edge Computing Systems With Heterogeneous Task Demands [J ] . IEEE Transactions on Vehicular Technology , 2025 , 74 ( 7 ): 11337 - 11352 .

Rahman Z , Jobson D J , Woodell G A . Investigating the relationship between image enhancement and image compression in the context of the multi-scale retinex [J ] . Journal of Visual Communication and Image Representation , 2011 , 22 ( 3 ): 237 - 250 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning

Multi-service routing algorithm for satellite networks based on deep Q-network with long short-term memory

Research on heterogeneous task offloading methods based on CA3C in satellite-ground edge computing networks

A layered collaborative resource allocation algorithm for cell-free RAN aimed at efficient resource utilization

Research on a DSTM-based trusted mesh security mechanism for 6G distributed autonomous networks

Related Author

CHEN Bo

DONG Yongji

FENG Xu

HU Yuxiang

WEI Shuai

TIAN Le

YUE Qiqiang

Pan Chengsheng

Related Institution

Key Laboratory of Cyberspace Security, Ministry of Education

National Key Laboratory of Advanced Communication Networks

Information Engineering University

National Local Joint Engineering Research Center for Communication and Network Technology, Nanjing University of Posts and Telecommunications

College of Information Engineering, Dalian University

⁰