Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning

YUE Qiqiang; TIAN Le; WEI Shuai; HU Yuxiang; FENG Xu; DONG Yongji; CHEN Bo

doi:10.11959/j.issn.1000-0801.2025171

您当前的位置：

首页 >

文章列表页 >

Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning

Intelligent Computing Network | 更新时间：2025-09-04

- Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning
- Telecommunications Science Vol. 41, Issue 8, Pages: 33-50(2025)
- 作者机构：
  
  1.信息工程大学信息技术研究所，河南郑州 450002
  2.先进通信网全国重点实验室，河南郑州 450002
  3.网络空间安全教育部重点实验室，河南郑州 450002
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2024YFB2906704);The Major Special Projects in Henan Province(22110021090003)
- DOI：10.11959/j.issn.1000-0801.2025171
  CLC： TP393
- Received：31 March 2025，
  
  Revised：2025-07-07，
  
  Accepted：08 July 2025，
  
  Published：20 August 2025
- 稿件说明：
移动端阅览
越奇强,田乐,魏帅等.基于深度强化学习的算网协同动态路由调度算法[J].电信科学,2025,41(08):33-50.

YUE Qiqiang,TIAN Le,WEI Shuai,et al.Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning[J].Telecommunications Science,2025,41(08):33-50.
越奇强,田乐,魏帅等.基于深度强化学习的算网协同动态路由调度算法[J].电信科学,2025,41(08):33-50. DOI： 10.11959/j.issn.1000-0801.2025171.

YUE Qiqiang,TIAN Le,WEI Shuai,et al.Computing-network collaborative dynamic routing and scheduling algorithm based on deep reinforcement learning[J].Telecommunications Science,2025,41(08):33-50. DOI： 10.11959/j.issn.1000-0801.2025171.

摘要

针对算力网络中算网资源协同不足、任务需求适配性差的问题，将算力路由问题建模为序列决策问题，提出了基于深度强化学习的算网协同动态路由调度算法。该算法借鉴混合专家模型思想，针对时延敏感型、普通型以及计算密集型3类典型场景，设计了基于编码器-解码器结构的差异化专家网络进行专项优化，并通过动作屏蔽机制约束路由选择空间，实现高效的逐跳决策，输出包含最优计算节点的路径。仿真实验结果表明，相较于其他路由调度算法，该算法在服务成功率上提升约17%，降低了端到端时延，优化了节点间的负载均衡度，展现出良好的网络拓扑适应性，能够有效满足多样化计算任务的差异化需求。

Abstract

To address the issues of insufficient collaboration among computing resources and poor adaptability to task requirements in computing power networks

the computing power routing problem was modeled as a sequential decision problem. A deep reinforcement learning-based computing-aware routing algorithm was proposed for dynamic routing scheduling of computing network collaboration. The idea of hybrid expert models was drawn on and a differentiated expert network was designed based on an encoder-decoder structure for specialized optimization in three typical scenarios: delay-sensitive

ordinary

and computationally intensive. The routing selection space was constrained through an action masking mechanism to achieve efficient hop-by-hop decision-making and output a path containing the optimal computing node. The simulation experiment results show that compared with other routing scheduling algorithms

the proposed algorithm improves service success rate by about 17%

reduces end-to-end latency

optimizes load balancing between nodes

demonstrates good network topology adaptability

and can effectively meet the differentiated needs of diverse computing tasks.

关键词

Keywords

references

SUN Y K , LEI B , LIU J L , et al . Computing power network: a survey [J ] . China Communications , 2024 , 21 ( 9 ): 109 - 145 .

姚惠娟 , 陆璐 , 段晓东 . 算力感知网络架构与关键技术 [J ] . 中兴通讯技术 , 2021 , 27 ( 3 ): 7 - 11 .

YAO H J , LU L , DUAN X D . Computing power perception network architecture and key technologies [J ] . ZTE Technology Journal , 2021 , 27 ( 3 ): 7 - 11 .

庞冉 , 易昕昕 , 辛亮 , 等 . 算力网络路由调度技术研究 [J ] . 电信科学 , 2023 , 39 ( 8 ): 149 - 156 .

PANG R , YI X X , XIN L , et al . Research on routing and scheduling technology in computing power networks [J ] . Telecommunications Science , 2023 , 39 ( 8 ): 149 - 156 .

GONG X M , REN S Y , WANG C J , et al . Research on computing resource measurement and routing methods in software defined computing first network [J ] . Sensors , 2024 , 24 ( 4 ): 1086 .

雷波 , 刘增义 , 王旭亮 , 等 . 基于云、网、边融合的边缘计算新方案: 算力网络 [J ] . 电信科学 , 2019 , 35 ( 9 ): 44 - 51 .

LEI B , LIU Z Y , WANG X L , et al . New edge computing scheme based on cloud, network and edge integration: computational network [J ] . Telecommunications Science , 2019 , 35 ( 9 ): 44 - 51 .

杨烨 . 基于算网一体化演进的算力网络技术研究 [J ] . 现代传输 , 2022 , ( 4 ): 45 - 48 .

YANG Y . Research on computing power network technology based on the evolution of computing network integration [J ] . Modern Transmission , 2022 , ( 4 ): 45 - 48 .

贾庆民 , 丁瑞 , 刘辉 , 等 . 算力网络研究进展综述 [J ] . 网络与信息安全学报 , 2021 , 7 ( 5 ): 1 - 12 .

JIA Q M , DING R , LIU H , et al . A review of the research progress of computing power networks [J ] . Journal of Network and Information Security , 2021 , 7 ( 5 ): 1 - 12 .

段晓东 , 姚惠娟 , 付月霞 , 等 . 面向算网一体化演进的算力网络技术 [J ] . 电信科学 , 2021 , 37 ( 10 ): 76 - 85 .

DUAN X D , YAO H J , FU Y X , et al . Computing power network technology for the evolution of integrated computing network [J ] . Telecommunications Science , 2021 , 37 ( 10 ): 76 - 85 .

任杰 , 王洪超 , 王钦定 , 等 . 服务感知网络 [J ] . 电子学报 , 2025 , 53 ( 2 ): 371 - 384 .

REN J , WANG H C , WANG Q D , et al . Service aware network [J ] Acta Electronica Sinica , 2025 , 53 ( 2 ): 371 - 384 .

魏汝翔 , 刘琦 , 赵广 , 等 . 东数西算下面向业务的路由策略分析与探索 [J ] . 中兴通讯技术 , 2023 , 29 ( 4 ): 14 - 18 .

WEI R X , LIU Q , ZHAO G , et al . Analysis and exploration of business-oriented routing strategy under eastern data and western computing [J ] . ZTE Technology Journal , 2023 , 29 ( 4 ): 14 - 18 .

孙钰坤 , 张兴 , 雷波 . 边缘算力网络中智能算力感知路由分配策略研究 [J ] . 无线电通信技术 , 2022 , 48 ( 1 ): 60 - 67 .

SUN Y K , ZHANG X , LEI B . Research on intelligent computing aware routing allocation strategy in edge computing networks [J ] . Radio Communications Technology , 2022 , 48 ( 1 ): 60 - 67 .

金天骄 , 栗蔚 . 基于算力网络的大数据计算资源智能调度分配方法 [J ] . 数据与计算发展前沿 , 2022 , 4 ( 6 ): 29 - 37 .

JIN T J , LI W . Intelligent scheduling and allocation method for big data computing resources based on computing power network [J ] . Frontiers of Data and Computing Development , 2022 , 4 ( 6 ): 29 - 37 .

GUO L J , GUO F X , PENG M G . A novel routing method for dynamic control in distributed computing power networks [J ] . Digital Communications and Networks , 2024 , 10 ( 6 ): 1644 - 1652 .

ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M , et al . Deep reinforcement learning: a brief survey [J ] . IEEE Signal Processing Magazine , 2017 , 34 ( 6 ): 26 - 38 .

FRANCOIS-LAVET V , HENDERSON P , ISLAM R , et al . An introduction to deep reinforcement learning [J ] . Foundations and Trends® in Machine Learning , 2018 , 11 ( 3-4 ): 219 - 354 .

MOUSAVI S S , SCHUKAT M , HOWLEY E . Deep reinforcement learning: an overview [C ] // Proceedings of SAI Intelligent Systems Conference . Cham : Springer International Publishing , 2016 : 426 - 440 .

TANG F X , MAO B M , FADLULLAH Z M , et al . ST-DeLTA: a novel spatial-temporal value network aided deep learning based intelligent network traffic control system [J ] . IEEE Transactions on Sustainable Computing , 2019 , 5 ( 4 ): 568 - 580 .

张维庭 , 孙呈蕙 , 王洪超 , 等 . 算网资源智能适配与融合调度方法 [J ] . 电信科学 , 2023 , 39 ( 9 ): 12 - 20 .

ZHANG W T , SUN C H , WANG H C , et al . Intelligent adaptation and fusion scheduling method for computing network resources [J ] . Telecommunications Science , 2023 , 39 ( 9 ): 12 - 20 .

孙国玮 , 许方敏 , 朱瑾瑜 , 等 . 算力网络中的确定性调度与路由联合智能优化方案 [J ] . 北京邮电大学学报 , 2023 , 46 ( 2 ): 9 .

SUN G W , XU F M , ZHU J Y , et al . Deterministic scheduling and routing joint intelligent optimization scheme in computing power networks [J ] . Journal of Beijing University of Posts and Telecommunications , 2023 , 46 ( 2 ): 9 .

XIE Y P , HUANG X Y , LI J C , et al . Computing power network: multi-objective optimization-based routing [J ] . Sensors , 2023 , 23 ( 15 ): 6702 .

赵丹丹 , 黄德根 , 孟佳娜 , 等 . 基于BERT-GRU-ATT模型的中文实体关系分类 [J ] . 计算机科学 , 2022 , 49 ( 6 ): 319 - 325 .

ZHAO D D , HUANG D G , MENG J N , et al . Chinese entity relationship classification based on BERT-GRU-ATT model [J ] . Computer Science , 2022 , 49 ( 6 ): 319 - 325 .

后同佳 , 周良 . 基于双向GRU神经网络和注意力机制的中文船舶故障关系抽取方法 [J ] . 计算机科学 , 2021 , 48 ( S2 ): 154 - 158 .

HOU T J , ZHOU L . Chinese ship fault relation extraction method based on bidirectional GRU neural network and attention mechanism [J ] . Computer Science , 2021 , 48 ( S2 ): 154 - 158 .

HE X , ZHAO W L , GAO Z J , et al . Short-term load forecasting by GRU neural network and DDPG algorithm for adaptive optimization of hyperparameters [J ] . Electric Power Systems Research , 2025 , 238 : 111119 .

Views

471

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on heterogeneous task offloading methods based on CA3C in satellite-ground edge computing networks

A layered collaborative resource allocation algorithm for cell-free RAN aimed at efficient resource utilization

Research on optimization of data transmission strategies based on deep reinforcement learning

Heterogeneous computing network resource management based on virtual network embedding

Related Author

Pan Chengsheng

Wang Lin

Jin Xiaoyong

Liu Zhiguo

ZHANG Mingxuan

WANG Boye

XIA Weiwei

MIAO Weiwei

Related Institution

School of Electronics and Information Engineering, Nanjing University of Information Science and Technology

School of Environmental and Chemical Engineering, Dalian University

Key Laboratory of Communication and Network, Dalian University

School of Information Engineering, Dalian University

National Mobile Communication Research Laboratory of Southeast University

⁰