A DDQN-based resource management method for edge computing fusion network

DONG Yuchi; YAN Yaqi; RAN Pei; WANG Dong; ZHANG Kuo; ZHANG Wenlong

doi:10.11959/j.issn.1000-0801.2025137

您当前的位置：

首页 >

文章列表页 >

A DDQN-based resource management method for edge computing fusion network

Engineering and Application | 更新时间：2025-09-04

- A DDQN-based resource management method for edge computing fusion network
- Telecommunications Science Vol. 41, Issue 8, Pages: 197-206(2025)
- 作者机构：
  
  中国铁塔股份有限公司，北京 100080
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2025137
  CLC： TP393
- Received：06 November 2024，
  
  Revised：2025-05-14，
  
  Published：20 August 2025
- 稿件说明：
移动端阅览
董玉池,闫亚旗,冉沛等.基于DDQN的边缘算力融合网络资源管理[J].电信科学,2025,41(08):197-206.

DONG Yuchi,YAN Yaqi,RAN Pei,et al.A DDQN-based resource management method for edge computing fusion network[J].Telecommunications Science,2025,41(08):197-206.
董玉池,闫亚旗,冉沛等.基于DDQN的边缘算力融合网络资源管理[J].电信科学,2025,41(08):197-206. DOI： 10.11959/j.issn.1000-0801.2025137.

DONG Yuchi,YAN Yaqi,RAN Pei,et al.A DDQN-based resource management method for edge computing fusion network[J].Telecommunications Science,2025,41(08):197-206. DOI： 10.11959/j.issn.1000-0801.2025137.

摘要

边缘算力融合网络将算力下沉至近用户侧，通过分布式边缘算力节点相互协同以在本地完成计算任务，显著降低云端负担和传输时延。然而，随着用户接入密度提高和场景复杂化，如何动态优化网络资源以协同应对多样化服务需求和大规模数据处理任务成为重大挑战。因此，提出了一种基于双深度Q网络（double deep Q network，DDQN）边缘算力融合网络资源管理方法，结合虚拟网络嵌入（virtual network embedding，VNE）技术，建立了以长期资源收益成本比最大化为目标的多约束优化模型。通过DDQN架构的在线学习能力，利用环境交互反馈实现动态优化决策。仿真实验表明，该方法在虚拟网络请求（virtual network request，VNR）接受成功率、长期嵌入收益和长期嵌入收益成本比3个指标上，较现有方法分别提升了13.3%、25.7%和8.5%。

Abstract

The edge computing fusion network sinks the computing resources to the user side and completes the computing tasks locally through the coordination of distributed edge computing nodes

which significantly reduces the cloud burden and transmission delay. However

with the increase of user access density and the complexity of scenarios

how to dynamically optimize network resources to cope with diversified service demands and large-scale data processing tasks has become a major challenge. Therefore

a resource management method for edge computing networks based on double deep Q network (DDQN) was proposed. Integrating the virtual network embedding (VNE) technology

the proposed method formulated a multi-constraint optimization model to maximize the long-term embedding revenue-to-cost ratio. By leveraging the online learning capabilities of the DDQN framework

it enabled dynamic decision-making through interaction and feedback with the environment. Simulation results demonstrate that the proposed method achieves average improvements of 13.3%

25.7% and 8.5% of virtual network request (VNR) acceptance rate

long-term embedding revenue

and long-term revenue-to-cost ratio

respectively

compared with the existing methods.

关键词

Keywords

references

刘云毅 , 张建敏 , 冯晓丽 , 等 . 5G MEC系统安全能力部署方案 [J ] . 电信科学 , 2022 , 38 ( 11 ): 143 - 152 .

LIU Y Y , ZHANG J M , FENG X L , et al . 5G MEC system security capability deployment scheme [J ] . Telecommunications Science , 2022 , 38 ( 11 ): 143 - 152 .

WANG S , SHENG H , ZHANG Y , et al . Blockchain-empowered distributed multicamera multitarget tracking in edge computing [J ] . IEEE Transactions on Industrial Informatics , 2024 , 20 ( 1 ): 369 - 379 .

雷波 , 刘增义 , 王旭亮 , 等 . 基于云、网、边融合的边缘计算新方案: 算力网络 [J ] . 电信科学 , 2019 , 35 ( 9 ): 44 - 51 .

LEI B , LIU Z Y , WANG X L , et al . Computing network: a new multi-access edge computing [J ] . Telecommunications Science , 2019 , 35 ( 9 ): 44 - 51 .

谢人超 , 廉晓飞 , 贾庆民 , 等 . 移动边缘计算卸载技术综述 [J ] . 通信学报 , 2018 , 39 ( 11 ): 138 - 155 .

XIE R C , LIAN X F , JIA Q M , et al . Survey on computation offloading in mobile edge computing [J ] . Journal on Communications , 2018 , 39 ( 11 ): 138 - 155 .

XIAO A L , WU S , OU Y K , et al . QoE-fairness-aware bandwidth allocation design for MEC-assisted ABR video transmission [J ] . IEEE Transactions on Network and Service Management , 2024 , PP(99): 1.

雷波 , 赵倩颖 , 赵慧玲 . 边缘计算与算力网络综述 [J ] . 中兴通讯技术 , 2021 , 27 ( 3 ): 3 - 6 .

LEI B , ZHAO Q Y , ZHAO H L . Overview of edge computing and computing power network [J ] . ZTE Technology Journal , 2021 , 27 ( 3 ): 3 - 6 .

冯川 , 张旭 , 马天纯 , 等 . 节点链路同时映射的多类型服务资源编排算法 [J ] . 通信学报 , 2024 , 45 ( 4 ): 54 - 64 .

FENG C , ZHANG X , MA T C , et al . Node-link simultaneous mapping based multi-type service resource orchestration algorithm [J ] . Journal on Communications , 2024 , 45 ( 4 ): 54 - 64 .

ZHANG P Y , CHEN N , KUMAR N , et al . Energy allocation for vehicle-to-grid settings: a low-cost proposal combining DRL and VNE [J ] . IEEE Transactions on Sustainable Computing , 2024 , 9 ( 1 ): 75 - 87 .

WU S , CHEN N , WEN G H , et al . Virtual network embedding for task offloading in IIoT: a DRL-assisted federated learning scheme [J ] . IEEE Transactions on Industrial Informatics , 2024 , 20 ( 4 ): 6814 - 6824 .

CHENG X , SU S , ZHANG Z B , et al . Virtual network embedding through topology-aware node ranking [J ] . ACM SIGCOMM Computer Communication Review , 2011 , 41 ( 2 ): 38 - 47 .

MELO M , SARGENTO S , KILLAT U , et al . Optimal virtual network embedding: node-link formulation [J ] . IEEE Transactions on Network and Service Management , 2013 , 10 ( 4 ): 356 - 368 .

WU S , CHEN N , XIAO A L , et al . AI-empowered virtual network embedding: a comprehensive survey [J ] . IEEE Communications Surveys & Tutorials , 2024 ( 99 ): 1 .

丁世飞 , 杜威 , 张健 , 等 . 多智能体深度强化学习研究进展 [J ] . 计算机学报 , 2024 , 47 ( 7 ): 1547 - 1567 .

DING S F , DU W , ZHANG J , et al . Research progress of multi-agent deep reinforcement learning [J ] . Chinese Journal of Computers , 2024 , 47 ( 7 ): 1547 - 1567 .

YAO H P , MA S H , WANG J J , et al . A continuous-decision virtual network embedding scheme relying on reinforcement learning [J ] . IEEE Transactions on Network and Service Management , 2020 , 17 ( 2 ): 864 - 875 .

YAN Z X , GE J G , WU Y L , et al . Automatic virtual network embedding: a deep reinforcement learning approach with graph convolutional networks [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 6 ): 1040 - 1057 .

ZHANG P Y , CHEN N , LI S B , et al . Multi-domain virtual network embedding algorithm based on horizontal federated learning [J ] . IEEE Transactions on Information Forensics and Security , 2023 , 18 : 3363 - 3375 .

ZHANG P Y , CHEN N , SHEN S G , et al . AI-enabled space-air-ground integrated networks: management and optimization [J ] . IEEE Network , 2024 , 38 ( 2 ): 186 - 192 .

CHEN N , ZHANG P Y , KUMAR N , et al . Spectral graph theory-based virtual network embedding for vehicular fog computing: a deep reinforcement learning architecture [J ] . Knowledge- Based Systems , 2022 , 257 : 109931 .

ZHANG P Y , CHEN N , XU G J , et al . Multi-target-aware dynamic resource scheduling for cloud-fog-edge multi-tier computing network [J ] . IEEE Transactions on Intelligent Transportation Systems , 2024 , 25 ( 5 ): 3885 - 3897 .

LIM H K , ULLAH I , KIM J B , et al . Virtual network embedding based on hierarchical cooperative multiagent reinforcement learning [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 5 ): 8552 - 8568 .

余竞航 , 赵一辰 , 宋浒 . 基于强化学习的边缘计算智能电网资源调度算法 [J ] . 电信科学 , 2024 , 40 ( 1 ): 115 - 122 .

YU J H , ZHAO Y C , SONG H . Edge computing smart grid resource scheduling algorithm based on reinforcement learning [J ] . Telecommunications Science , 2024 , 40 ( 1 ): 115 - 122 .

VAN HASSELT H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2016 , 30 ( 1 ): 1 - 7 .

XIAO A L , CHEN N , WU S , et al . DNFS-VNE: deep neuro fuzzy system driven virtual network embedding [J ] . IEEE Internet of Things Journal , 2024 ( 99 ): 1 .

Views

358

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on deterministic networking technology for space-ground integrated

Resource management in sliced networks with mobile edge computing for B5G and 6G

Computation offloading and resource management scheme in mobile edge computing

Intelligent interference management in 5G wireless networks

An SLA-based resource scheduling method in hybrid cloud environment

Related Author

Xia ZHU

Guoxin LIANG

Jiaen ZHOU

Yafei ZHAO

Mugen PENG

Yuman ZHANG

Jihua ZHOU

Yuanyuan WANG

Related Institution

State Grid Siji Location based Service Co., Ltd.

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

Chongqing Jinmei Communication Co., Ltd.

University of Chinese Academy of Sciences

Institute of Computing Technology, Chinese Academy of Sciences

⁰