基于DDQN的边缘算力融合网络资源管理

董玉池; 闫亚旗; 冉沛; 王东; 张阔; 张文龙

doi:10.11959/j.issn.1000-0801.2025137

您当前的位置：

首页 >

文章列表页 >

基于DDQN的边缘算力融合网络资源管理

工程与应用 | 更新时间：2025-09-04

- 基于DDQN的边缘算力融合网络资源管理
- A DDQN-based resource management method for edge computing fusion network
- 电信科学 2025年41卷第8期页码：197-206
- 作者机构：
  
  中国铁塔股份有限公司，北京 100080
- 作者简介：
  
  [ "董玉池（1988- ），男，现就职于中国铁塔股份有限公司通信技术研究院，主要研究方向为边缘计算、算力网络和在网计算等。" ]
  [ "闫亚旗（1988- ），男，中国铁塔股份有限公司通信技术研究院高级工程师，主要研究方向为物联网、边缘计算、算力网络相关技术及产品创新。" ]
  [ "冉沛（1981- ），男，中国铁塔股份有限公司通信技术研究院高级工程师，主要研究方向为云及边缘计算、AI工程、行业数字化应用等。" ]
  [ "王东（1994- ），男，现就职于中国铁塔股份有限公司通信技术研究院，主要研究方向为云及边缘计算。" ]
  [ "张阔（1988- ），男，中国铁塔股份有限公司通信技术研究院高级工程师，主要研究方向为边缘计算、算力网络。" ]
  [ "张文龙（1998- ），男，现就职于中国铁塔通信技术研究院，主要研究方向为边缘计算、算力网络等。" ]
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2025137
  中图分类号： TP393
- 收稿：2024-11-06，
  
  修回：2025-05-14，
  
  纸质出版：2025-08-20
- 稿件说明：
移动端阅览
董玉池,闫亚旗,冉沛等.基于DDQN的边缘算力融合网络资源管理[J].电信科学,2025,41(08):197-206.

DONG Yuchi,YAN Yaqi,RAN Pei,et al.A DDQN-based resource management method for edge computing fusion network[J].Telecommunications Science,2025,41(08):197-206.
董玉池,闫亚旗,冉沛等.基于DDQN的边缘算力融合网络资源管理[J].电信科学,2025,41(08):197-206. DOI： 10.11959/j.issn.1000-0801.2025137.

DONG Yuchi,YAN Yaqi,RAN Pei,et al.A DDQN-based resource management method for edge computing fusion network[J].Telecommunications Science,2025,41(08):197-206. DOI： 10.11959/j.issn.1000-0801.2025137.

摘要

边缘算力融合网络将算力下沉至近用户侧，通过分布式边缘算力节点相互协同以在本地完成计算任务，显著降低云端负担和传输时延。然而，随着用户接入密度提高和场景复杂化，如何动态优化网络资源以协同应对多样化服务需求和大规模数据处理任务成为重大挑战。因此，提出了一种基于双深度Q网络（double deep Q network，DDQN）边缘算力融合网络资源管理方法，结合虚拟网络嵌入（virtual network embedding，VNE）技术，建立了以长期资源收益成本比最大化为目标的多约束优化模型。通过DDQN架构的在线学习能力，利用环境交互反馈实现动态优化决策。仿真实验表明，该方法在虚拟网络请求（virtual network request，VNR）接受成功率、长期嵌入收益和长期嵌入收益成本比3个指标上，较现有方法分别提升了13.3%、25.7%和8.5%。

Abstract

The edge computing fusion network sinks the computing resources to the user side and completes the computing tasks locally through the coordination of distributed edge computing nodes

which significantly reduces the cloud burden and transmission delay. However

with the increase of user access density and the complexity of scenarios

how to dynamically optimize network resources to cope with diversified service demands and large-scale data processing tasks has become a major challenge. Therefore

a resource management method for edge computing networks based on double deep Q network (DDQN) was proposed. Integrating the virtual network embedding (VNE) technology

the proposed method formulated a multi-constraint optimization model to maximize the long-term embedding revenue-to-cost ratio. By leveraging the online learning capabilities of the DDQN framework

it enabled dynamic decision-making through interaction and feedback with the environment. Simulation results demonstrate that the proposed method achieves average improvements of 13.3%

25.7% and 8.5% of virtual network request (VNR) acceptance rate

long-term embedding revenue

and long-term revenue-to-cost ratio

respectively

compared with the existing methods.

关键词

Keywords

references

刘云毅 , 张建敏 , 冯晓丽 , 等 . 5G MEC系统安全能力部署方案 [J ] . 电信科学 , 2022 , 38 ( 11 ): 143 - 152 .

LIU Y Y , ZHANG J M , FENG X L , et al . 5G MEC system security capability deployment scheme [J ] . Telecommunications Science , 2022 , 38 ( 11 ): 143 - 152 .

WANG S , SHENG H , ZHANG Y , et al . Blockchain-empowered distributed multicamera multitarget tracking in edge computing [J ] . IEEE Transactions on Industrial Informatics , 2024 , 20 ( 1 ): 369 - 379 .

雷波 , 刘增义 , 王旭亮 , 等 . 基于云、网、边融合的边缘计算新方案: 算力网络 [J ] . 电信科学 , 2019 , 35 ( 9 ): 44 - 51 .

LEI B , LIU Z Y , WANG X L , et al . Computing network: a new multi-access edge computing [J ] . Telecommunications Science , 2019 , 35 ( 9 ): 44 - 51 .

谢人超 , 廉晓飞 , 贾庆民 , 等 . 移动边缘计算卸载技术综述 [J ] . 通信学报 , 2018 , 39 ( 11 ): 138 - 155 .

XIE R C , LIAN X F , JIA Q M , et al . Survey on computation offloading in mobile edge computing [J ] . Journal on Communications , 2018 , 39 ( 11 ): 138 - 155 .

XIAO A L , WU S , OU Y K , et al . QoE-fairness-aware bandwidth allocation design for MEC-assisted ABR video transmission [J ] . IEEE Transactions on Network and Service Management , 2024 , PP(99): 1.

雷波 , 赵倩颖 , 赵慧玲 . 边缘计算与算力网络综述 [J ] . 中兴通讯技术 , 2021 , 27 ( 3 ): 3 - 6 .

LEI B , ZHAO Q Y , ZHAO H L . Overview of edge computing and computing power network [J ] . ZTE Technology Journal , 2021 , 27 ( 3 ): 3 - 6 .

冯川 , 张旭 , 马天纯 , 等 . 节点链路同时映射的多类型服务资源编排算法 [J ] . 通信学报 , 2024 , 45 ( 4 ): 54 - 64 .

FENG C , ZHANG X , MA T C , et al . Node-link simultaneous mapping based multi-type service resource orchestration algorithm [J ] . Journal on Communications , 2024 , 45 ( 4 ): 54 - 64 .

ZHANG P Y , CHEN N , KUMAR N , et al . Energy allocation for vehicle-to-grid settings: a low-cost proposal combining DRL and VNE [J ] . IEEE Transactions on Sustainable Computing , 2024 , 9 ( 1 ): 75 - 87 .

WU S , CHEN N , WEN G H , et al . Virtual network embedding for task offloading in IIoT: a DRL-assisted federated learning scheme [J ] . IEEE Transactions on Industrial Informatics , 2024 , 20 ( 4 ): 6814 - 6824 .

CHENG X , SU S , ZHANG Z B , et al . Virtual network embedding through topology-aware node ranking [J ] . ACM SIGCOMM Computer Communication Review , 2011 , 41 ( 2 ): 38 - 47 .

MELO M , SARGENTO S , KILLAT U , et al . Optimal virtual network embedding: node-link formulation [J ] . IEEE Transactions on Network and Service Management , 2013 , 10 ( 4 ): 356 - 368 .

WU S , CHEN N , XIAO A L , et al . AI-empowered virtual network embedding: a comprehensive survey [J ] . IEEE Communications Surveys & Tutorials , 2024 ( 99 ): 1 .

丁世飞 , 杜威 , 张健 , 等 . 多智能体深度强化学习研究进展 [J ] . 计算机学报 , 2024 , 47 ( 7 ): 1547 - 1567 .

DING S F , DU W , ZHANG J , et al . Research progress of multi-agent deep reinforcement learning [J ] . Chinese Journal of Computers , 2024 , 47 ( 7 ): 1547 - 1567 .

YAO H P , MA S H , WANG J J , et al . A continuous-decision virtual network embedding scheme relying on reinforcement learning [J ] . IEEE Transactions on Network and Service Management , 2020 , 17 ( 2 ): 864 - 875 .

YAN Z X , GE J G , WU Y L , et al . Automatic virtual network embedding: a deep reinforcement learning approach with graph convolutional networks [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 6 ): 1040 - 1057 .

ZHANG P Y , CHEN N , LI S B , et al . Multi-domain virtual network embedding algorithm based on horizontal federated learning [J ] . IEEE Transactions on Information Forensics and Security , 2023 , 18 : 3363 - 3375 .

ZHANG P Y , CHEN N , SHEN S G , et al . AI-enabled space-air-ground integrated networks: management and optimization [J ] . IEEE Network , 2024 , 38 ( 2 ): 186 - 192 .

CHEN N , ZHANG P Y , KUMAR N , et al . Spectral graph theory-based virtual network embedding for vehicular fog computing: a deep reinforcement learning architecture [J ] . Knowledge- Based Systems , 2022 , 257 : 109931 .

ZHANG P Y , CHEN N , XU G J , et al . Multi-target-aware dynamic resource scheduling for cloud-fog-edge multi-tier computing network [J ] . IEEE Transactions on Intelligent Transportation Systems , 2024 , 25 ( 5 ): 3885 - 3897 .

LIM H K , ULLAH I , KIM J B , et al . Virtual network embedding based on hierarchical cooperative multiagent reinforcement learning [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 5 ): 8552 - 8568 .

余竞航 , 赵一辰 , 宋浒 . 基于强化学习的边缘计算智能电网资源调度算法 [J ] . 电信科学 , 2024 , 40 ( 1 ): 115 - 122 .

YU J H , ZHAO Y C , SONG H . Edge computing smart grid resource scheduling algorithm based on reinforcement learning [J ] . Telecommunications Science , 2024 , 40 ( 1 ): 115 - 122 .

VAN HASSELT H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2016 , 30 ( 1 ): 1 - 7 .

XIAO A L , CHEN N , WU S , et al . DNFS-VNE: deep neuro fuzzy system driven virtual network embedding [J ] . IEEE Internet of Things Journal , 2024 ( 99 ): 1 .

浏览量

358

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向天地一体化的确定性网络技术研究

面向B5G和6G的边缘计算与网络切片资源管理

移动边缘计算中的计算卸载和资源管理方案

面向5G无线网络的智能干扰管理技术

一种混合云环境下SLA约束的云资源调度方法