SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究

蒋守花; 王以伍

doi:10.11959/j.issn.1000-0801.2024025

您当前的位置：

首页 >

文章列表页 >

SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究

研究与开发 | 更新时间：2024-06-05

- SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究
- Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN
- 电信科学 2024年40卷第2期页码：96-106
- 作者机构：
- 作者简介：
  
  [ "蒋守花（1988- ），女，成都医学院现代教育技术中心工程师，主要研究方向为边缘计算、人工智能、物联网" ]
  [ "王以伍（1982- ），男，成都医学院现代教育技术中心工程师，主要研究方向为数据挖掘、网络安全" ]
- 基金信息：
  
  四川省高等学校人文社会科学重点研究基地·四川省教育信息化应用与发展研究中心项目;The Project of Sichuan Provincial Key Research Base of Humanities and Social Sciences and Sichuan Provincial Research Center for Application and Development of Education Informatization(JYXX23-002);成都医学院校基金科研项目;The Chengdu Medical College Funded Research Project(CYSYB23-02)
- DOI：10.11959/j.issn.1000-0801.2024025
  中图分类号： TP393
- 网络出版日期：2024-02，
  
  纸质出版日期：2024-02-20
- 稿件说明：
移动端阅览
蒋守花, 王以伍. SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究[J]. 电信科学, 2024,40(2):96-106.

Shouhua JIANG, Yiwu WANG. Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN[J]. Telecommunications science, 2024, 40(2): 96-106.
蒋守花, 王以伍. SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究[J]. 电信科学, 2024,40(2):96-106. DOI： 10.11959/j.issn.1000-0801.2024025.

Shouhua JIANG, Yiwu WANG. Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN[J]. Telecommunications science, 2024, 40(2): 96-106. DOI： 10.11959/j.issn.1000-0801.2024025.

摘要

随着网络技术的不断发展，基于Fat-Tree的网络拓扑结构分布式网络控制模式逐渐显露出其局限性，软件定义数据中心网络（software-defined data center network，SDCN）技术作为Fat-Tree网络拓扑的改进技术，受到越来越多研究者的关注。首先搭建了一个 SDCN 中的边缘计算架构和基于移动边缘计算（mobile edge computing，MEC）平台三层服务架构的任务卸载模型，结合移动边缘计算平台的实际应用场景，利用同策略经验回放和熵正则改进传统的深度Q网络（deep Q-leaning network，DQN）算法，优化了MEC平台的任务卸载策略，并设计了实验对基于同策略经验回放和熵正则的改进深度Q网络算法（improved DQN algorithm based on same strategy empirical playback and entropy regularization，RSS2E-DQN）和其他3种算法在负载均衡、能耗、时延、网络使用量几个方面进行对比分析，验证了改进算法在上述4个方面具有更优越的性能。

Abstract

With the continuous development of network technology

the network topology distributed network control mode based on Fat-Tree gradually reveals its limitations.Software-defined data center network (SDCN) technology

as an improved technology of Fat-Tree network topology

has attracted more and more researchers’ attention.Firstly

an edge computing architecture in SDCN and a task offloading model based on the three-layer service architecture of the mobile edge computing (MEC) platform were built

combined with the actual application scenarios of the MEC platform.Through the same strategy experience playback and entropy regularization

the traditional deep Q-leaning network (DQN) algorithm was improved

and the task offloading strategy of MEC platform was optimized.An improved DQN algorithm based on same strategy empirical playback and entropy regularization (RSS2E-DQN) was compared with three other algorithms in load balancing

energy consumption

delay and network usage.It is verified that the improved algorithm has better performance in the above four aspects.

关键词

Keywords

references

李丹 , 陈贵海 , 任丰原 , 等 . 数据中心网络的研究进展与趋势 [J ] . 计算机学报 , 2014 , 37 ( 2 ): 259 - 274 .

LI D , CHEN G H , REN F Y , et al . Data center network research progress and trends [J ] . Chinese Journal of Computers , 2014 , 37 ( 2 ): 259 - 274 .

王斌锋 , 苏金树 , 陈琳 . 云计算数据中心网络设计综述 [J ] . 计算机研究与发展 , 2016 , 53 ( 9 ): 2085 - 2106 .

WANG B F , SU J S , CHEN L . Review of the design of data center network for cloud computing [J ] . Journal of Computer Research and Development , 2016 , 53 ( 9 ): 2085 - 2106 .

NUNES B A A , MENDONCA M , NGUYEN X N , et al . A survey of software-defined networking:past,present,and future of programmable networks [J ] . IEEE Communications Surveys＆ Tutorials , 2014 , 16 ( 3 ): 1617 - 1634 .

吴强 , 徐鑫 , 刘国燕 . 基于SDN技术的数据中心基础网络构建 [J ] . 电信科学 , 2013 , 29 ( 1 ): 130 - 133 , 142 .

WU Q , XU X , LIU G Y . Construction of basic network in data center based on SDN technology [J ] . Telecommunications Science , 2013 , 29 ( 1 ): 130 - 133 , 142 .

曹腾飞 , 刘延亮 , 王晓英 . 基于改进深度强化学习的边缘计算服务卸载算法 [J ] . 计算机应用 , 2023 , 43 ( 5 ): 1543 - 1550 .

CAO T F , LIU Y L , WANG X Y . Edge computing and service offloading algorithm based on improved deep reinforcement learning [J ] . Journal of Computer Applications , 2023 , 43 ( 5 ): 1543 - 1550 .

卢海峰 , 顾春华 , 罗飞 , 等 . 基于深度强化学习的移动边缘计算任务卸载研究 [J ] . 计算机研究与发展 , 2020 , 57 ( 7 ): 1539 - 1554 .

LU H F , GU C H , LUO F , et al . Research on task offloading based on deep reinforcement learning in mobile edge computing [J ] . Journal of Computer Research and Development , 2020 , 57 ( 7 ): 1539 - 1554 .

邝祝芳 , 陈清林 , 李林峰 , 等 . 基于深度强化学习的多用户边缘计算任务卸载调度与资源分配算法 [J ] . 计算机学报 , 2022 , 45 ( 4 ): 812 - 824 .

KUANG Z F , CHEN Q L , LI L F , et al . Multi-user edge computing task offloading scheduling and resource allocation based on deep reinforcement learning [J ] . Chinese Journal of Computers , 2022 , 45 ( 4 ): 812 - 824 .

盛煜 , 朱正伟 , 朱晨阳 , 等 . 基于深度强化学习的多目标边缘任务调度研究 [J ] . 电子测量技术 , 2023 , 46 ( 8 ): 74 - 81 .

SHENG Y , ZHU Z W , ZHU C Y , et al . Research on multi-objective edge task scheduling based on deep reinforcement learning [J ] . Electronic Measurement Technology , 2023 , 46 ( 8 ): 74 - 81 .

JAISWAL A , BABU A R , ZADEH M Z , et al . A survey on contrastive self-supervised learning [J ] . Technologies , 2020 , 9 ( 1 ): 2 .

GRAVES A , MOHAMED A R , HINTON G . Speech recognition with deep recurrent neural networks [C ] // Proceedings of the 2013 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2013 : 6645 - 6649 .

SADIKI A , BENTAHAR J , DSSOULI R , et al . Deep reinforcement learning for the computation offloading in MIMO-based Edge Computing [J ] . Ad Hoc Networks , 2023 ( 141 ): 103080 .

LI M S , GAO J , ZHAO L , et al . Deep reinforcement learning for collaborative edge computing in vehicular networks [J ] . IEEE Transactions on Cognitive Communications and Networking , 2020 , 6 ( 4 ): 1122 - 1135 .

LI D J , XU S Y , LI P Y . Deep reinforcement learning-empowered resource allocation for mobile edge computing in cellular V2X networks [J ] . Sensors , 2021 , 21 ( 2 ): 372 .

XIAO Z , DAI X X , JIANG H B , et al . Vehicular task offloading via heat-aware MEC cooperation using game-theoretic method [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 3 ): 2038 - 2052 .

SHENG M , DAI Y P , LIU J Y , et al . Delay-aware computation offloading in NOMA MEC under differentiated uploading delay [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 4 ): 2813 - 2826 .

ZHAN W H , LUO C B , MIN G Y , et al . Mobility-aware multi-user offloading optimization for mobile edge computing [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 3 ): 3341 - 3356 .

ALFAKIH T , HASSAN M M , GUMAEI A , et al . Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA [J ] . IEEE Access , 2020 8 : 54074 - 54084 .

JOŠILO S , DÁN G . Joint management of wireless and computing resources for computation offloading in mobile edge clouds [J ] . IEEE Transactions on Cloud Computing , 2021 , 9 ( 4 ): 1507 - 1520 .

YUAN H T , ZHOU M C . Profit-maximized collaborative computation offloading and resource allocation in distributed cloud and edge computing systems [J ] . IEEE Transactions on Automation Science and Engineering , 2021 , 18 ( 3 ): 1277 - 1287 .

ZHAO T C , ZHOU S , SONG L Q , et al . Energy-optimal and delay-bounded computation offloading in mobile edge computing with heterogeneous clouds [J ] . China Communications , 2020 , 17 ( 5 ): 191 - 210 .

KUANG Z F , LI L F , GAO J , et al . Partial offloading scheduling and power allocation for mobile edge computing systems [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 4 ): 6774 - 6785 .

杨思明 , 单征 , 丁煜 , 等 . 深度强化学习研究综述 [J ] . 计算机工程 , 2021 , 47 ( 12 ): 19 - 29 .

YANG S M , SHAN Z , DING Y , et al . Survey of research on deep reinforcement learning [J ] . Computer Engineering , 2021 , 47 ( 12 ): 19 - 29 .

ABADI M , AGARWAL A , BARHAM P , et al . TensorFlow:large-scale machine learning on heterogeneous distributed systems [J ] . arXiv prepnht , 2016 ,arXiv:1603.04467.

何荣希 , 雷田颖 , 林子薇 . 软件定义数据中心网络多约束节能路由算法 [J ] . 计算机研究与发展 , 2019 , 56 ( 6 ): 1219 - 1230 .

HE R X , LEI T Y , LIN Z W . Multi-constrained energy-saving routing algorithm in software-defined data center networks [J ] . Journal of Computer Research and Development , 2019 , 56 ( 6 ): 1219 - 1230 .

浏览量

104

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于虚拟网络嵌入的异构算力网络资源管理

算网资源智能适配与融合调度方法

面向电力业务质量保障的NR-U与Wi-Fi频谱共享

基于数字孪生的城市交通流智能预测与导引策略

AI算法在车联网通信与计算中的应用综述