Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN

Shouhua JIANG; Yiwu WANG

doi:10.11959/j.issn.1000-0801.2024025

您当前的位置：

首页 >

文章列表页 >

Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN

Research and Development | 更新时间：2024-06-05

- Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN
- Telecommunications Science Vol. 40, Issue 2, Pages: 96-106(2024)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2024025
  CLC： TP393
- Published Online：2024-02，
  
  Published：20 February 2024
- 稿件说明：
移动端阅览
Shouhua JIANG, Yiwu WANG. Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN[J]. Telecommunications science, 2024, 40(2): 96-106.
DOI：

Shouhua JIANG, Yiwu WANG. Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN[J]. Telecommunications science, 2024, 40(2): 96-106. DOI： 10.11959/j.issn.1000-0801.2024025.

摘要

随着网络技术的不断发展，基于Fat-Tree的网络拓扑结构分布式网络控制模式逐渐显露出其局限性，软件定义数据中心网络（software-defined data center network，SDCN）技术作为Fat-Tree网络拓扑的改进技术，受到越来越多研究者的关注。首先搭建了一个 SDCN 中的边缘计算架构和基于移动边缘计算（mobile edge computing，MEC）平台三层服务架构的任务卸载模型，结合移动边缘计算平台的实际应用场景，利用同策略经验回放和熵正则改进传统的深度Q网络（deep Q-leaning network，DQN）算法，优化了MEC平台的任务卸载策略，并设计了实验对基于同策略经验回放和熵正则的改进深度Q网络算法（improved DQN algorithm based on same strategy empirical playback and entropy regularization，RSS2E-DQN）和其他3种算法在负载均衡、能耗、时延、网络使用量几个方面进行对比分析，验证了改进算法在上述4个方面具有更优越的性能。

Abstract

With the continuous development of network technology

the network topology distributed network control mode based on Fat-Tree gradually reveals its limitations.Software-defined data center network (SDCN) technology

as an improved technology of Fat-Tree network topology

has attracted more and more researchers’ attention.Firstly

an edge computing architecture in SDCN and a task offloading model based on the three-layer service architecture of the mobile edge computing (MEC) platform were built

combined with the actual application scenarios of the MEC platform.Through the same strategy experience playback and entropy regularization

the traditional deep Q-leaning network (DQN) algorithm was improved

and the task offloading strategy of MEC platform was optimized.An improved DQN algorithm based on same strategy empirical playback and entropy regularization (RSS2E-DQN) was compared with three other algorithms in load balancing

energy consumption

delay and network usage.It is verified that the improved algorithm has better performance in the above four aspects.

关键词

Keywords

references

李丹 , 陈贵海 , 任丰原 , 等 . 数据中心网络的研究进展与趋势 [J ] . 计算机学报 , 2014 , 37 ( 2 ): 259 - 274 .

LI D , CHEN G H , REN F Y , et al . Data center network research progress and trends [J ] . Chinese Journal of Computers , 2014 , 37 ( 2 ): 259 - 274 .

王斌锋 , 苏金树 , 陈琳 . 云计算数据中心网络设计综述 [J ] . 计算机研究与发展 , 2016 , 53 ( 9 ): 2085 - 2106 .

WANG B F , SU J S , CHEN L . Review of the design of data center network for cloud computing [J ] . Journal of Computer Research and Development , 2016 , 53 ( 9 ): 2085 - 2106 .

NUNES B A A , MENDONCA M , NGUYEN X N , et al . A survey of software-defined networking:past,present,and future of programmable networks [J ] . IEEE Communications Surveys＆ Tutorials , 2014 , 16 ( 3 ): 1617 - 1634 .

吴强 , 徐鑫 , 刘国燕 . 基于SDN技术的数据中心基础网络构建 [J ] . 电信科学 , 2013 , 29 ( 1 ): 130 - 133 , 142 .

WU Q , XU X , LIU G Y . Construction of basic network in data center based on SDN technology [J ] . Telecommunications Science , 2013 , 29 ( 1 ): 130 - 133 , 142 .

曹腾飞 , 刘延亮 , 王晓英 . 基于改进深度强化学习的边缘计算服务卸载算法 [J ] . 计算机应用 , 2023 , 43 ( 5 ): 1543 - 1550 .

CAO T F , LIU Y L , WANG X Y . Edge computing and service offloading algorithm based on improved deep reinforcement learning [J ] . Journal of Computer Applications , 2023 , 43 ( 5 ): 1543 - 1550 .

卢海峰 , 顾春华 , 罗飞 , 等 . 基于深度强化学习的移动边缘计算任务卸载研究 [J ] . 计算机研究与发展 , 2020 , 57 ( 7 ): 1539 - 1554 .

LU H F , GU C H , LUO F , et al . Research on task offloading based on deep reinforcement learning in mobile edge computing [J ] . Journal of Computer Research and Development , 2020 , 57 ( 7 ): 1539 - 1554 .

邝祝芳 , 陈清林 , 李林峰 , 等 . 基于深度强化学习的多用户边缘计算任务卸载调度与资源分配算法 [J ] . 计算机学报 , 2022 , 45 ( 4 ): 812 - 824 .

KUANG Z F , CHEN Q L , LI L F , et al . Multi-user edge computing task offloading scheduling and resource allocation based on deep reinforcement learning [J ] . Chinese Journal of Computers , 2022 , 45 ( 4 ): 812 - 824 .

盛煜 , 朱正伟 , 朱晨阳 , 等 . 基于深度强化学习的多目标边缘任务调度研究 [J ] . 电子测量技术 , 2023 , 46 ( 8 ): 74 - 81 .

SHENG Y , ZHU Z W , ZHU C Y , et al . Research on multi-objective edge task scheduling based on deep reinforcement learning [J ] . Electronic Measurement Technology , 2023 , 46 ( 8 ): 74 - 81 .

JAISWAL A , BABU A R , ZADEH M Z , et al . A survey on contrastive self-supervised learning [J ] . Technologies , 2020 , 9 ( 1 ): 2 .

GRAVES A , MOHAMED A R , HINTON G . Speech recognition with deep recurrent neural networks [C ] // Proceedings of the 2013 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2013 : 6645 - 6649 .

SADIKI A , BENTAHAR J , DSSOULI R , et al . Deep reinforcement learning for the computation offloading in MIMO-based Edge Computing [J ] . Ad Hoc Networks , 2023 ( 141 ): 103080 .

LI M S , GAO J , ZHAO L , et al . Deep reinforcement learning for collaborative edge computing in vehicular networks [J ] . IEEE Transactions on Cognitive Communications and Networking , 2020 , 6 ( 4 ): 1122 - 1135 .

LI D J , XU S Y , LI P Y . Deep reinforcement learning-empowered resource allocation for mobile edge computing in cellular V2X networks [J ] . Sensors , 2021 , 21 ( 2 ): 372 .

XIAO Z , DAI X X , JIANG H B , et al . Vehicular task offloading via heat-aware MEC cooperation using game-theoretic method [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 3 ): 2038 - 2052 .

SHENG M , DAI Y P , LIU J Y , et al . Delay-aware computation offloading in NOMA MEC under differentiated uploading delay [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 4 ): 2813 - 2826 .

ZHAN W H , LUO C B , MIN G Y , et al . Mobility-aware multi-user offloading optimization for mobile edge computing [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 3 ): 3341 - 3356 .

ALFAKIH T , HASSAN M M , GUMAEI A , et al . Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA [J ] . IEEE Access , 2020 8 : 54074 - 54084 .

JOŠILO S , DÁN G . Joint management of wireless and computing resources for computation offloading in mobile edge clouds [J ] . IEEE Transactions on Cloud Computing , 2021 , 9 ( 4 ): 1507 - 1520 .

YUAN H T , ZHOU M C . Profit-maximized collaborative computation offloading and resource allocation in distributed cloud and edge computing systems [J ] . IEEE Transactions on Automation Science and Engineering , 2021 , 18 ( 3 ): 1277 - 1287 .

ZHAO T C , ZHOU S , SONG L Q , et al . Energy-optimal and delay-bounded computation offloading in mobile edge computing with heterogeneous clouds [J ] . China Communications , 2020 , 17 ( 5 ): 191 - 210 .

KUANG Z F , LI L F , GAO J , et al . Partial offloading scheduling and power allocation for mobile edge computing systems [J ] . IEEE Internet of Things Journal , 2019 , 6 ( 4 ): 6774 - 6785 .

杨思明 , 单征 , 丁煜 , 等 . 深度强化学习研究综述 [J ] . 计算机工程 , 2021 , 47 ( 12 ): 19 - 29 .

YANG S M , SHAN Z , DING Y , et al . Survey of research on deep reinforcement learning [J ] . Computer Engineering , 2021 , 47 ( 12 ): 19 - 29 .

ABADI M , AGARWAL A , BARHAM P , et al . TensorFlow:large-scale machine learning on heterogeneous distributed systems [J ] . arXiv prepnht , 2016 ,arXiv:1603.04467.

何荣希 , 雷田颖 , 林子薇 . 软件定义数据中心网络多约束节能路由算法 [J ] . 计算机研究与发展 , 2019 , 56 ( 6 ): 1219 - 1230 .

HE R X , LEI T Y , LIN Z W . Multi-constrained energy-saving routing algorithm in software-defined data center networks [J ] . Journal of Computer Research and Development , 2019 , 56 ( 6 ): 1219 - 1230 .

Views

104

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Heterogeneous computing network resource management based on virtual network embedding

Intelligent adaptation and integrated scheduling method for computing and networking resources

NR-U and Wi-Fi spectrum sharing for quality guaranteeing of power services

Digital twin based intelligent urban traffic forecasting and guidance strategy

A survey on AI algorithms applied in communication and computation in Internet of vehicles

Related Author

ZOU Haodong

CHEN Xin

WANG Ling

ZHAO Yichen

YU Jinghang

Chenghui SUN

Weiting ZHANG

Hongchao WANG

Related Institution

Information and Communication Company, State Grid Jiangsu Electric Power Co., Ltd.

National Mobile Communication Research Laboratory of Southeast University

State Grid Shandong Electric Power Company

Jinan Power Supply Company, State Grid Shandong Electronic Power Company

Beijing Laboratory of Advanced Information Networks

⁰