基于OS-MBRL的网络切片资源动态分配方法研究

严嘉辉; 钟玮轩; 董黎刚; 蒋献; 王广昌; 陆凌蓉

doi:10.11959/j.issn.1000-0801.2024228

您当前的位置：

首页 >

文章列表页 >

基于OS-MBRL的网络切片资源动态分配方法研究

研究与开发 | 更新时间：2024-12-02

- 基于OS-MBRL的网络切片资源动态分配方法研究
- Research on dynamic allocation of network slicing resources based on OS-MBRL
- 电信科学 2024年40卷第10期页码：61-77
- 作者机构：
  
  1.浙江工商大学，浙江杭州 310018
  2.UT斯达康通讯有限公司，浙江杭州 310051
- 作者简介：
  
  [ "严嘉辉（2000- ），男，浙江工商大学在读，主要研究方向为通信与网络。" ]
  [ "钟玮轩（1999- ），男，浙江工商大学在读，主要研究方向为通信与网络。" ]
  [ "董黎刚（1973- ），男，博士，浙江工商大学信息与电子工程学院院长、教授、硕士生导师，主要研究方向为新一代网络和分布式系统。" ]
  [ "蒋献（1988- ），男，浙江工商大学讲师，主要研究方向为智慧网络与智慧教育。" ]
  [ "王广昌（1970- ），男，UT斯达康通讯有限公司高级工程师，主要研究方向为计算机网络、网络管理系统、SDN管控系统等。" ]
  [ "陆凌蓉（1972- ），女，UT斯达康通讯有限公司高级工程师，主要研究方向为软件定义网络等。" ]
- 基金信息：
  
  国家自然科学基金资助项目(62301488);浙江省重点研发计划项目;Zhejiang Province Key Research and Devel-opment Program(2021C01036);2022年度浙江省高等教育学会高等教育研究重点立项课题(KT2022017);浙江省新型网络标准与应用技术重点实验室项目(2013E10012)
- DOI：10.11959/j.issn.1000-0801.2024228
  中图分类号： TP393
- 收稿日期：2024-05-08，
  
  修回日期：2024-07-29，
  
  纸质出版日期：2024-10-20
- 稿件说明：
移动端阅览
严嘉辉,钟玮轩,董黎刚等.基于OS-MBRL的网络切片资源动态分配方法研究[J].电信科学,2024,40(10):61-77.

YAN Jiahui,ZHONG Weixuan,DONG Ligang,et al.Research on dynamic allocation of network slicing resources based on OS-MBRL[J].Telecommunications Science,2024,40(10):61-77.
严嘉辉,钟玮轩,董黎刚等.基于OS-MBRL的网络切片资源动态分配方法研究[J].电信科学,2024,40(10):61-77. DOI： 10.11959/j.issn.1000-0801.2024228.

YAN Jiahui,ZHONG Weixuan,DONG Ligang,et al.Research on dynamic allocation of network slicing resources based on OS-MBRL[J].Telecommunications Science,2024,40(10):61-77. DOI： 10.11959/j.issn.1000-0801.2024228.

摘要

随着网络用户业务需求的增长，如何实现网络切片动态和准确的资源分配是当下网络必须解决的问题。考虑传统无模型强化学习方法需要较长的模型训练时间，提出了一种基于OS-MBRL（model based RL supported by online SVM）的网络资源动态分配方法。该方法利用在线支持向量机算法构建了一个系统模型，保证在分配较少资源的情况下产生较低的服务等级协议（service level agreement，SLA）违规次数。仿真实验结果表明，与归一化优势函数（normalized advantage function，NAF）算法、深度Q网络（deep Q-network，DQN）算法和双延迟深度确定性策略梯度（twin delayed deep deterministic dolicy gradient，TD3）算法相比，该方法能够最高减少80%的SLA违规次数，同时降低9%的资源分配。

Abstract

With the growth of business needs of network users

how to achieve dynamic and accurate resource allocation of network slicing is a problem that must be solved in the current network. Considering that traditional modelless reinforcement learning methods require a longer model training time

a dynamic resource allocation method based on OS-MBRL was proposed. The online support vector machines algorithm was utilized to construct a system model that could handle dynamically changing data streams and continuously update the model to adapt to new data

ensuring a lower number of SLA violations when allocating fewer resources. Simulation experiment results show that compared with NAF algorithm

DQN algorithm

and TD3 algorithm

the proposed method can reduce SLA violations by up to 80% and resource allocation by 9%.

关键词

Keywords

references

何承卓 . 基于DAG认证的网络切片场景切换算法 [J ] . 科学技术创新 , 2024 ( 5 ): 70 - 73 .

HE C Z . Network slicing scene switching algorithm based on DAG authentication [J ] . Scientific and Technological Innovation , 2024 ( 5 ): 70 - 73 .

KAMAL M A , RAZA H W , ALAM M M , et al . Resource allocation schemes for 5G network: a systematic review [J ] . Sensors , 2021 , 21 ( 19 ): 6588 .

ALBONDA H D R , PEREZ-ROMERO J . An efficient RAN slicing strategy for a heterogeneous network with eMBB and V2X services [J ] . IEEE Access , 2019 , 7 : 44771 - 44782 .

WANG Z , WEI Y , YU F R , et al . Utility optimization for resource allocation in multi-access edge network slicing: a twin-actor deep deterministic policy gradient approach [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 8 ): 5842 - 5856 .

LI R , ZHAO Z , SUN Q , et al . Deep reinforcement learning for resource management in network slicing [J ] . IEEE Access , 2018 , 6 : 74429 - 74441 .

GUO J , NIE G , TIAN H , et al . Puncture-predictive fairness scheduling scheme for eMBB and URLLC based on TD3 algorithm [C ] // Proceedings of the CIC International Conference on Communications in China (ICCC) . Piscataway : IEEE Pree , 2023 : 1 - 6 .

QI C , HUA Y , LI R , et al . Deep reinforcement learning with discrete normalized advantage functions for resource management in network slicing [J ] . IEEE Communications Letters , 2019 , 23 ( 8 ): 1337 - 1341 .

RODRIGUEZ V Q , GUILLEMIN F , BOUBENDIR A . 5G E2E network slicing management with onap [C ] // Proceedings of the 2020 23rd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN) . Piscataway : IEEE Press , 2020 : 87 - 94 .

SOHAIB R M , ONIRETI O , SAMBO Y , et al . Intelligent resource management for eMBB and URLLC in 5G and beyond wireless networks [J ] . IEEE Access , 2023 .

NASSAR A , YILMAZ Y . Reinforcement learning for adaptive resource allocation in fog RAN for IoT with heterogeneous latency requirements [J ] . IEEE Access , 2019 , 7 : 128014 - 128025 .

ZHANG W , DERAKHSHANI M , ZHENG G , et al . Constrained risk-sensitive deep reinforcement learning for eMBB-URLLC joint scheduling [J ] . IEEE Transactions on Wireless Communications , 2024 .

SUN G , XIONG K , BOATENG G O , et al . Autonomous resource provisioning and resource customization for mixed traffics in virtualized radio access network [J ] . IEEE Systems Journal , 2019 , 13 ( 3 ): 2454 - 2465 .

陈赓 , 齐书虎 , 沈斐 , 等 . 面向B5G多业务场景基于D3QN的双时间尺度网络切片算法 [J ] . 通信学报 , 2022 , 43 ( 11 ): 213 - 224 .

CHEN G , QI S H , SHEN F , et al . Dual time scale network slicing algorithm based on D3QN for B5G multi-service scenarios [J ] . Journal on Communications , 2022 , 43 ( 11 ): 213 - 224 .

HURTADO S J A , CASILIMAS K , CAICEDO R O M . Deep reinforcement learning for resource management on network slicing: a survey [J ] . Sensors , 2022 , 22 ( 8 ): 3031 .

赵晨 , 张铖 , 黄永明 . 基于流量感知的无线接入网智能切片资源分配方法研究 [J ] . 信号处理 , 2024 , 40 ( 4 ): 719 - 732 .

ZHAO C , ZHANG C , HUANG Y M . Research on traffic-aware intelligent slicing resource allocation for radio access network [J ] . Journal of Signal Processing , 2024 , 40 ( 4 ): 719 - 732 .

FILALI A , MLIKA Z , CHERKAOUI S , et al . Dynamic SDN-based radio access network slicing with deep reinforcement learning for URLLC and eMBB services [J ] . IEEE Transactions on Network Science and Engineering , 2022 , 9 ( 4 ): 2174 - 2187 .

王再见 , 谷慧敏 . 基于联合优化的网络切片资源分配策略 [J ] . 通信学报 , 2023 , 44 ( 5 ): 234 - 245 .

WANG Z J , GU H M . Network slicing resource allocation strategy based on joint optimization [J ] . Journal on Communications , 2023 , 44 ( 5 )： 234 - 245 .

LIETO A , MALANCHINI I , MANDELLI S , et al . Strategic network slicing management in radio access networks [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 4 ): 1434 - 1448 .

TRAN T D , LE L B . Resource allocation for multi-tenant network slicing: a multi-leader multi-follower stackelberg game approach [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 8 ): 8886 - 8899 .

RAVEENDRAN N , GU Y , JIANG C , et al . Cyclic three-sided matching game inspired wireless network virtualization [J ] . IEEE Transactions on Mobile Computing , 2021 , 20 ( 2 ): 416 - 428 .

AFOLABI I , TALEB T , SAMDANIS K , et al . Network slicing and softwarization: a survey on principles, enabling technologies, and solutions [J ] . IEEE Communications Surveys & Tutorials , 2018 , 20 ( 3 ): 2429 - 2453 .

LI R , ZHAO Z , SUN Q , et al . Deep reinforcement learning for resource management in network slicing [J ] . IEEE Access , 2018 , 6 : 74429 - 74441 .

MEI J , WANG X , ZHENG K , et al . Intelligent radio access network slicing for service provisioning in 6G: a hierarchical deep reinforcement learning approach [J ] . IEEE Transactions on Communications , 2021 , 69 ( 9 ): 6063 - 6078 .

BERTSEKAS D . Reinforcement learning and optimal control [M ] . Cambridge : Athena Scientific , 2019 .

HU L , HU C , HUO Z , et al . Online support vector machine with a single pass for streaming data [J ] . Mathematics , 2022 , 10 ( 17 ): 3113 .

AL-ALI M , YAACOUB E . Resource allocation scheme for eMBB and URLLC coexistence in 6G networks [J ] . Wireless Networks , 2023 , 29 ( 6 ): 2519 - 2538 .

3GPP. Evolved universal terrestrial radio access(E-UTRA); radio frequency(RF) system scenarios, document 36.942: Version 15.0.0 [S ] . 2018 .

BAUMGARTNER M , JUHAR J , PAPAJ J . Simulation of 5G and LTE-A access technologies via network simulator NS-3 [C ] // Proceedings of the 2021 44th International Conference on Telecommunications and Signal Processing(TSP) . Piscataway : IEEE Press , 2021 : 77 - 80 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的毫米波大规模MIMO系统资源联合优化

面向多元可控负荷调控的云边协同负荷资源分配策略

大语言模型对齐研究综述

面向工业时敏业务的5G TSN融合网络切片资源调度

工业物联网中基于任务紧急程度的资源分配算法