Research on dynamic allocation of network slicing resources based on OS-MBRL

YAN Jiahui; ZHONG Weixuan; DONG Ligang; JIANG Xian; WANG Guangchang; LU Lingrong

doi:10.11959/j.issn.1000-0801.2024228

您当前的位置：

首页 >

文章列表页 >

Research on dynamic allocation of network slicing resources based on OS-MBRL

Research and Development | 更新时间：2024-12-02

- Research on dynamic allocation of network slicing resources based on OS-MBRL
- Telecommunications Science Vol. 40, Issue 10, Pages: 61-77(2024)
- 作者机构：
  
  1.浙江工商大学，浙江杭州 310018
  2.UT斯达康通讯有限公司，浙江杭州 310051
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62301488);Key Project of Higher Education Research of Zhejiang Higher Education Association in 2022(KT2022017);Zhejiang Key Laboratory of New Network Standards and Application Technology(2013E10012)
- DOI：10.11959/j.issn.1000-0801.2024228
  CLC： TP393
- Received：08 May 2024，
  
  Revised：29 July 2024，
  
  Published：20 October 2024
- 稿件说明：
移动端阅览
严嘉辉,钟玮轩,董黎刚等.基于OS-MBRL的网络切片资源动态分配方法研究[J].电信科学,2024,40(10):61-77.

YAN Jiahui,ZHONG Weixuan,DONG Ligang,et al.Research on dynamic allocation of network slicing resources based on OS-MBRL[J].Telecommunications Science,2024,40(10):61-77.
严嘉辉,钟玮轩,董黎刚等.基于OS-MBRL的网络切片资源动态分配方法研究[J].电信科学,2024,40(10):61-77. DOI： 10.11959/j.issn.1000-0801.2024228.

YAN Jiahui,ZHONG Weixuan,DONG Ligang,et al.Research on dynamic allocation of network slicing resources based on OS-MBRL[J].Telecommunications Science,2024,40(10):61-77. DOI： 10.11959/j.issn.1000-0801.2024228.

摘要

随着网络用户业务需求的增长，如何实现网络切片动态和准确的资源分配是当下网络必须解决的问题。考虑传统无模型强化学习方法需要较长的模型训练时间，提出了一种基于OS-MBRL（model based RL supported by online SVM）的网络资源动态分配方法。该方法利用在线支持向量机算法构建了一个系统模型，保证在分配较少资源的情况下产生较低的服务等级协议（service level agreement，SLA）违规次数。仿真实验结果表明，与归一化优势函数（normalized advantage function，NAF）算法、深度Q网络（deep Q-network，DQN）算法和双延迟深度确定性策略梯度（twin delayed deep deterministic dolicy gradient，TD3）算法相比，该方法能够最高减少80%的SLA违规次数，同时降低9%的资源分配。

Abstract

With the growth of business needs of network users

how to achieve dynamic and accurate resource allocation of network slicing is a problem that must be solved in the current network. Considering that traditional modelless reinforcement learning methods require a longer model training time

a dynamic resource allocation method based on OS-MBRL was proposed. The online support vector machines algorithm was utilized to construct a system model that could handle dynamically changing data streams and continuously update the model to adapt to new data

ensuring a lower number of SLA violations when allocating fewer resources. Simulation experiment results show that compared with NAF algorithm

DQN algorithm

and TD3 algorithm

the proposed method can reduce SLA violations by up to 80% and resource allocation by 9%.

关键词

Keywords

references

何承卓 . 基于DAG认证的网络切片场景切换算法 [J ] . 科学技术创新 , 2024 ( 5 ): 70 - 73 .

HE C Z . Network slicing scene switching algorithm based on DAG authentication [J ] . Scientific and Technological Innovation , 2024 ( 5 ): 70 - 73 .

KAMAL M A , RAZA H W , ALAM M M , et al . Resource allocation schemes for 5G network: a systematic review [J ] . Sensors , 2021 , 21 ( 19 ): 6588 .

ALBONDA H D R , PEREZ-ROMERO J . An efficient RAN slicing strategy for a heterogeneous network with eMBB and V2X services [J ] . IEEE Access , 2019 , 7 : 44771 - 44782 .

WANG Z , WEI Y , YU F R , et al . Utility optimization for resource allocation in multi-access edge network slicing: a twin-actor deep deterministic policy gradient approach [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 8 ): 5842 - 5856 .

LI R , ZHAO Z , SUN Q , et al . Deep reinforcement learning for resource management in network slicing [J ] . IEEE Access , 2018 , 6 : 74429 - 74441 .

GUO J , NIE G , TIAN H , et al . Puncture-predictive fairness scheduling scheme for eMBB and URLLC based on TD3 algorithm [C ] // Proceedings of the CIC International Conference on Communications in China (ICCC) . Piscataway : IEEE Pree , 2023 : 1 - 6 .

QI C , HUA Y , LI R , et al . Deep reinforcement learning with discrete normalized advantage functions for resource management in network slicing [J ] . IEEE Communications Letters , 2019 , 23 ( 8 ): 1337 - 1341 .

RODRIGUEZ V Q , GUILLEMIN F , BOUBENDIR A . 5G E2E network slicing management with onap [C ] // Proceedings of the 2020 23rd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN) . Piscataway : IEEE Press , 2020 : 87 - 94 .

SOHAIB R M , ONIRETI O , SAMBO Y , et al . Intelligent resource management for eMBB and URLLC in 5G and beyond wireless networks [J ] . IEEE Access , 2023 .

NASSAR A , YILMAZ Y . Reinforcement learning for adaptive resource allocation in fog RAN for IoT with heterogeneous latency requirements [J ] . IEEE Access , 2019 , 7 : 128014 - 128025 .

ZHANG W , DERAKHSHANI M , ZHENG G , et al . Constrained risk-sensitive deep reinforcement learning for eMBB-URLLC joint scheduling [J ] . IEEE Transactions on Wireless Communications , 2024 .

SUN G , XIONG K , BOATENG G O , et al . Autonomous resource provisioning and resource customization for mixed traffics in virtualized radio access network [J ] . IEEE Systems Journal , 2019 , 13 ( 3 ): 2454 - 2465 .

陈赓 , 齐书虎 , 沈斐 , 等 . 面向B5G多业务场景基于D3QN的双时间尺度网络切片算法 [J ] . 通信学报 , 2022 , 43 ( 11 ): 213 - 224 .

CHEN G , QI S H , SHEN F , et al . Dual time scale network slicing algorithm based on D3QN for B5G multi-service scenarios [J ] . Journal on Communications , 2022 , 43 ( 11 ): 213 - 224 .

HURTADO S J A , CASILIMAS K , CAICEDO R O M . Deep reinforcement learning for resource management on network slicing: a survey [J ] . Sensors , 2022 , 22 ( 8 ): 3031 .

赵晨 , 张铖 , 黄永明 . 基于流量感知的无线接入网智能切片资源分配方法研究 [J ] . 信号处理 , 2024 , 40 ( 4 ): 719 - 732 .

ZHAO C , ZHANG C , HUANG Y M . Research on traffic-aware intelligent slicing resource allocation for radio access network [J ] . Journal of Signal Processing , 2024 , 40 ( 4 ): 719 - 732 .

FILALI A , MLIKA Z , CHERKAOUI S , et al . Dynamic SDN-based radio access network slicing with deep reinforcement learning for URLLC and eMBB services [J ] . IEEE Transactions on Network Science and Engineering , 2022 , 9 ( 4 ): 2174 - 2187 .

王再见 , 谷慧敏 . 基于联合优化的网络切片资源分配策略 [J ] . 通信学报 , 2023 , 44 ( 5 ): 234 - 245 .

WANG Z J , GU H M . Network slicing resource allocation strategy based on joint optimization [J ] . Journal on Communications , 2023 , 44 ( 5 )： 234 - 245 .

LIETO A , MALANCHINI I , MANDELLI S , et al . Strategic network slicing management in radio access networks [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 4 ): 1434 - 1448 .

TRAN T D , LE L B . Resource allocation for multi-tenant network slicing: a multi-leader multi-follower stackelberg game approach [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 8 ): 8886 - 8899 .

RAVEENDRAN N , GU Y , JIANG C , et al . Cyclic three-sided matching game inspired wireless network virtualization [J ] . IEEE Transactions on Mobile Computing , 2021 , 20 ( 2 ): 416 - 428 .

AFOLABI I , TALEB T , SAMDANIS K , et al . Network slicing and softwarization: a survey on principles, enabling technologies, and solutions [J ] . IEEE Communications Surveys & Tutorials , 2018 , 20 ( 3 ): 2429 - 2453 .

LI R , ZHAO Z , SUN Q , et al . Deep reinforcement learning for resource management in network slicing [J ] . IEEE Access , 2018 , 6 : 74429 - 74441 .

MEI J , WANG X , ZHENG K , et al . Intelligent radio access network slicing for service provisioning in 6G: a hierarchical deep reinforcement learning approach [J ] . IEEE Transactions on Communications , 2021 , 69 ( 9 ): 6063 - 6078 .

BERTSEKAS D . Reinforcement learning and optimal control [M ] . Cambridge : Athena Scientific , 2019 .

HU L , HU C , HUO Z , et al . Online support vector machine with a single pass for streaming data [J ] . Mathematics , 2022 , 10 ( 17 ): 3113 .

AL-ALI M , YAACOUB E . Resource allocation scheme for eMBB and URLLC coexistence in 6G networks [J ] . Wireless Networks , 2023 , 29 ( 6 ): 2519 - 2538 .

3GPP. Evolved universal terrestrial radio access(E-UTRA); radio frequency(RF) system scenarios, document 36.942: Version 15.0.0 [S ] . 2018 .

BAUMGARTNER M , JUHAR J , PAPAJ J . Simulation of 5G and LTE-A access technologies via network simulator NS-3 [C ] // Proceedings of the 2021 44th International Conference on Telecommunications and Signal Processing(TSP) . Piscataway : IEEE Press , 2021 : 77 - 80 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems

Adaptive distributed cloud edge collaborative load control strategy for load management

Survey on large language models alignment research

Slice scheduling for industrial time-critical services in 5G TSN integration network

Task urgency-based resource allocation algorithm in industrial Internet of things

Related Author

LIU Qingli

LI Xiaoyu

LI Rui

LI Siwei

JIN Li

YU Long

DU Lishi

YUE Liang

Related Institution

Key Laboratory of Communication and Network, Dalian University

School of Information Engineering, Dalian University

Beijing FibrLINK Communications Co., Ltd.

Key Laboratory of Smart Grid of Ministry of Education, Tianjin University

ZTE Corporation

⁰