基于深度强化学习的毫米波大规模MIMO系统资源联合优化

刘庆利; 李晓宇; 李蕊

doi:10.11959/j.issn.1000-0801.2024217

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的毫米波大规模MIMO系统资源联合优化

研究与开发 | 更新时间：2024-12-02

- 基于深度强化学习的毫米波大规模MIMO系统资源联合优化
- Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems
- 电信科学 2024年40卷第10期页码：39-51
- 作者机构：
  
  1.大连大学信息工程学院，辽宁大连 116622
  2.大连大学通信与网络重点实验室，辽宁大连 116622
- 作者简介：
  
  [ "刘庆利（1981- ），男，大连大学信息工程学院、通信与网络重点实验室教授，主要研究方向为无人机系统、网络通信等。" ]
  [ "李晓宇（1998- ），女，大连大学信息工程学院、通信与网络重点实验室硕士生，主要研究方向为网络通信。" ]
  [ "李蕊（1997- ），女，大连大学信息工程学院、通信与网络重点实验室硕士生，主要研究方向为网络通信。" ]
- 基金信息：
  
  国家自然科学基金资助项目(61931004)
- DOI：10.11959/j.issn.1000-0801.2024217
  中图分类号： TN929.5
- 收稿日期：2024-06-06，
  
  修回日期：2024-09-30，
  
  纸质出版日期：2024-10-20
- 稿件说明：
移动端阅览
刘庆利,李晓宇,李蕊.基于深度强化学习的毫米波大规模MIMO系统资源联合优化[J].电信科学,2024,40(10):39-51.

LIU Qingli,LI Xiaoyu,LI Rui.Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems[J].Telecommunications Science,2024,40(10):39-51.
刘庆利,李晓宇,李蕊.基于深度强化学习的毫米波大规模MIMO系统资源联合优化[J].电信科学,2024,40(10):39-51. DOI： 10.11959/j.issn.1000-0801.2024217.

LIU Qingli,LI Xiaoyu,LI Rui.Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems[J].Telecommunications Science,2024,40(10):39-51. DOI： 10.11959/j.issn.1000-0801.2024217.

摘要

针对毫米波大规模多输入多输出（multiple-input multiple-output，MIMO）系统中无线资源受限、功耗巨大且能量效率与系统容量相互制约而导致吞吐量和能量效率低的问题，提出一种基于深度强化学习（deep reinforcement learning，DRL）的资源联合优化方法。该方法采用三阶段策略，首先，构建射频波束成形器，通过少量的射频链降低硬件成本和总功耗；其次，利用有效信道状态信息设计基带预编码器；最后，设计应用双层DRL架构实现动态离散带宽和连续功率资源的分配。实验结果表明，与单级全数字预编码和混合预编码的均等资源分配方法及基于粒子群优化的资源分配算法相比，所提的联合优化方法显著提高了系统的吞吐量和能量效率。

Abstract

Aiming at the problem of low throughput and energy efficiency caused by limited wireless resources

huge power consumption

and mutual constraints between energy efficiency and system capacity in millimeter-wave large-scale multiple-input multiple-output systems

a resource co-optimization method based on deep reinforcement learning was proposed. The method was adopted in a three-stage strategy

firstly

an RF beamformer was constructed to reduce the hardware cost and total power consumption through a small number of RF chains; secondly

a baseband precoder was designed using the effective channel state information; and finally

a two-tier deep reinforcement learning architecture was designed and applied to realize dynamic discrete bandwidth and continuous power resource allocation. Experimental results show that the proposed joint optimization method significantly improves the throughput and energy efficiency of the system compared with the single-stage all-digital precoding and hybrid precoding equal resource allocation methods and the particle swarm optimization-based resource allocation algorithm.

关键词

Keywords

references

ASHRAF S , SHEIKH J A , ASHRAF A , et al . 5G millimeter wave technology: an overview [M ] //SHEIKH J A, KHAN T, KANAUJIA B K, eds. Energy Systems in Electrical Engineering . Singapore : Springer Nature Singapore , 2024 : 97 - 112 .

ELIJAH O , ABDUL RAHIM S K , NEW W K , et al . Intelligent massive MIMO systems for beyond 5G networks: an overview and future trends [J ] . IEEE Access , 2022 , 10 : 102532 - 102563 .

AHMED I , KHAMMARI H , SHAHID A , et al . A survey on hybrid beamforming techniques in 5G: architecture and system model perspectives [J ] . IEEE Communications Surveys & Tutorials , 2018 , 20 ( 4 ): 3060 - 3097 .

KOC A , LE-NGOC T . Full-duplex mmWave massive MIMO systems: a joint hybrid precoding/combining and self-interference cancellation design [J ] . IEEE Open Journal of the Communications Society , 2021 ( 2 ): 754 - 774 .

KOC A , MASMOUDI A , LE-NGOC T . 3D angular-based hybrid precoding and user grouping for uniform rectangular arrays in massive MU-MIMO systems [J ] . IEEE Access , 2020 , 8 : 84689 - 84712 .

MAHMOOD M , KOC A , LE-NGOC T . Energy-efficient MU-massive-MIMO hybrid precoder design: low-resolution phase shifters and digital-to-analog converters for 2D antenna array structures [J ] . IEEE Open Journal of the Communications Society , 2021 , 2 : 1842 - 1861 .

ZHENG W Y , KOC A , LE-NGOC T . Sub-connected hybrid precoding architectures in massive MIMO systems [C ] // Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference . Piscataway : IEEE Press , 2020 : 1 - 6 .

HARKAT H , MONTEIRO P , GAMEIRO A , et al . A survey on MIMO-OFDM systems: Review of recent trends [J ] . Signals , 2022 , 3 ( 2 ): 359 - 395 .

BISHE F , KOC A , LE-NGOC T . Intelligent subcarrier allocation in hybrid beamforming multi-user mMIMO-OFDM systems [C ] // Proceedings of the 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring) . Piscataway : IEEE Press , 2023 : 1 - 5 .

YILDIRIM I , KOC A , BASAR E , et al . RIS-Aided Angular-Based Hybrid Beamforming Design in mmWave Massive MIMO Systems [C ] // Proceedings of the GLOBECOM 2022-2022 IEEE Global Communications Conference . Piscataway : IEEE Press , 2022 : 5267 - 5272 .

CHOI J , OH H . Hybrid precoding and combining based on rate balancing for wideband mmWave multiuser MIMO systems [J ] . IEEE Access , 2023 , 11 : 12987 - 12999 .

KAUR D , GUPTA G , JHA V . A game theoretic bandwidth allocation scheme towards improving the fairness of XG-PON systems [C ] // Proceedings of the 2021 International Conference on Information and Communication Technology Convergence (ICTC) . Piscataway : IEEE Press , 2021 : 921 - 926 .

ZENG M , HAO W M , DOBRE O A , et al . Energy-efficient power allocation in uplink mmWave massive MIMO with NOMA [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 3 ): 3000 - 3004 .

PENG J B , HU Y , ZHANG F , et al . Energy efficiency resource allocation in downlink massive MIMO system with ZF precoding [C ] // Proceedings of the International Conference on Frontiers of Electronics, Information and Computation Technologies . New York : ACM , 2021 .

SUN Q , ZHAO H , WANG J , et al . Deep learning-based joint CSI feedback and hybrid precoding in FDD mmwave massive MIMO systems [J ] . Entropy , 2022 , 24 ( 4 ): 441 .

KOC A , LE-NGOC T . Swarm intelligence based power allocation in hybrid millimeter-wave massive MIMO systems [C ] // IEEE Wireless Communications and Networking Conference (WCNC 2021) . IEEE , 2021

ZHU X Y , KOC A , MORAWSKI R , et al . A deep learning and geospatial data-based channel estimation technique for hybrid massive MIMO systems [J ] . IEEE Access , 2021 , 9 : 145115 - 145132 .

MAHMOOD M , KOC A , MORAWSKI R , et al . Achieving capacity gains in practical full-duplex massive MIMO systems: a multi-objective optimization approach using hybrid beamforming [J ] . IEEE Open Journal of the Communications Society , 2024 , 5 : 2268 - 2286 .

XU Z Y , WANG Y Z , TANG J , et al . A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs [C ] // Proceedings of the 2017 IEEE International Conference on Communications (ICC) . Piscataway : IEEE Press , 2017 : 1 - 6 .

ZHAO N , LIANG Y C , NIYATO D , et al . Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5141 - 5152 .

NASIR Y S , GUO D N . Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 10 ): 2239 - 2250 .

NAPARSTEK O , COHEN K . Deep multi-user reinforcement learning for distributed dynamic spectrum access [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 1 ): 310 - 323 .

WANG X M , ZHANG Y H , SHEN R J , et al . DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 8 ): 7279 - 7294 .

DU J B , XU W , ZHAO C M , et al . Weighted spectral efficiency optimization for hybrid beamforming in multiuser massive MIMO-OFDM systems [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 10 ): 9698 - 9712 .

3GPP. 5G: study on channel model for frequencies from 0.5 to 100 GHz: TR 38.901 [S ] . 2020 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

高速率广义空间码索引调制系统的性能分析

基于OS-MBRL的网络切片资源动态分配方法研究

面向多元可控负荷调控的云边协同负荷资源分配策略

6G网络任务卸载与细粒度切片资源调度联合优化算法

工业物联网中基于任务紧急程度的资源分配算法