Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems

LIU Qingli; LI Xiaoyu; LI Rui

doi:10.11959/j.issn.1000-0801.2024217

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems

Research and Development | 更新时间：2024-12-02

- Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems
- Telecommunications Science Vol. 40, Issue 10, Pages: 39-51(2024)
- 作者机构：
  
  1.大连大学信息工程学院，辽宁大连 116622
  2.大连大学通信与网络重点实验室，辽宁大连 116622
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(61931004)
- DOI：10.11959/j.issn.1000-0801.2024217
  CLC： TN929.5
- Received：06 June 2024，
  
  Revised：30 September 2024，
  
  Published：20 October 2024
- 稿件说明：
移动端阅览
刘庆利,李晓宇,李蕊.基于深度强化学习的毫米波大规模MIMO系统资源联合优化[J].电信科学,2024,40(10):39-51.

LIU Qingli,LI Xiaoyu,LI Rui.Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems[J].Telecommunications Science,2024,40(10):39-51.
刘庆利,李晓宇,李蕊.基于深度强化学习的毫米波大规模MIMO系统资源联合优化[J].电信科学,2024,40(10):39-51. DOI： 10.11959/j.issn.1000-0801.2024217.

LIU Qingli,LI Xiaoyu,LI Rui.Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems[J].Telecommunications Science,2024,40(10):39-51. DOI： 10.11959/j.issn.1000-0801.2024217.

摘要

针对毫米波大规模多输入多输出（multiple-input multiple-output，MIMO）系统中无线资源受限、功耗巨大且能量效率与系统容量相互制约而导致吞吐量和能量效率低的问题，提出一种基于深度强化学习（deep reinforcement learning，DRL）的资源联合优化方法。该方法采用三阶段策略，首先，构建射频波束成形器，通过少量的射频链降低硬件成本和总功耗；其次，利用有效信道状态信息设计基带预编码器；最后，设计应用双层DRL架构实现动态离散带宽和连续功率资源的分配。实验结果表明，与单级全数字预编码和混合预编码的均等资源分配方法及基于粒子群优化的资源分配算法相比，所提的联合优化方法显著提高了系统的吞吐量和能量效率。

Abstract

Aiming at the problem of low throughput and energy efficiency caused by limited wireless resources

huge power consumption

and mutual constraints between energy efficiency and system capacity in millimeter-wave large-scale multiple-input multiple-output systems

a resource co-optimization method based on deep reinforcement learning was proposed. The method was adopted in a three-stage strategy

firstly

an RF beamformer was constructed to reduce the hardware cost and total power consumption through a small number of RF chains; secondly

a baseband precoder was designed using the effective channel state information; and finally

a two-tier deep reinforcement learning architecture was designed and applied to realize dynamic discrete bandwidth and continuous power resource allocation. Experimental results show that the proposed joint optimization method significantly improves the throughput and energy efficiency of the system compared with the single-stage all-digital precoding and hybrid precoding equal resource allocation methods and the particle swarm optimization-based resource allocation algorithm.

关键词

Keywords

references

ASHRAF S , SHEIKH J A , ASHRAF A , et al . 5G millimeter wave technology: an overview [M ] //SHEIKH J A, KHAN T, KANAUJIA B K, eds. Energy Systems in Electrical Engineering . Singapore : Springer Nature Singapore , 2024 : 97 - 112 .

ELIJAH O , ABDUL RAHIM S K , NEW W K , et al . Intelligent massive MIMO systems for beyond 5G networks: an overview and future trends [J ] . IEEE Access , 2022 , 10 : 102532 - 102563 .

AHMED I , KHAMMARI H , SHAHID A , et al . A survey on hybrid beamforming techniques in 5G: architecture and system model perspectives [J ] . IEEE Communications Surveys & Tutorials , 2018 , 20 ( 4 ): 3060 - 3097 .

KOC A , LE-NGOC T . Full-duplex mmWave massive MIMO systems: a joint hybrid precoding/combining and self-interference cancellation design [J ] . IEEE Open Journal of the Communications Society , 2021 ( 2 ): 754 - 774 .

KOC A , MASMOUDI A , LE-NGOC T . 3D angular-based hybrid precoding and user grouping for uniform rectangular arrays in massive MU-MIMO systems [J ] . IEEE Access , 2020 , 8 : 84689 - 84712 .

MAHMOOD M , KOC A , LE-NGOC T . Energy-efficient MU-massive-MIMO hybrid precoder design: low-resolution phase shifters and digital-to-analog converters for 2D antenna array structures [J ] . IEEE Open Journal of the Communications Society , 2021 , 2 : 1842 - 1861 .

ZHENG W Y , KOC A , LE-NGOC T . Sub-connected hybrid precoding architectures in massive MIMO systems [C ] // Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference . Piscataway : IEEE Press , 2020 : 1 - 6 .

HARKAT H , MONTEIRO P , GAMEIRO A , et al . A survey on MIMO-OFDM systems: Review of recent trends [J ] . Signals , 2022 , 3 ( 2 ): 359 - 395 .

BISHE F , KOC A , LE-NGOC T . Intelligent subcarrier allocation in hybrid beamforming multi-user mMIMO-OFDM systems [C ] // Proceedings of the 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring) . Piscataway : IEEE Press , 2023 : 1 - 5 .

YILDIRIM I , KOC A , BASAR E , et al . RIS-Aided Angular-Based Hybrid Beamforming Design in mmWave Massive MIMO Systems [C ] // Proceedings of the GLOBECOM 2022-2022 IEEE Global Communications Conference . Piscataway : IEEE Press , 2022 : 5267 - 5272 .

CHOI J , OH H . Hybrid precoding and combining based on rate balancing for wideband mmWave multiuser MIMO systems [J ] . IEEE Access , 2023 , 11 : 12987 - 12999 .

KAUR D , GUPTA G , JHA V . A game theoretic bandwidth allocation scheme towards improving the fairness of XG-PON systems [C ] // Proceedings of the 2021 International Conference on Information and Communication Technology Convergence (ICTC) . Piscataway : IEEE Press , 2021 : 921 - 926 .

ZENG M , HAO W M , DOBRE O A , et al . Energy-efficient power allocation in uplink mmWave massive MIMO with NOMA [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 3 ): 3000 - 3004 .

PENG J B , HU Y , ZHANG F , et al . Energy efficiency resource allocation in downlink massive MIMO system with ZF precoding [C ] // Proceedings of the International Conference on Frontiers of Electronics, Information and Computation Technologies . New York : ACM , 2021 .

SUN Q , ZHAO H , WANG J , et al . Deep learning-based joint CSI feedback and hybrid precoding in FDD mmwave massive MIMO systems [J ] . Entropy , 2022 , 24 ( 4 ): 441 .

KOC A , LE-NGOC T . Swarm intelligence based power allocation in hybrid millimeter-wave massive MIMO systems [C ] // IEEE Wireless Communications and Networking Conference (WCNC 2021) . IEEE , 2021

ZHU X Y , KOC A , MORAWSKI R , et al . A deep learning and geospatial data-based channel estimation technique for hybrid massive MIMO systems [J ] . IEEE Access , 2021 , 9 : 145115 - 145132 .

MAHMOOD M , KOC A , MORAWSKI R , et al . Achieving capacity gains in practical full-duplex massive MIMO systems: a multi-objective optimization approach using hybrid beamforming [J ] . IEEE Open Journal of the Communications Society , 2024 , 5 : 2268 - 2286 .

XU Z Y , WANG Y Z , TANG J , et al . A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs [C ] // Proceedings of the 2017 IEEE International Conference on Communications (ICC) . Piscataway : IEEE Press , 2017 : 1 - 6 .

ZHAO N , LIANG Y C , NIYATO D , et al . Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5141 - 5152 .

NASIR Y S , GUO D N . Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks [J ] . IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 10 ): 2239 - 2250 .

NAPARSTEK O , COHEN K . Deep multi-user reinforcement learning for distributed dynamic spectrum access [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 1 ): 310 - 323 .

WANG X M , ZHANG Y H , SHEN R J , et al . DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 8 ): 7279 - 7294 .

DU J B , XU W , ZHAO C M , et al . Weighted spectral efficiency optimization for hybrid beamforming in multiuser massive MIMO-OFDM systems [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 10 ): 9698 - 9712 .

3GPP. 5G: study on channel model for frequencies from 0.5 to 100 GHz: TR 38.901 [S ] . 2020 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Performance analysis of high-rate generalized space code index modulation system

Research on dynamic allocation of network slicing resources based on OS-MBRL

Adaptive distributed cloud edge collaborative load control strategy for load management

Joint optimization algorithm for 6G network task offloading and fine-grained slice resource scheduling

Task urgency-based resource allocation algorithm in industrial Internet of things

Related Author

BIAN Lina

ZHANG Qianyu

WU Jiayan

YAN Jiahui

ZHONG Weixuan

DONG Ligang

JIANG Xian

WANG Guangchang

Related Institution

Keyi College of Zhejiang Sci-Tech University

UT Starcom Telecom Co., Ltd.

Zhejiang Gongshang University

Beijing FibrLINK Communications Co., Ltd.

Key Laboratory of Smart Grid of Ministry of Education, Tianjin University

⁰