Intelligent deployment and optimization of movable base stations based on multi-agent reinforcement learning

ZHAO Xinran; CHEN Meijuan; YUAN Zhiwei; ZHU Xiaorong

doi:10.11959/j.issn.1000-0801.2025035

您当前的位置：

首页 >

文章列表页 >

Intelligent deployment and optimization of movable base stations based on multi-agent reinforcement learning

Research and Development | 更新时间：2025-03-11

- Intelligent deployment and optimization of movable base stations based on multi-agent reinforcement learning
- Telecommunications Science Vol. 41, Issue 2, Pages: 68-83(2025)
- 作者机构：
  
  南京邮电大学通信与信息工程学院，江苏南京 210003
- 作者简介：
- 基金信息：
  
  Jiangsu Key Research and Development Program of China(BE2021013-2)
- DOI：10.11959/j.issn.1000-0801.2025035
  CLC： TN925
- Received：03 October 2024，
  
  Revised：2025-01-27，
  
  Published：20 February 2025
- 稿件说明：
移动端阅览
赵欣然,陈美娟,袁志伟等.基于多智能体强化学习的可移动基站智能规划与优化[J].电信科学,2025,41(02):68-83.

ZHAO Xinran,CHEN Meijuan,YUAN Zhiwei,et al.Intelligent deployment and optimization of movable base stations based on multi-agent reinforcement learning[J].Telecommunications Science,2025,41(02):68-83.
赵欣然,陈美娟,袁志伟等.基于多智能体强化学习的可移动基站智能规划与优化[J].电信科学,2025,41(02):68-83. DOI： 10.11959/j.issn.1000-0801.2025035.

ZHAO Xinran,CHEN Meijuan,YUAN Zhiwei,et al.Intelligent deployment and optimization of movable base stations based on multi-agent reinforcement learning[J].Telecommunications Science,2025,41(02):68-83. DOI： 10.11959/j.issn.1000-0801.2025035.

摘要

为了在城市环境中快速部署可移动基站并实现运维优化，针对终端用户移动带来的网络覆盖率下降问题与密集部署基站带来的干扰问题，提出了一种基于多智能体强化学习的网络覆盖规划与优化方法。在部署阶段，使用粒子群与果蝇混合优化算法，在建站成本最小化的情况下确定基站最优站址；在运维阶段，设计了多智能体深度确定性策略梯度算法与轻量级梯度提升机算法的联合优化算法，根据终端接收信号强度优化站址，在性能指标仍无法达到要求时，能自动在合适位置新增基站。仿真结果表明，所提出的站址规划算法在覆盖率与服务率方面均优于传统启发式算法；所设计的联合运维优化算法在网络覆盖率恢复能力方面优于传统

均值（

-means）聚类算法，并且能适应更多场景。

Abstract

To enable the rapid deployment of mobile base stations and optimize operations in urban environments

a network coverage planning and optimization method based on multi-agent reinforcement learning was proposed. This method was designed to address the issue of reducing network coverage due to user mobility and the interference caused by densely deployed base stations. During the deployment phase

a hybrid optimization algorithm combining parti

cle swarm and fruit fly optimization was employed to determine the optimal base station locations while minimizing construction costs. In the operational phase

a joint optimization algorithm featuring multi-agent deep deterministic policy gradient and lightweight gradient boosting algorithms was designed to optimize base station locations based on terminal signal strength. Additionally

when performance indicators failed to meet requirements

new base stations were automatically added in suitable locations. Simulation results demonstrate that the proposed algorithm outperforms traditional heuristic algorithms in terms of coverage and service rates

while the designed joint operational optimization algorithm shows superior recovery capability in network coverage compared to the traditional

-means clustering algorithm

adapting to a wider range of scenarios.

关键词

Keywords

references

KARVOUNAS D , VLACHEAS P , GEORGAKOPOULOS A , et al . An opportunistic approach for coverage and capacity optimization in Self-Organizing Networks [C ] // Proceedings of the 2013 Future Network & Mobile Summit . Piscataway : IEEE Press , 2013 : 1 - 10 .

VATSH I , GUPTA V , BHATTACHARYYA B . Optimizing base station deployment for LTE using metaheuristic algorithms [C ] // Proceedings of the 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN) . Piscataway : IEEE Press , 2019 : 1 - 5 .

SHI G L , QIU D X , HU Q L . Three-dimensional position deployment of UAV base stations based on improved grey wolf optimization algorithm [C ] // Proceedings of the 2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI) . Piscataway : IEEE Press , 2024 : 997 - 1002 .

马力鹏 , 冀涵叶 . 改进遗传算法在站址与小区工参联合优化中的应用 [J ] . 移动通信 , 2023 , 47 ( 5 ): 76 - 82 .

MA L P , JI H Y . Application of improved genetic algorithm in joint optimization of station location and community engineering parameters [J ] . Mobile Communications , 2023 , 47 ( 5 ): 76 - 82 .

WANG Y , ZHU X R . A novel network planning algorithm of three-dimensional dense networks based on adaptive variable-length particle swarm optimization [J ] . IEEE Access , 2019 , 7 : 45940 - 45950 .

HANH N T , BINH H T T , TRUONG V Q , et al . Node placement optimization under Q-Coverage and Q-Connectivity constraints in wireless sensor networks [J ] . Journal of Network and Computer Applications , 2023 , 212 : 103578 .

YU G J , YEH K Y . A k -means based small cell deployment algorithm for wireless access networks [C ] // Proceedings of the 2016 International Conference on Networking and Network Applications (NaNA) . Piscataway : IEEE Press , 2016 : 393 - 398 .

GHAZZAI H , YAACOUB E , ALOUINI M S , et al . Optimized LTE cell planning with varying spatial and temporal user densities [J ] . IEEE Transactions on Vehicular Technology , 2016 , 65 ( 3 ): 1575 - 1589 .

朱晓荣 , 沈瑶 . 基于数据挖掘的RPMA低功耗广域网网络规划方法 [J ] . 通信学报 , 2019 , 40 ( 3 ): 28 - 35 .

ZHU X R , SHEN Y . RPMA low-power wide-area network planning method based on data mining [J ] . Journal on Communications , 2019 , 40 ( 3 ): 28 - 35 .

LU F X , MI Z C , ZHAO N , et al . 3D deployment of dynamic UAV base station based on mobile users [C ] // Proceedings of the 2021 International Conference on Advanced Computing and Endogenous Security . Piscataway : IEEE Press , 2022 : 1 - 5 .

KIM T Y , LEE J , KIM J H . Deep reinforcement learning-based full-duplex communication UAV base station trajectory optimization in disaster environments [C ] // Proceedings of the 2023 VTS Asia Pacific Wireless Communications Symposium (APWCS) . Piscataway : IEEE Press , 2023 : 1 - 5 .

LEE H , EOM C . LEE C. QoS-Aware UAV-BS deployment optimization based on reinforcement Learning [C ] // Proceedings of the 2023 International Conference on Electronics, Information, and Communication (ICEIC) . Piscataway : IEEE Press , 2023 : 1 - 4 .

张尚伟 , 和思梦 . 空地网络资源分配与无人机基站动态部署算法 [J ] . 西安交通大学学报 , 2024 , 58 ( 3 ): 172 - 182 .

ZHANG S W , HE S M . Resource allocation of air-ground network and dynamic deployment algorithm of UAV base station [J ] . Journal of Xi’an Jiaotong University , 2024 , 58 ( 3 ): 172 - 182 .

黄标 , 彭木根 . 无线网络规划与优化导论 [M ] . 北京 : 北京邮电大学出版社 , 2011 .

HUANG B , PENG M G . Introduction to wireless network planning and optimization [M ] . Beijing ： Beijing University of Posts and Telecommunications Press , 2011 .

杨燕玲 . LTE移动网络规划与优化 [M ] . 北京 : 北京邮电大学出版社 , 2018 .

YANG Y L . LTE mobile network planning and optimization [M ] . Beijing ： Beijing University of Posts and Telecommunications Press , 2018 .

杨光 , 陈锦浩 . 5G移动通信系统的传播模型研究 [J ] . 移动通信 , 2018 , 42 ( 10 ): 28 - 33 .

YANG G , CHEN J H . Research on propagation model for 5G mobile communication systems [J ] . Mobile Communications , 2018 , 42 ( 10 ): 28 - 33 .

杨英杰 . 粒子群算法及其应用研究 [M ] . 北京 : 北京理工大学出版社 , 2017 .

YANG Y J . Particle swarm optimization and its applications [M ] . Beijing ： Beijing Institute of Technology Press , 2017 .

刘娟 , 杨春花 . 粒子群果蝇混合改进算法在基站选址优化问题中的应用 [J ] . 计算机与数字工程 , 2021 , 49 ( 7 ): 1341 - 1345, 1356 .

LIU J , YANG C H . Application of improved hybrid algorithm based on PSO & FOA in base station location planning problem [J ] . Computer & Digital Engineering , 2021 , 49 ( 7 ): 1341 - 1345, 1356 .

许文俊 , 吴思雷 , 王凤玉 , 等 . 基于多智能体强化学习的大规模灾后用户分布式覆盖优化 [J ] . 通信学报 , 2022 , 43 ( 8 ): 1 - 16 .

XU W J , WU S L , WANG F Y , et al . Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning [J ] . Journal on Communications , 2022 , 43 ( 8 ): 1 - 16 .

吴官翰 , 贾维敏 , 赵建伟 , 等 . 基于多智能体强化学习的混合博弈模式下多无人机辅助通信系统设计 [J ] . 电子与信息学报 , 2022 , 44 ( 3 ): 940 - 950 .

WU G H , JIA W M , ZHAO J W , et al . MARL-based design of multi-unmanned aerial vehicle assisted communication system with hybrid gaming mode [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 3 ): 940 - 950 .

张硕伟 , 裴明丽 , 高有利 , 等 . 基于LightGBM算法的MR网络信号预测 [J ] . 邮电设计技术 , 2020 ( 10 ): 21 - 25 .

ZHANG S W , PEI M L , GAO Y L , et al . MR network signal prediction based on LightGBM algorithm [J ] . Designing Techniques of Posts and Telecommunications , 2020 ( 10 ): 21 - 25 .

袁周阳 , 赵伟康 , 吴迪 . 基于UMa和RMa传播模型的5G覆盖性能研究 [J ] . 移动通信 , 2020 , 44 ( 10 ): 1 - 6 .

YUAN Z Y , ZHAO W K , WU D . Research on 5G coverage performance based on UMa and RMa propagation models [J ] . Mobile Communications , 2020 , 44 ( 10 ): 1 - 6 .

Views

550

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-head graph attention driven feature aggregation for large-scale remote-sening constellation mission planning algorithm

Intelligent resource scheduling scheme for UAV swarm collaborative sensing

Research on optimization of air flow organization in substation room air conditioning based on CFD

Construction scheme analysis of computing force network

Related Author

Xiao Lixia

Zhou Jiaxi

Wei Zixiang

Dai Ao

Zhou Kangyan

She Yuxuan

XIONG Kai

LENG Supeng

Related Institution

School of Cyber Science and Engineering, 6G Research Center, Huazhong University of Science and Technology

School of Electronic Engineering, Xidian University

School of Information and Communication Engineering, University of Electronic Science and Technology of China

Hebei Branch of China Tower Co.,Ltd.

China Academy of Information and Communication Technology

⁰