机器类通信中集中式与分布式Q学习的资源分配算法研究

余云河; 孙君

doi:10.11959/j.issn.1000-0801.2021244

您当前的位置：

首页 >

文章列表页 >

机器类通信中集中式与分布式Q学习的资源分配算法研究

研究与开发 | 更新时间：2024-06-05

- 机器类通信中集中式与分布式Q学习的资源分配算法研究
- Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication
- 电信科学 2021年37卷第11期页码：41-50
- 作者机构：
- 作者简介：
  
  [ "余云河（1995− ），男，南京邮电大学通信与信息工程学院硕士生，主要研究方向为大规模机器类通信网络中的资源分配" ]
  [ "孙君（1980− ），女，南京邮电大学硕士生导师，主要研究方向为无线网络资源管理" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61771255);省部级重点实验室开放课题项目;Provincial and Ministerial Key Laboratory Open Project(20190904)
- DOI：10.11959/j.issn.1000-0801.2021244
  中图分类号： TP929.5
- 网络出版日期：2021-11，
  
  纸质出版日期：2021-11-20
- 稿件说明：
移动端阅览
余云河, 孙君. 机器类通信中集中式与分布式Q学习的资源分配算法研究[J]. 电信科学, 2021,37(11):41-50.

Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication[J]. Telecommunications science, 2021, 37(11): 41-50.
余云河, 孙君. 机器类通信中集中式与分布式Q学习的资源分配算法研究[J]. 电信科学, 2021,37(11):41-50. DOI： 10.11959/j.issn.1000-0801.2021244.

Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication[J]. Telecommunications science, 2021, 37(11): 41-50. DOI： 10.11959/j.issn.1000-0801.2021244.

摘要

针对海量机器类通信（massive machine type communication，mMTC）场景，以最大化系统吞吐量为目标，且在保证部分机器类通信设备（machine type communication device，MTCD）的服务质量（quality of service，QoS）要求前提下，提出两种基于Q学习的资源分配算法：集中式Q学习算法（team-Q）和分布式Q学习算法（dis-Q）。首先基于余弦相似度（cosine similarity，CS）聚类算法，考虑到MTCD地理位置和多级别QoS要求，构造代表MTCD和数据聚合器（data aggregator，DA）的多维向量，根据向量间CS值完成分组。然后分别利用team-Q学习算法和dis-Q学习算法为MTCD分配资源块（resource block，RB）和功率。吞吐量性能上，team-Q 和 dis-Q 算法相较于动态资源分配算法、贪婪算法分别平均提高了 16%、23%；复杂度性能上，dis-Q算法仅为team-Q算法的25%及以下，收敛速度则提高了近40%。

Abstract

Under the premise of ensuring partial machine type communication device (MTCD)’s quality of service (QoS) requirements

the resource allocation problem was studied with the goal of maximizing system throughput in the massive machine type communication (mMTC) scenario.Two resource allocation algorithms based on Q-learning were proposed: centralized Q-learning algorithm (team-Q) and distributed Q-learning algorithm (dis-Q).Firstly

taking into account MTCD’s geographic location and multi-level QoS requirements

a clustering algorithm based on cosine similarity (CS) was designed.In the clustering algorithm

multi-dimensional vectors that represent MTCD and data aggregator (DA) were constructed

and MTCDs can be grouped according to the CS value between multi-dimensional vectors.Then in the MTC network

the team-Q learning algorithm and dis-Q learning algorithm were used to allocate resource blocks and power for the MTCD.In terms of throughput performance

team-Q and dis-Q algorithms have an average increase of 16% and 23% compared to the dynamic resource allocation algorithm and the greedy algorithm

respectively.In terms of complexity performance

the dis-Q algorithm is only 25% of team-Q algorithm and even below

the convergence speed is increased by nearly 40%.

关键词

Keywords

references

CHEN S Y , MA R F , CHEN H H , et al . Machine-to-machine communications in ultra-dense networks—A survey [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 3 ): 1478 - 1503 .

钱志鸿 , 王义君 . 物联网技术与应用研究 [J ] . 电子学报 , 2012 , 40 ( 5 ): 1023 - 1029 .

QIAN Z H , WANG Y J . IoT technology and application [J ] . Acta Electronica Sinica , 2012 , 40 ( 5 ): 1023 - 1029 .

Service-aware transport network:opportunities and chanenges [J ] . Proceedings of SPIE - The International Society for Optical Engineering , 2005 .

ZHOU Y Q , TIAN L , LIU L , et al . Fog computing enabled future mobile communication networks:a convergence of communication and computing [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 20 - 27 .

Cisco visual networking index:global mobile data traffic forecast update 2014-2019 [EB ] . 2014 .

LIANG L , XU L , CAO B , et al . A cluster-based congestion-mitigating access scheme for massive M2M communications in internet of things [J ] . IEEE Internet of Things Journal , 2018 , 5 ( 3 ): 2200 - 2211 .

GHAVIMI F , LU Y W , CHEN H H . Uplink scheduling and power allocation for M2M communications in SC-FDMA-based LTE-A networks with QoS guarantees [J ] . IEEE Transactions on Vehicular Technology , 2017 , 66 ( 7 ): 6160 - 6170 .

GAO H , XU X D , HAN S J . Homogeneous clustering algorithm based on average residual energy for energy-efficient MTC networks [C ] // Proceedings of 2018 24th Asia-Pacific Conference on Communications (APCC) . Piscataway:IEEE Press , 2018 : 28 - 33 .

HUSSAIN F , HUSSAIN R , ANPALAGAN A , et al . A new block-based reinforcement learning approach for distributed resource allocation in clustered IoT networks [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 3 ): 2891 - 2904 .

XU Y Q , FENG G , LIANG L , et al . MTC data aggregation for 5G network slicing [C ] // Proceedings of 2017 23rd Asia-Pacific Conference on Communications (APCC) . Piscataway:IEEE Press , 2017 : 1 - 6 .

王鑫 , 邱玲 . H2H与M2M共存场景的准入控制及资源分配 [J ] . 中国科学院大学学报 , 2016 , 33 ( 3 ): 427 - 432 .

WANG X , QIU L . Admission control and resource allocation of H2H ＆ M2M co-existence scenario [J ] . Journal of University of Chinese Academy of Sciences , 2016 , 33 ( 3 ): 427 - 432 .

蒋继胜 , 朱晓荣 . H2H 与 M2M 共存场景下的上行资源分配算法 [J ] . 电子学报 , 2018 , 46 ( 5 ): 1259 - 1264 .

JIANG J S , ZHU X R . An uplink resource allocation algorithm under the scenario of coexistence of H2H ＆ M2M based on knapsack model [J ] . Acta Electronica Sinica , 2018 , 46 ( 5 ): 1259 - 1264 .

SALAM T , REHMAN W U , TAO X F . Cooperative data aggregation and dynamic resource allocation for massive machine type communication [J ] . IEEE Access , 2018 , 6 : 4145 - 4158 .

郭涛 , 李有明 , 雷鹏 , 等 . MIMO 中继系统中一种基于用户QoS的资源分配方法 [J ] . 电信科学 , 2015 , 31 ( 4 ): 121 - 126 .

GUO T , LI Y M , LEI P , et al . A resource allocation scheme based on user’s QoS in MIMO relay system [J ] . Telecommunications Science , 2015 , 31 ( 4 ): 121 - 126 .

张海波 , 向煜 , 刘开健 , 等 . 基于D2D通信的V2X资源分配方案 [J ] . 北京邮电大学学报 , 2017 , 40 ( 5 ): 92 - 97 .

ZHANG H B , XIANG Y , LIU K J , et al . V2X resource allocation scheme based on D2D communication [J ] . Journal of Beijing University of Posts and Telecommunications , 2017 , 40 ( 5 ): 92 - 97 .

刘佳言 , 秦鹏 , 赵雄文 , 等 . 基于容量最大化的 mMTC 场景的资源分配问题研究 [J ] . 电力信息与通信技术 , 2020 , 18 ( 12 ): 17 - 22 .

LIU J Y , QIN P , ZHAO X W , et al . Research on resource allocation of m MTC scenario based on capacity maximization [J ] . Electric Power Information and Communication Technology , 2020 , 18 ( 12 ): 17 - 22 .

SHARMA S K , WANG X B . Toward massive machine type communications in ultra-dense cellular IoT networks:current issues and machine learning-assisted solutions [J ] . IEEE Communications Surveys ＆ Tutorials , 2020 , 22 ( 1 ): 426 - 471 .

HUSSAIN F , HASSAN S A , HUSSAIN R , et al . Machine learning for resource management in cellular and IoT networks:potentials,current solutions,and open challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2020 , 22 ( 2 ): 1251 - 1275 .

NIKOPOUR H , BALIGH H . Sparse code multiple access [C ] // Proceedings of 2013 IEEE 24th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications (PIMRC) . Piscataway:IEEE Press , 2013 : 332 - 336 .

KAI C H , LI H , XU L , et al . Joint subcarrier assignment with power allocation for sum rate maximization of D2D communications in wireless cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 5 ): 4748 - 4759 .

浏览量

114

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于OS-MBRL的网络切片资源动态分配方法研究

基于深度强化学习的毫米波大规模MIMO系统资源联合优化

面向多元可控负荷调控的云边协同负荷资源分配策略

工业物联网中基于任务紧急程度的资源分配算法

面向温控负荷聚合调控的云边端网络资源分配