Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication

Yunhe YU; Jun SUN

doi:10.11959/j.issn.1000-0801.2021244

您当前的位置：

首页 >

文章列表页 >

Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication

Research and Development | 更新时间：2024-06-05

- Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication
- Telecommunications Science Vol. 37, Issue 11, Pages: 41-50(2021)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2021244
  CLC： TP929.5
- Published Online：2021-11，
  
  Published：20 November 2021
- 稿件说明：
移动端阅览
Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication[J]. Telecommunications science, 2021, 37(11): 41-50.
DOI：

Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication[J]. Telecommunications science, 2021, 37(11): 41-50. DOI： 10.11959/j.issn.1000-0801.2021244.

摘要

针对海量机器类通信（massive machine type communication，mMTC）场景，以最大化系统吞吐量为目标，且在保证部分机器类通信设备（machine type communication device，MTCD）的服务质量（quality of service，QoS）要求前提下，提出两种基于Q学习的资源分配算法：集中式Q学习算法（team-Q）和分布式Q学习算法（dis-Q）。首先基于余弦相似度（cosine similarity，CS）聚类算法，考虑到MTCD地理位置和多级别QoS要求，构造代表MTCD和数据聚合器（data aggregator，DA）的多维向量，根据向量间CS值完成分组。然后分别利用team-Q学习算法和dis-Q学习算法为MTCD分配资源块（resource block，RB）和功率。吞吐量性能上，team-Q 和 dis-Q 算法相较于动态资源分配算法、贪婪算法分别平均提高了 16%、23%；复杂度性能上，dis-Q算法仅为team-Q算法的25%及以下，收敛速度则提高了近40%。

Abstract

Under the premise of ensuring partial machine type communication device (MTCD)’s quality of service (QoS) requirements

the resource allocation problem was studied with the goal of maximizing system throughput in the massive machine type communication (mMTC) scenario.Two resource allocation algorithms based on Q-learning were proposed: centralized Q-learning algorithm (team-Q) and distributed Q-learning algorithm (dis-Q).Firstly

taking into account MTCD’s geographic location and multi-level QoS requirements

a clustering algorithm based on cosine similarity (CS) was designed.In the clustering algorithm

multi-dimensional vectors that represent MTCD and data aggregator (DA) were constructed

and MTCDs can be grouped according to the CS value between multi-dimensional vectors.Then in the MTC network

the team-Q learning algorithm and dis-Q learning algorithm were used to allocate resource blocks and power for the MTCD.In terms of throughput performance

team-Q and dis-Q algorithms have an average increase of 16% and 23% compared to the dynamic resource allocation algorithm and the greedy algorithm

respectively.In terms of complexity performance

the dis-Q algorithm is only 25% of team-Q algorithm and even below

the convergence speed is increased by nearly 40%.

关键词

Keywords

references

CHEN S Y , MA R F , CHEN H H , et al . Machine-to-machine communications in ultra-dense networks—A survey [J ] . IEEE Communications Surveys ＆ Tutorials , 2017 , 19 ( 3 ): 1478 - 1503 .

钱志鸿 , 王义君 . 物联网技术与应用研究 [J ] . 电子学报 , 2012 , 40 ( 5 ): 1023 - 1029 .

QIAN Z H , WANG Y J . IoT technology and application [J ] . Acta Electronica Sinica , 2012 , 40 ( 5 ): 1023 - 1029 .

Service-aware transport network:opportunities and chanenges [J ] . Proceedings of SPIE - The International Society for Optical Engineering , 2005 .

ZHOU Y Q , TIAN L , LIU L , et al . Fog computing enabled future mobile communication networks:a convergence of communication and computing [J ] . IEEE Communications Magazine , 2019 , 57 ( 5 ): 20 - 27 .

Cisco visual networking index:global mobile data traffic forecast update 2014-2019 [EB ] . 2014 .

LIANG L , XU L , CAO B , et al . A cluster-based congestion-mitigating access scheme for massive M2M communications in internet of things [J ] . IEEE Internet of Things Journal , 2018 , 5 ( 3 ): 2200 - 2211 .

GHAVIMI F , LU Y W , CHEN H H . Uplink scheduling and power allocation for M2M communications in SC-FDMA-based LTE-A networks with QoS guarantees [J ] . IEEE Transactions on Vehicular Technology , 2017 , 66 ( 7 ): 6160 - 6170 .

GAO H , XU X D , HAN S J . Homogeneous clustering algorithm based on average residual energy for energy-efficient MTC networks [C ] // Proceedings of 2018 24th Asia-Pacific Conference on Communications (APCC) . Piscataway:IEEE Press , 2018 : 28 - 33 .

HUSSAIN F , HUSSAIN R , ANPALAGAN A , et al . A new block-based reinforcement learning approach for distributed resource allocation in clustered IoT networks [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 3 ): 2891 - 2904 .

XU Y Q , FENG G , LIANG L , et al . MTC data aggregation for 5G network slicing [C ] // Proceedings of 2017 23rd Asia-Pacific Conference on Communications (APCC) . Piscataway:IEEE Press , 2017 : 1 - 6 .

王鑫 , 邱玲 . H2H与M2M共存场景的准入控制及资源分配 [J ] . 中国科学院大学学报 , 2016 , 33 ( 3 ): 427 - 432 .

WANG X , QIU L . Admission control and resource allocation of H2H ＆ M2M co-existence scenario [J ] . Journal of University of Chinese Academy of Sciences , 2016 , 33 ( 3 ): 427 - 432 .

蒋继胜 , 朱晓荣 . H2H 与 M2M 共存场景下的上行资源分配算法 [J ] . 电子学报 , 2018 , 46 ( 5 ): 1259 - 1264 .

JIANG J S , ZHU X R . An uplink resource allocation algorithm under the scenario of coexistence of H2H ＆ M2M based on knapsack model [J ] . Acta Electronica Sinica , 2018 , 46 ( 5 ): 1259 - 1264 .

SALAM T , REHMAN W U , TAO X F . Cooperative data aggregation and dynamic resource allocation for massive machine type communication [J ] . IEEE Access , 2018 , 6 : 4145 - 4158 .

郭涛 , 李有明 , 雷鹏 , 等 . MIMO 中继系统中一种基于用户QoS的资源分配方法 [J ] . 电信科学 , 2015 , 31 ( 4 ): 121 - 126 .

GUO T , LI Y M , LEI P , et al . A resource allocation scheme based on user’s QoS in MIMO relay system [J ] . Telecommunications Science , 2015 , 31 ( 4 ): 121 - 126 .

张海波 , 向煜 , 刘开健 , 等 . 基于D2D通信的V2X资源分配方案 [J ] . 北京邮电大学学报 , 2017 , 40 ( 5 ): 92 - 97 .

ZHANG H B , XIANG Y , LIU K J , et al . V2X resource allocation scheme based on D2D communication [J ] . Journal of Beijing University of Posts and Telecommunications , 2017 , 40 ( 5 ): 92 - 97 .

刘佳言 , 秦鹏 , 赵雄文 , 等 . 基于容量最大化的 mMTC 场景的资源分配问题研究 [J ] . 电力信息与通信技术 , 2020 , 18 ( 12 ): 17 - 22 .

LIU J Y , QIN P , ZHAO X W , et al . Research on resource allocation of m MTC scenario based on capacity maximization [J ] . Electric Power Information and Communication Technology , 2020 , 18 ( 12 ): 17 - 22 .

SHARMA S K , WANG X B . Toward massive machine type communications in ultra-dense cellular IoT networks:current issues and machine learning-assisted solutions [J ] . IEEE Communications Surveys ＆ Tutorials , 2020 , 22 ( 1 ): 426 - 471 .

HUSSAIN F , HASSAN S A , HUSSAIN R , et al . Machine learning for resource management in cellular and IoT networks:potentials,current solutions,and open challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2020 , 22 ( 2 ): 1251 - 1275 .

NIKOPOUR H , BALIGH H . Sparse code multiple access [C ] // Proceedings of 2013 IEEE 24th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications (PIMRC) . Piscataway:IEEE Press , 2013 : 332 - 336 .

KAI C H , LI H , XU L , et al . Joint subcarrier assignment with power allocation for sum rate maximization of D2D communications in wireless cellular networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 5 ): 4748 - 4759 .

Views

114

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on dynamic allocation of network slicing resources based on OS-MBRL

Deep reinforcement learning-based resource joint optimization for millimeter-wave massive MIMO systems

Adaptive distributed cloud edge collaborative load control strategy for load management

Task urgency-based resource allocation algorithm in industrial Internet of things

Cloud edge end network resource allocation for thermostatically controlled load aggregation regulation

Related Author

YAN Jiahui

ZHONG Weixuan

DONG Ligang

JIANG Xian

WANG Guangchang

LU Lingrong

LIU Qingli

LI Xiaoyu

Related Institution

UT Starcom Telecom Co., Ltd.

Zhejiang Gongshang University

Key Laboratory of Communication and Network, Dalian University

School of Information Engineering, Dalian University

Beijing FibrLINK Communications Co., Ltd.

⁰