Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning

Chenwei Feng; Jiangnan Yuan

doi:10.11959/j.issn.1000-0801.2015195

您当前的位置：

首页 >

文章列表页 >

Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning

research and development | 更新时间：2024-06-05

- Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning
- Telecommunications Science Vol. 31, Issue 8, Pages: 99-106(2015)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2015195
  CLC：
- Published Online：2015-08，
  
  Published：20 August 2015
- 稿件说明：
移动端阅览
Chenwei Feng, Jiangnan Yuan. Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning[J]. Telecommunications science, 2015, 31(8): 99-106.
DOI：

Chenwei Feng, Jiangnan Yuan. Heterogeneous Wireless Network Resource Management Algorithm Based on Reinforcement Learning[J]. Telecommunications science, 2015, 31(8): 99-106. DOI： 10.11959/j.issn.1000-0801.2015195.

摘要

为了充分利用各种无线网络的资源，需要实现异构网络的融合，而异构网络的融合又面临接入控制与资源分配的问题。为此，提出一种基于强化学习的异构无线网络资源管理算法，该算法引入 D2D （device-to-device）通信模式，并可以根据终端不同的业务类型、终端移动性及网络负载条件等状态，选择合适的网络接入方式。同时，为降低存储需求，采用神经网络技术解决连续状态空间问题。仿真结果表明，该算法具有高效的在线学习能力，能够有效地提升网络的频谱效用，降低阻塞率，从而实现自主的无线资源管理。

Abstract

In order to make full use of the resources of all kinds of wireless network，the integration of heterogeneous network is necessary.However，when it comes to the heterogeneous network integration，the problems of call request access control and resource management emerge.A reinforcement-learning-based algorithm was presented for heterogeneous wireless network resource management.D2D （device-to-device）communication was introduced into the proposed algorithm and the appropriate network for access could be selected according to different traffic types，terminal mobility，network load status and so on.Meanwhile，to reduce the storage requirement，the neural network technology was used to solve the problem of continuous state space.Simulation results show that the proposed algorithm has an efficient learning ability to achieve autonomous radio resource management，which effectively improves the spectrum utility and reduces the blocking probability.

关键词

Keywords

references

Luo J ， Mukerjee R ， Dillinger M ， et al . Investigation of radio resource scheduling in WLANs coupled with 3G cellular network . IEEE Communications Magazine ， 2003 ， 41 （ 6 ）： 108 ～ 115

3GPP TR 25.881 v5.0.0.Improvement of RRM across RNS and RNS/BSS（Release5） . http://www.3gpp.org http://www.3gpp.org ， 2001

Song Q ， Jamalipour A . Network selection in an integrated wireless LAN and UMTS environment using mathematical modeling and computing techniques . IEEE Wireless Communications ， 2005 ， 12 （ 3 ）： 42 ～ 48

Zhang Y ， Chen J ， Zhang P . Autonomic joint radio resource management in B3G environment using reinforcement learning . Proceedings of the 6th Annual Wireless Telecommunications Symposium（WTS 2007）， Pomona，California，USA ， 2007

Barto A G . Reinforcement Learning：an Introduction . Cambridge ： MIT Press , 1998

Kaelbling L P ， Littman M L ， Moore A W . Reinforcement learning：a survey . Journal of Artificial Intelligence Research ， 1996 （ 4 ）： 237 ～ 285

Nie J ， Haykin S . A Q-learning-based dynamic channel assignment technique for mobile communication systems . IEEE Transactions on Vehicular Technology ， 1999 ， 48 （ 5 ）： 1676 ～ 1687

Senouci S M ， Beylot A L ， Pujolle G . Call admission control in cellular networks：a reinforcement learning solution . International Journal of Network Management ， 2004 ， 14 （ 2 ）： 89 ～ 103

Haddad M ， Altman Z ， Elayoubi S E ， et al . A nash-stackelberg fuzzy Q-learning decision approach in heterogeneous cognitive networks . Proceedings of Global Telecommunications Conference （GLOBECOM 2010）， Miami，Florida，USA ， 2010 ： 1 ～ 6

Simsek M ， Czylwik A . Decentralized Q-learning of LTE-femtocells for interference reduction in heterogeneous networks using cooperation . Proceedings of 2012 International ITG Workshop on Smart Antennas（WSA）， Dresden，Germany ， 2012 ： 86 ～ 91

Saker L ， Ben Jemaa S ， Elayoubi S E . Q-learning for joint access decision in heterogeneous networks . Proceedings of Wireless Communications and Networking Conference ， Budapest，Hungary ， 2009 ： 1 ～ 5

Tabrizi H ， Farhadi G ， Cioffi J . Dynamic handoff decision in heterogeneous wireless systems：Q-learning approach . Proceedings of 2012 IEEE International Conference on Communications （ICC）， Ottawa，Canada ， 2012 ： 3217 ～ 3222

张永靖，冯志勇，张平 . 基于Q学习的自主联合无线资源管理算法 . 电子与信息学报， 2008 ， 30 （ 3 ）： 676 ～ 680

Zhang Y ， Feng Z ， Zhang P . A Q-learning based autonomic join radio resource management algorithm . Journal of Electronics and Information Technology ， 2008 ， 33 （ 3 ）： 676 ～ 680

Zhao Y ， Zhou W ， Zhu Q . Q-learning based heterogeneous network selection algorithm . Proceedings of the 2nd World Congress on Computer Science and Information Engineering ， Changchun，China ， 2011 ： 471 ～ 477

Chen Y H ， Chang C J ， Huang C Y . Fuzzy Q-learning admission control for WCDMA/WLAN heterogeneous networks with multimedia traffic . IEEE Transactions on Mobile Computing ， 2009 ， 8 （ 11 ）： 1469 ～ 1479

谭力，陈亚迷，李一喆等 . 一种基于 Q-学习的认知无线网络接入方案 . 新型工业化， 2011 （ 9 ）： 37 ～ 44

Tan L ， Chen Y ， Li Y ， et al . Q-learning based cognitive wireless network access mechanism . Journal of New Industrialization ， 2011 （ 9 ）： 37 ～ 44

陈旋 . 基于Q学习的LTE/WLAN网络接入控制算法研究（硕士学位论文） . 哈尔滨工业大学， 2013

Chen X . Call admission control in LTE/WLAN network based on Q-learning（master dissertation） . Harbin Institute of Technology ， 2013

Tan X ， Luan X ， Cheng Y ， et al . Cell selection in two-tier femtocell networks using Q-learning algorithm . Proceedings of the 16th International Conference on Advanced Communication Technology ， Pyeongchang，Korea ， 2014 ： 1031 ～ 1035

Doppler K ， Rinne M ， Wijting C ， et al . Device-to-device communication as an underlay to LTE-Advanced networks . IEEE Communications Magazine ， 2009 ， 47 （ 12 ）： 42 ～ 49

Barto A G ， Bradtke S J ， Singh S P . Learning to act using real-time dynamic programming . Artificial Intelligence ， 1995 ， 72 （ 1 ）： 81 ～ 138

Watkins C J C H ， Dayan P . Q-learning . Machine Learning ， 1992 ， 8 （ 3 ～ 4 ）： 279 ～ 292

叶培智 . 异构无线网络接入选择算法的研究（硕士学位论文） . 厦门大学， 2013

Ye P Z . Research on access selection in heterogeneous wireless networks（master dissertation） . Xiamen University ， 2013

Senouci S ， Beylot A ， Pujolle G Call admission control in cellular networks：a reinforcement learning solution . International Journal of Network Management ， 2004 ， 14 （ 2 ）： 89 ～ 103

Views

1278

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Intelligent interference management in 5G wireless networks

Research on dynamic allocation of network slicing resources based on OS-MBRL

Survey on large language models alignment research

Energy efficiency optimization under joint transmittion power and channel allocation for cellular network

Research on deterministic networking technology for space-ground integrated

Related Author

Jiandong LI

Lei LIU

Min SHENG

UChao X

LU Lingrong

WANG Guangchang

JIANG Xian

DONG Ligang

Related Institution

UT Starcom Telecom Co., Ltd.

Zhejiang Gongshang University

ZTE Corporation

State Grid Siji Location based Service Co., Ltd.

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

⁰