A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning

Lin XU; Zhijin ZHAO

doi:10.11959/j.issn.1000-0801.2019005

您当前的位置：

首页 >

文章列表页 >

A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning

research and development | 更新时间：2024-06-05

- A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning
- Telecommunications Science Vol. 35, Issue 2, Pages: 35-42(2019)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2019005
  CLC： TN925
- Published Online：2019-02，
  
  Published：20 February 2019
- 稿件说明：
移动端阅览
Lin XU, Zhijin ZHAO. A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning[J]. Telecommunications science, 2019, 35(2): 35-42.
DOI：

Lin XU, Zhijin ZHAO. A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning[J]. Telecommunications science, 2019, 35(2): 35-42. DOI： 10.11959/j.issn.1000-0801.2019005.

摘要

针对分布式认知无线电网络的信道和功率分配问题，提出一种基于案例推理与合作的Q学习算法。为了优化Q学习算法的Q初始化，将当前问题和历史案例依据相似度函数进行匹配，提取匹配案例的Q值并在归一化后作为初始值，进行合作Q学习。合作Q学习是基于总奖赏值进行的，各Agent以不同权值融合其他具有更高奖赏值的Agent的Q值来获取学习经验，以减少不必要的探索。仿真结果表明，该算法提高了认知系统信道和功率分配的能量效率，加快了系统的收敛速度。

Abstract

In order to solve the problem of channel and power allocation in distributed cognitive radio networks (CRN)

a case-based reasoning (CBR) and cooperative Q-learning algorithm was proposed.In order to optimize the Q initialization of Q-learning algorithm

the current problem and the historical case were matched according to the similarity function

the Q value of the matching case was extracted and normalized as the initial value.Cooperative Q-learning was based on the total reward value

and each agent integrates the Q values of other agents with higher reward values with different weights to gain learning experience to reduce unnecessary exploration.Simulations show that the proposed algorithm can improve the energy efficiency of the cognitive system’s channel and power allocation

and accelerate the convergence speed of the system.

关键词

Keywords

references

罗骥 . 基于自适应中继的认知协作网络功率分配 [J ] . 计算机工程 , 2017 , 43 ( 7 ): 151 - 155 .

LUO J . Cognitive cooperative network power allocation based on adaptive relay [J ] . Computer Engineering , 2017 , 43 ( 7 ): 151 - 155 .

HE J , PENG J , JIANG F , et al . A distributed Q learning spectrum decision scheme for cognitive radio sensor network [J ] . International Journal of Distributed Sensor Networks , 2015 : 1 - 10 .

MOROZS N , CLARKE T , GRACE D . Distributed heuristically accelerated Q-Learning for robust cognitive spectrum management in LTE cellular systems [J ] . IEEE Transactions on Mobile Computing , 2016 , 15 ( 4 ): 817 - 825 .

李晓静 . 基于强化学习的动态频谱分配算法研究 [D ] . 南京:南京邮电大学 , 2011 .

LI X J . The study of the dynamic spectrum allocation algorithm based on reinforcement learning [D ] . Nanjing:Nanjing University of Posts and Telecommunications , 2011 .

BOUMEDIENE L , ZGEN G G , LIU S . Distributed multi-agent Q-learning for joint channel allocation and power control in cognitive radio networks [J ] . Journal of Computational Information Systems , 2012 , 8 ( 17 ): 7071 - 7078 .

MOROZS N , CLARKE T , GRACE D . Distributed Q-learning based dynamic spectrum management in cognitive cellular systems:choosing the right learning rate [C ] // 2014 IEEE Symposium on Computers and Communication,June 23-26,2014,Funchal,Portugal . Piscataway:IEEE Press , 2014 : 1 - 10 .

CHO B , KIM K J , CHUNG J W . CBR-based network performance management with multi-agent approach [J ] . Cluster Computing , 2017 , 20 ( 1 ): 1 - 11 .

张文柱 , 周雪婷 , 刘玉琦 . 认知无线电 NC-OFDM 中基于案例推理的无线资源分配 [J ] . 移动通信 , 2017 , 41 ( 14 ): 82 - 88 .

ZHANG W Z , ZHOU X T , LIU Y Q . Radio resource allocation based on case-reasoning in NC-OFDM systems for cognitive radio [J ] . Mobile Communication , 2017 , 41 ( 14 ): 82 - 88 .

冯陈伟 , 袁江南 . 基于强化学习的异构无线网络资源管理算法 [J ] . 电信科学 , 2015 , 31 ( 8 ): 99 - 106 .

FENG C W , YUAN J N . Heterogeneous wireless network resource management algorithm based on reinforcement learning [J ] . Telecommunications Science , 2015 , 31 ( 8 ): 99 - 106 .

CHI C , ZHANG Q Z , BO X et al . A multi-agent reinforcement learning algorithm based on Stackelberg game [C ] // IEEE 6th Data Driven Control and Learning Systems Conference,May 26-27,2017,Chongqing,China . Piscataway:IEEE Press , 2017 : 727 - 732 .

赖海超 , 赵知劲 , 郑仕链 . 应用案例推理技术的快速认知引擎 [J ] . 信号处理 , 2012 , 28 ( 12 ): 1700 - 1705 .

LAI H C , ZHAO Z J , ZHENG S L . Fast cognitive engine using case-based reasoning [J ] . Signal Processing , 2012 , 28 ( 12 ): 1700 - 1705 .

LALL S , SADHU A K , AMIT K . Multi-agent reinforcement learning for stochastic power management in cognitive radio network [C ] // International Conference on Microelectronics,Computing and Communications,Jan 23-25,2016,Durgapur,India . Piscataway:IEEE Press , 2016 : 1 - 6 .

曾庆瑾 . 认知无线电动态频谱的分配和接入问题研究 [D ] . 南宁:广西大学 , 2015 .

ZENG Q J . Study on the allocation and access of the dynamic frequency spectrum for cognitive radio [D ] . Nanning:Guangxi University , 2015 .

Views

613

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Design method of energy efficiency optimization in PLC-RF wireless transmission networks

Enhanced zebra optimization algorithm and applications of power allocation in SDWN

Research on energy efficiency optimization assisted by active reconfigurable intelligent surface in high-speed railway scenarios

A spatial diversity cooperative spectrum sensing system based on blockchain and deep learning

Distributed high-efficiency entropy energy-saving clustering routing algorithm for SDWSN

Related Author

Yueyu JIANG

Haoxin CHENG

Kang WANG

Wenjun DAI

Xiaoyu LIU

LI Jiajia

DONG Ligang

JIANG Xian

Related Institution

Information and Communication Branch of State Grid Changzhou Power Supply Company

School of Communication and Information Engineering, Nanjing University of Posts and Telecommunication

School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University

School of Electrical and Automation Engineering, East China Jiaotong University

The State Radio Monitoring Center

⁰