基于CBR与合作Q学习的分布式CRN资源分配算法

徐琳; 赵知劲

doi:10.11959/j.issn.1000-0801.2019005

您当前的位置：

首页 >

文章列表页 >

基于CBR与合作Q学习的分布式CRN资源分配算法

研究与开发 | 更新时间：2024-06-05

- 基于CBR与合作Q学习的分布式CRN资源分配算法
- A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning
- 电信科学 2019年35卷第2期页码：35-42
- 作者机构：
- 作者简介：
  
  [ "徐琳（1994- ），女，杭州电子科技大学通信工程学院硕士生，主要研究方向为认知无线电、信号处理。" ]
  [ "赵知劲（1959- ），女，博士，杭州电子科技大学教授、博士生导师，主要研究方向为认知无线电、通信信号处理和自适应信号处理等。" ]
- 基金信息：
  
  “十二五”国防预研项目;The National Defense Pre-Research Project During the Twelfth Five-year Plan Period(41001010401)
- DOI：10.11959/j.issn.1000-0801.2019005
  中图分类号： TN925
- 网络出版日期：2019-02，
  
  纸质出版日期：2019-02-20
- 稿件说明：
移动端阅览
徐琳, 赵知劲. 基于CBR与合作Q学习的分布式CRN资源分配算法[J]. 电信科学, 2019,35(2):35-42.

Lin XU, Zhijin ZHAO. A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning[J]. Telecommunications science, 2019, 35(2): 35-42.
徐琳, 赵知劲. 基于CBR与合作Q学习的分布式CRN资源分配算法[J]. 电信科学, 2019,35(2):35-42. DOI： 10.11959/j.issn.1000-0801.2019005.

Lin XU, Zhijin ZHAO. A distributed CRN resource allocation algorithm based on CBR and cooperative Q-learning[J]. Telecommunications science, 2019, 35(2): 35-42. DOI： 10.11959/j.issn.1000-0801.2019005.

摘要

针对分布式认知无线电网络的信道和功率分配问题，提出一种基于案例推理与合作的Q学习算法。为了优化Q学习算法的Q初始化，将当前问题和历史案例依据相似度函数进行匹配，提取匹配案例的Q值并在归一化后作为初始值，进行合作Q学习。合作Q学习是基于总奖赏值进行的，各Agent以不同权值融合其他具有更高奖赏值的Agent的Q值来获取学习经验，以减少不必要的探索。仿真结果表明，该算法提高了认知系统信道和功率分配的能量效率，加快了系统的收敛速度。

Abstract

In order to solve the problem of channel and power allocation in distributed cognitive radio networks (CRN)

a case-based reasoning (CBR) and cooperative Q-learning algorithm was proposed.In order to optimize the Q initialization of Q-learning algorithm

the current problem and the historical case were matched according to the similarity function

the Q value of the matching case was extracted and normalized as the initial value.Cooperative Q-learning was based on the total reward value

and each agent integrates the Q values of other agents with higher reward values with different weights to gain learning experience to reduce unnecessary exploration.Simulations show that the proposed algorithm can improve the energy efficiency of the cognitive system’s channel and power allocation

and accelerate the convergence speed of the system.

关键词

Keywords

references

罗骥 . 基于自适应中继的认知协作网络功率分配 [J ] . 计算机工程 , 2017 , 43 ( 7 ): 151 - 155 .

LUO J . Cognitive cooperative network power allocation based on adaptive relay [J ] . Computer Engineering , 2017 , 43 ( 7 ): 151 - 155 .

HE J , PENG J , JIANG F , et al . A distributed Q learning spectrum decision scheme for cognitive radio sensor network [J ] . International Journal of Distributed Sensor Networks , 2015 : 1 - 10 .

MOROZS N , CLARKE T , GRACE D . Distributed heuristically accelerated Q-Learning for robust cognitive spectrum management in LTE cellular systems [J ] . IEEE Transactions on Mobile Computing , 2016 , 15 ( 4 ): 817 - 825 .

李晓静 . 基于强化学习的动态频谱分配算法研究 [D ] . 南京:南京邮电大学 , 2011 .

LI X J . The study of the dynamic spectrum allocation algorithm based on reinforcement learning [D ] . Nanjing:Nanjing University of Posts and Telecommunications , 2011 .

BOUMEDIENE L , ZGEN G G , LIU S . Distributed multi-agent Q-learning for joint channel allocation and power control in cognitive radio networks [J ] . Journal of Computational Information Systems , 2012 , 8 ( 17 ): 7071 - 7078 .

MOROZS N , CLARKE T , GRACE D . Distributed Q-learning based dynamic spectrum management in cognitive cellular systems:choosing the right learning rate [C ] // 2014 IEEE Symposium on Computers and Communication,June 23-26,2014,Funchal,Portugal . Piscataway:IEEE Press , 2014 : 1 - 10 .

CHO B , KIM K J , CHUNG J W . CBR-based network performance management with multi-agent approach [J ] . Cluster Computing , 2017 , 20 ( 1 ): 1 - 11 .

张文柱 , 周雪婷 , 刘玉琦 . 认知无线电 NC-OFDM 中基于案例推理的无线资源分配 [J ] . 移动通信 , 2017 , 41 ( 14 ): 82 - 88 .

ZHANG W Z , ZHOU X T , LIU Y Q . Radio resource allocation based on case-reasoning in NC-OFDM systems for cognitive radio [J ] . Mobile Communication , 2017 , 41 ( 14 ): 82 - 88 .

冯陈伟 , 袁江南 . 基于强化学习的异构无线网络资源管理算法 [J ] . 电信科学 , 2015 , 31 ( 8 ): 99 - 106 .

FENG C W , YUAN J N . Heterogeneous wireless network resource management algorithm based on reinforcement learning [J ] . Telecommunications Science , 2015 , 31 ( 8 ): 99 - 106 .

CHI C , ZHANG Q Z , BO X et al . A multi-agent reinforcement learning algorithm based on Stackelberg game [C ] // IEEE 6th Data Driven Control and Learning Systems Conference,May 26-27,2017,Chongqing,China . Piscataway:IEEE Press , 2017 : 727 - 732 .

赖海超 , 赵知劲 , 郑仕链 . 应用案例推理技术的快速认知引擎 [J ] . 信号处理 , 2012 , 28 ( 12 ): 1700 - 1705 .

LAI H C , ZHAO Z J , ZHENG S L . Fast cognitive engine using case-based reasoning [J ] . Signal Processing , 2012 , 28 ( 12 ): 1700 - 1705 .

LALL S , SADHU A K , AMIT K . Multi-agent reinforcement learning for stochastic power management in cognitive radio network [C ] // International Conference on Microelectronics,Computing and Communications,Jan 23-25,2016,Durgapur,India . Piscataway:IEEE Press , 2016 : 1 - 6 .

曾庆瑾 . 认知无线电动态频谱的分配和接入问题研究 [D ] . 南宁:广西大学 , 2015 .

ZENG Q J . Study on the allocation and access of the dynamic frequency spectrum for cognitive radio [D ] . Nanning:Guangxi University , 2015 .

浏览量

613

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

PLC-RF无线传输网络中的能效最优设计方法

增强型斑马优化算法及其在SDWN功率分配中的应用

高铁场景下有源智能超表面辅助的能效优化研究

基于区块链与深度学习的空间分集协作频谱感知系统

面向SDWSN的分布式高效熵节能分簇路由算法