基于A3C的无线异构网络自适应视频流传输控制方法

罗志强; 王伟; 朱晓荣

doi:10.11959/j.issn.1000-0801.2020308

您当前的位置：

首页 >

文章列表页 >

基于A3C的无线异构网络自适应视频流传输控制方法

研究与开发 | 更新时间：2024-06-05

- 基于A3C的无线异构网络自适应视频流传输控制方法
- An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C
- 电信科学 2020年36卷第12期页码：65-76
- 作者机构：
- 作者简介：
  
  [ "罗志强（1997- ），男，南京邮电大学通信与信息工程学院在读，主要研究方向为无线异构网络视频流自适应传输" ]
  [ "王伟（1997- ），男，南京邮电大学通信与信息工程学院硕士生，主要研究方向为无线异构网络视频流自适应传输" ]
  [ "朱晓荣（1977- ），女，博士，南京邮电大学通信与信息工程学院教授、博士生导师，主要研究方向为下一代无线网络、异构网络" ]
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2020308
  中图分类号： TP3
- 网络出版日期：2020-12，
  
  纸质出版日期：2020-12-20
- 稿件说明：
移动端阅览
罗志强, 王伟, 朱晓荣. 基于A3C的无线异构网络自适应视频流传输控制方法[J]. 电信科学, 2020,36(12):65-76.

Zhiqiang LUO, Wei WANG, Xiaorong ZHU. An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C[J]. Telecommunications science, 2020, 36(12): 65-76.
罗志强, 王伟, 朱晓荣. 基于A3C的无线异构网络自适应视频流传输控制方法[J]. 电信科学, 2020,36(12):65-76. DOI： 10.11959/j.issn.1000-0801.2020308.

Zhiqiang LUO, Wei WANG, Xiaorong ZHU. An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C[J]. Telecommunications science, 2020, 36(12): 65-76. DOI： 10.11959/j.issn.1000-0801.2020308.

摘要

比特率自适应（ABR）算法已经成为视频传输中研究的热点之一。然而，由于5G无线异构网络具有信道带宽波动大、不同网络间差异明显等特点，多终端协同的自适应视频流传输面临着巨大挑战。提出了一种基于深度强化学习的自适应视频流传输控制方法。首先，建立了视频流动态规划模型，对传输码率以及分流策略进行联合优化。由于该优化问题的求解依赖于精确的信道估计，这在信道状态动态变化的网络中很难实现。因此，将动态规划问题改进为强化学习任务，并采用A3C算法，动态决策视频码率和分流策略。最后，根据实测的网络数据进行仿真，与传统的优化方法相比，本文所提的方法较好地提高了用户QoE。

Abstract

The adaptive bit rate (ABR) algorithm has become the focus research in video transmission.However

due to the characteristics of 5G wireless heterogeneous networks

such as large fluctuation of channel bandwidth and obvious differences between different networks

the adaptive video stream transmission with multi-terminal cooperation was faced with great challenges.An adaptive video stream transmission control method based on deep reinforcement learning was proposed.First of all

a video stream dynamic programming model was established to jointly optimize the transmission rate and diversion strategy.Since the solution of this optimization problem depended on accurate channel estimation

dynamically changing channel state was difficult to achieve.Therefore

the dynamic programming problem was improved to reinforcement learning task

and the A3C algorithm was used to dynamically determine the video bit rate and diversion strategy.Finally

the simulation was carried out according to the measured network data

and compared with the traditional optimization method

the method proposed better improved the user QoE.

关键词

Keywords

references

Cisco . Cisco visual networking index:global mobile data traffic forecast update 2016–2021 [R ] . 2018 .

SUN L , TIAN H , SUN Q Y , et al . Traffic allocation scheme with cooperation of WWAN and WPAN [J ] . IEEE Communications Letters , 2010 , 14 ( 6 ): 551 - 553 .

AKHSHABI S , BEGEN A C , DOVROLIS C . An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP [C ] // Proceedings of Second ACM SIGMM Conference on Multimedia Systems . New York:ACM Press , 2011 .

SUN Y , YIN X , JIANG J , et al . CS2P:improving video bitrate selection and adaptation with data-driven throughput prediction [C ] // Proceedings of the 2016 ACM SIGCOMM Conference . New York:ACM Press , 2016 .

JIANG J C , VYAS S , HENRY M , et al . CFA:a practical prediction system for video qoe optimization [C ] // Proceedings of USENIX Conference on Networked Systems Design ＆ Implementation.[S.l.:s.n] . 2016 .

WANG C R , TIAN H , MIAO J . Dynamic traffic allocation scheme for optimum distribution in heterogeneous networks [C ] // Proceedings of IEEE Vehicular Technology Conference . Piscataway:IEEE Press , 2011 .

ZHOU B , WANG J , ZOU Z , et al . Bandwidth estimation and rate adaptation in HTTP streaming [C ] // Proceedings of IEEE ICNC . Piscataway:IEEE Press , 2012 : 734 - 738 .

HUANG T Y , JOHARI R , MCKEOWN N , et al . A buffer-basedapproach to rate adaptation:evidence from a large video streaming service [J ] . Computer Communication Review , 2014 , 44 ( 4 ): 187 - 198 .

YIN X , JINDAL A , SEKAR V , et al . A control-theoretic approach for dynamic adaptive video streaming over HTTP [J ] . Computer Communication Review , 2015 , 45 ( 4 ): 325 - 338 .

ZHOU C , LIN C W , ZHANG X , et al . A control-theoretic approach to rate adaption for dash over multiple content distribution servers [J ] . IEEE Transactions on Circuits ＆ Systems for Video Technology , 2014 , 24 ( 4 ): 681 - 694 .

GUO Y , YU R , AN J , et al . Buffer-aware streaming in small scale wireless networks:a deep reinforcement learning approach [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 7 ): 6891 - 6902 .

MAO H , NETRAVALI R , ALIZADEH M . Neural adaptive video streaming with pensieve [C ] // Proceedings of the Conference of the ACM Special Interest Group . New York:ACM Press , 2017 .

ALLMAN M , PAXSON M , BLANTON E . TCP congestion control:RFC5681 [S ] . 2009 .

KIMURA H , KOBAYASHI S . An analysis of actor/critic algorithms using eligibility traces:reinforcement learning with imperfect value function [C ] // Proceedings of the Fifteenth International Conference on Machine Learning (ICML 1998).[S.l.:s.n . ] , 1998 .

MNIH V , . Asynchronous methods for deep reinforcement learning [C ] // Proceedings of International Conference on Machine Learning.[S.l.:s.n] . 2016 .

CHIARIOTTI F , . Online learning adaptation strategy for dash clients [C ] // Proceedings of the 7th International Conference on Multimedia Systems . New York:ACM Press , 2016 .

LI C G , YANG H J , SUN F , et al . Multiuser overhearing for cooperative two-way multiantenna relays [J ] . IEEE Transactions on Vehicular Technology , 2016 , 65 ( 5 ): 3796 - 3802 .

LI C G , ZHANG S , LIU P , et al . Overhearing protocol design exploiting inter-cell interference in cooperative green networks [J ] . IEEE Transactions on Vehicular Technology , 2016 , 65 ( 1 ): 441 - 446 .

董振江 , 张东卓 , 黄成 , 等 . 虚拟现实视频处理与传输技术 [J ] . 电信科学 , 2017 , 33 ( 8 ): 45 - 52 .

DONG Z J , ZHANG D Z , HUANG C , et al . Technology of virtual reality video processing and streaming [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 45 - 52 .

周建同 , 杨海涛 , 刘东 , 等 . 视频编码的技术基础及发展方向 [J ] . 电信科学 , 2017 , 33 ( 8 ): 16 - 25 .

ZHOU J T , YANG H T , LIU D , et al . Trends and technologies of video coding [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 16 - 25 .

马思伟 , 罗法蕾 , 黄铁军 . AVS2视频编码标准技术特色及应用 [J ] . 电信科学 , 2017 , 33 ( 8 ): 3 - 15 .

MA S W , LUO F L , HUANG T J . Kernel technologies and applications of AVS2 video coding standard [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 3 - 15 .

浏览量

535

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于大数据的网页浏览质差分析方法研究