An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C

Zhiqiang LUO; Wei WANG; Xiaorong ZHU

doi:10.11959/j.issn.1000-0801.2020308

您当前的位置：

首页 >

文章列表页 >

An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C

Research and Development | 更新时间：2024-06-05

- An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C
- Telecommunications Science Vol. 36, Issue 12, Pages: 65-76(2020)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2020308
  CLC： TP3
- Published Online：2020-12，
  
  Published：20 December 2020
- 稿件说明：
移动端阅览
Zhiqiang LUO, Wei WANG, Xiaorong ZHU. An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C[J]. Telecommunications science, 2020, 36(12): 65-76.
DOI：

Zhiqiang LUO, Wei WANG, Xiaorong ZHU. An adaptive video stream transmission control method for wireless heterogeneous networks based on A3C[J]. Telecommunications science, 2020, 36(12): 65-76. DOI： 10.11959/j.issn.1000-0801.2020308.

摘要

比特率自适应（ABR）算法已经成为视频传输中研究的热点之一。然而，由于5G无线异构网络具有信道带宽波动大、不同网络间差异明显等特点，多终端协同的自适应视频流传输面临着巨大挑战。提出了一种基于深度强化学习的自适应视频流传输控制方法。首先，建立了视频流动态规划模型，对传输码率以及分流策略进行联合优化。由于该优化问题的求解依赖于精确的信道估计，这在信道状态动态变化的网络中很难实现。因此，将动态规划问题改进为强化学习任务，并采用A3C算法，动态决策视频码率和分流策略。最后，根据实测的网络数据进行仿真，与传统的优化方法相比，本文所提的方法较好地提高了用户QoE。

Abstract

The adaptive bit rate (ABR) algorithm has become the focus research in video transmission.However

due to the characteristics of 5G wireless heterogeneous networks

such as large fluctuation of channel bandwidth and obvious differences between different networks

the adaptive video stream transmission with multi-terminal cooperation was faced with great challenges.An adaptive video stream transmission control method based on deep reinforcement learning was proposed.First of all

a video stream dynamic programming model was established to jointly optimize the transmission rate and diversion strategy.Since the solution of this optimization problem depended on accurate channel estimation

dynamically changing channel state was difficult to achieve.Therefore

the dynamic programming problem was improved to reinforcement learning task

and the A3C algorithm was used to dynamically determine the video bit rate and diversion strategy.Finally

the simulation was carried out according to the measured network data

and compared with the traditional optimization method

the method proposed better improved the user QoE.

关键词

Keywords

references

Cisco . Cisco visual networking index:global mobile data traffic forecast update 2016–2021 [R ] . 2018 .

SUN L , TIAN H , SUN Q Y , et al . Traffic allocation scheme with cooperation of WWAN and WPAN [J ] . IEEE Communications Letters , 2010 , 14 ( 6 ): 551 - 553 .

AKHSHABI S , BEGEN A C , DOVROLIS C . An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP [C ] // Proceedings of Second ACM SIGMM Conference on Multimedia Systems . New York:ACM Press , 2011 .

SUN Y , YIN X , JIANG J , et al . CS2P:improving video bitrate selection and adaptation with data-driven throughput prediction [C ] // Proceedings of the 2016 ACM SIGCOMM Conference . New York:ACM Press , 2016 .

JIANG J C , VYAS S , HENRY M , et al . CFA:a practical prediction system for video qoe optimization [C ] // Proceedings of USENIX Conference on Networked Systems Design ＆ Implementation.[S.l.:s.n] . 2016 .

WANG C R , TIAN H , MIAO J . Dynamic traffic allocation scheme for optimum distribution in heterogeneous networks [C ] // Proceedings of IEEE Vehicular Technology Conference . Piscataway:IEEE Press , 2011 .

ZHOU B , WANG J , ZOU Z , et al . Bandwidth estimation and rate adaptation in HTTP streaming [C ] // Proceedings of IEEE ICNC . Piscataway:IEEE Press , 2012 : 734 - 738 .

HUANG T Y , JOHARI R , MCKEOWN N , et al . A buffer-basedapproach to rate adaptation:evidence from a large video streaming service [J ] . Computer Communication Review , 2014 , 44 ( 4 ): 187 - 198 .

YIN X , JINDAL A , SEKAR V , et al . A control-theoretic approach for dynamic adaptive video streaming over HTTP [J ] . Computer Communication Review , 2015 , 45 ( 4 ): 325 - 338 .

ZHOU C , LIN C W , ZHANG X , et al . A control-theoretic approach to rate adaption for dash over multiple content distribution servers [J ] . IEEE Transactions on Circuits ＆ Systems for Video Technology , 2014 , 24 ( 4 ): 681 - 694 .

GUO Y , YU R , AN J , et al . Buffer-aware streaming in small scale wireless networks:a deep reinforcement learning approach [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 7 ): 6891 - 6902 .

MAO H , NETRAVALI R , ALIZADEH M . Neural adaptive video streaming with pensieve [C ] // Proceedings of the Conference of the ACM Special Interest Group . New York:ACM Press , 2017 .

ALLMAN M , PAXSON M , BLANTON E . TCP congestion control:RFC5681 [S ] . 2009 .

KIMURA H , KOBAYASHI S . An analysis of actor/critic algorithms using eligibility traces:reinforcement learning with imperfect value function [C ] // Proceedings of the Fifteenth International Conference on Machine Learning (ICML 1998).[S.l.:s.n . ] , 1998 .

MNIH V , . Asynchronous methods for deep reinforcement learning [C ] // Proceedings of International Conference on Machine Learning.[S.l.:s.n] . 2016 .

CHIARIOTTI F , . Online learning adaptation strategy for dash clients [C ] // Proceedings of the 7th International Conference on Multimedia Systems . New York:ACM Press , 2016 .

LI C G , YANG H J , SUN F , et al . Multiuser overhearing for cooperative two-way multiantenna relays [J ] . IEEE Transactions on Vehicular Technology , 2016 , 65 ( 5 ): 3796 - 3802 .

LI C G , ZHANG S , LIU P , et al . Overhearing protocol design exploiting inter-cell interference in cooperative green networks [J ] . IEEE Transactions on Vehicular Technology , 2016 , 65 ( 1 ): 441 - 446 .

董振江 , 张东卓 , 黄成 , 等 . 虚拟现实视频处理与传输技术 [J ] . 电信科学 , 2017 , 33 ( 8 ): 45 - 52 .

DONG Z J , ZHANG D Z , HUANG C , et al . Technology of virtual reality video processing and streaming [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 45 - 52 .

周建同 , 杨海涛 , 刘东 , 等 . 视频编码的技术基础及发展方向 [J ] . 电信科学 , 2017 , 33 ( 8 ): 16 - 25 .

ZHOU J T , YANG H T , LIU D , et al . Trends and technologies of video coding [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 16 - 25 .

马思伟 , 罗法蕾 , 黄铁军 . AVS2视频编码标准技术特色及应用 [J ] . 电信科学 , 2017 , 33 ( 8 ): 3 - 15 .

MA S W , LUO F L , HUANG T J . Kernel technologies and applications of AVS2 video coding standard [J ] . Telecommunications Science , 2017 , 33 ( 8 ): 3 - 15 .

Views

535

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Quality analysis method of Web browsing based on big data

Related Author

Weihua CHENG

Xiaorong HE

Related Institution

No data

⁰