Fast QTMT partition decision based on deep learning

Shuang PENG; Xiaodong WANG; Zongju PENG; Fen CHEN

doi:10.11959/j.issn.1000-0801.2021062

您当前的位置：

首页 >

文章列表页 >

Fast QTMT partition decision based on deep learning

Research and Development | 更新时间：2024-06-05

- Fast QTMT partition decision based on deep learning
- Telecommunications Science Vol. 37, Issue 4, Pages: 73-81(2021)
- 作者机构：
  
  1. 宁波大学信息科学与工程学院，浙江宁波 315211
  2. 重庆理工大学电气与电子学院，重庆 400054
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2021062
  CLC： TP393
- Published Online：2021-04，
  
  Published：20 April 2021
- 稿件说明：
移动端阅览
Shuang PENG, Xiaodong WANG, Zongju PENG, et al. Fast QTMT partition decision based on deep learning[J]. Telecommunications science, 2021, 37(4): 73-81.
DOI：

Shuang PENG, Xiaodong WANG, Zongju PENG, et al. Fast QTMT partition decision based on deep learning[J]. Telecommunications science, 2021, 37(4): 73-81. DOI： 10.11959/j.issn.1000-0801.2021062.

摘要

与之前的编码标准相比，多功能视频编码（versatile video coding，VVC）进一步提高了压缩效率。嵌套多类树的四叉树（quadtree with nested multi-type tree，QTMT）结构是提高编码增益的关键之一，同时极大地增加了编码复杂度。为降低VVC编码复杂度，提出了一种基于深度学习的快速QTMT划分方法。首先，提出了注意力-非对称卷积结构来预测划分模式的概率。然后，基于阈值提出了快速划分模式决策。最后，提出了编码性能与时间的代价函数来求解最优阈值，提出了阈值决策方法。实验表明，算法在不同档次下的时间节省分别为48.62%、52.93%、62.01%，BDBR分别为1.05%、1.33%、2.38%。结果表明，算法的时间节省和率失真性能优于其他快速算法。

Abstract

Compared with the predecessor standards

versatile video coding (VVC) significantly improves compression efficiency by a quadtree with nested multi-type tree (QTMT) structure but at the expense of extremely high coding complexity.To reduce the coding complexity of VVC

a fast QTMT partition method was proposed based on deep learning.Firstly

an attention-asymmetric convolutional neural network was proposed to predict the probability of partition modes.Then

the fast decision of partition modes based on the threshold was proposed.Finally

the cost of coding performance and time was proposed to obtain the optimal threshold

and the threshold decision method was proposed.Experimental results at different levels show that the proposed method achieves an average time saving of 48.62%/52.93%/62.01% with the negligible BDBR of 1.05%/1.33%/2.38%.Such results demonstrate that the proposed method significantly outperforms other state-of-the-art methods.

关键词

Keywords

references

JCT-VC . High efficiency video coding (HEVC) text specification draft 10:JCTVC-L1003 [S ] . 2013 .

JVET . Meeting report of the 10th JVET meeting:JVET-J1000 [S ] . 2018 .

JVET . Algorithm description for versatile video coding and test model 2:JEVT-K1002 [S ] . 2018 .

周芸 , 胡潇 , 郭晓强 . H.266/VVC视频编码图像划分技术研究 [J ] . 广播与电视技术 , 2019 , 46 ( 11 ): 40 - 44 .

ZHOU Y , HU X , GUO X Q . [J ] . Research on image partition technolo-gy in H.266/VVC , 2019 , 46 ( 11 ): 40 - 44 .

JVET . AHG report:test model software development (AHG3):JVET-J0003 [S ] . 2018 .

PAKDAMAN F , ADELIMANESH M A , GABBOUJ M , et al . Complexity analysis of next-generation VVC encoding and decod-ing [EB ] . 2020 .Arxiv:2005.10801.

姚英彪 , 李晓娟 . 基于图像空间相关性与纹理的HEVC块划分快速算法 [J ] . 电信科学 , 2015 , 31 ( 1 ): 38 - 46 .

YAO Y B , LI X J . Fast block partitioning algorithm for HEVC based on spatial correlation and image texture [J ] . Telecommunica-tions Science , 2015 , 31 ( 1 ): 38 - 46 .

KUO Y , CHEN P , LIN H . A spatiotemporal content-based CU size decision algorithm for HEVC [J ] . IEEE Transactions on Broadcast-ing , 2020 , 1 ( 66 ): 100 - 112 .

JAMALI M , COULOMBE S . Fast HEVC intra mode decision base on RDO cost prediction [J ] . IEEE Transactions on Broadcasting , 2019 , 1 ( 65 ): 109 - 122 .

HUANG B , CHEN Z , CAI Q , et al . Rate-distortion-complexity optimized coding mode decision for HEVC [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 3 ( 30 ): 795 - 809 .

LEI M , LUO F , ZHANG X , et al . Look-ahead prediction based coding unit size pruning for VVC intra coding [C ] // Proceedings of IEEE International Conference on Image Processing . Piscataway:IEEE Press , 2019 : 4120 - 4124 .

CHEN J , SUN H , KATTO J , et al . Fast QTMT partition decision algorithm in VVC intra coding based on variance and gradient [C ] // Proceedings of IEEE Visual Communications and Image Processing . Piscataway:IEEE Press , 2019 : 1 - 4 .

FAN Y , CHEN J , SUN H , et al . A fast QTMT partition decision strategy for VVC intra prediction [J ] . IEEE Access , 2020 ( 8 ): 107900 - 107911 .

PARK S , KANG J . Context-based ternary tree decision method in versatile video coding for fast intra coding [J ] . IEEE Access , 2019 ( 7 ): 172597 - 172605 .

PARK S , KANG J . Fast affine motion estimation for versatile video coding (VVC) encoding [J ] . IEEE Access , 2019 ( 7 ): 158075 - 158084 .

LIU X , LI Y , LIU D , et al . An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2019 , 1 ( 29 ): 144 - 155 .

CHEN Z , SHI J , LI W . Learned fast HEVC intra coding [J ] . IEEE Transactions on Image Processing , 2020 ( 29 ): 5431 - 5446 .

KATAYAMA T , KURODA K , SHI W , et al . Low-complexity intra coding algorithm based on convolutional neural network for HEVC [C ] // Proceedings of International Conference on Information and Computer Technologies . Piscataway:IEEE Press , 2018 : 115 - 118 .

KIM K , RO W W . Fast CU depth decision for HEVC using neural networks [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2019 , 5 ( 29 ): 1462 - 1473 .

XU M , LI T Y , WANG Z , et al . Reducing complexity of HEVC:a deep learning approach [J ] . IEEE Transactions on Image Processing , 2018 , 10 ( 27 ): 5044 - 5059 .

TANG G , JING M , ZENG X , et al . Adaptive CU split decision with pooling-variable CNN for VVC intra encoding [C ] // Proceedings of IEEE Visual Communications and Image Processing . Piscataway:IEEE Press , 2019 : 1 - 4 .

YANG H , SHEN L , DONG X , et al . Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 6 ( 30 ): 1668 - 1682 .

FU T , ZHANG H , MU F.Fast CU partitioning algorithm for H . 266/VVC intra-frame coding [C ] // Proceedings of IEEE International Conference on Multimedia and Expo . Piscataway:IEEE Press , 2019 : 55 - 60 .

AMESTOY T , MERCAT A , HAMIDOUCHE W , et al . Tunable VVC frame partitioning based on lightweight machine learning [J ] . IEEE Transactions on Image Processing , 2020 ( 29 ): 1313 - 1328 .

贾川民 , 赵政辉 , 王苫社 , 等 . 基于神经网络的图像视频编码 [J ] . 电信科学 , 2019 , 35 ( 5 ): 32 - 42 .

JIA C M , ZHAO Z H , WANG S S , et al . Neural network based im-age and video coding technologies [J ] . Telecommunications Science , 2019 , 35 ( 5 ): 32 - 42 .

WIECKOWSKI A , MA J , SCHWARZ H , et al . Fast partitioning decision strategies for the upcoming versatile video coding (VVC) standard [C ] // Proceedings of IEEE International Conference on Image Processing . Piscataway:IEEE Press , 2019 : 4130 - 4134 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 7132 - 7141 .

CHEN Y P , DAI X Y , LIU M C . Dynamic convolution:attention over convolution kernels [C ] // Proceedings of IEEE/CVF Conference on Computer Vision and Pattern RecognitionA . Piscataway:IEEE Press , 2020 : 11030 - 11039 .

JVET . Algorithm description for versatile video coding and test model 7:JEVT-P2002 [S ] . 2019 .

JVET . JVET common test conditions and software reference configurations:JEVT-K1010 [S ] . 2018 .

VCEG . Calculation of average PSNR differences between RD curves:VCEG-M33 [S ] . 2001 .

Views

378

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

GMTBLC: a deep learning-based bi-modal network traffic classification method

Research on antenna dip angle recognition method based on improved YOLOv5

Mixed signal modulation recognition method based on temporal depth residual shrinkage network

A VVC intra coding method based on fast partition for coding unit

Research on fault prediction of computer network nodes driven by log information

Related Author

WEI Debin

JIANG Qinlong

WEN Jinglong

WANG Xinrui

ZHANG Ning

PAN Feng

GENG Lujing

CHEN Zuhao

Related Institution

School of Information Engineering, Dalian University

China Mobile Group Design Institute Co., Ltd.

China Mobile Group Co., Ltd.

The 63rd Research Institute, National University of Defense Technology

School of Electronic and Information Engineering, Nanjing University of Information Science and Technology

⁰