基于深度学习的多层级恰可察觉失真预测

徐海峰; 王鸿奎; 殷海兵; 陈楚翘

doi:10.11959/j.issn.1000-0801.2024015

您当前的位置：

首页 >

文章列表页 >

基于深度学习的多层级恰可察觉失真预测

研究与开发 | 更新时间：2024-06-05

- 基于深度学习的多层级恰可察觉失真预测
- Deep learning-based prediction of multi-level just noticeable distortion
- 电信科学 2024年40卷第1期页码：35-47
- 作者机构：
  
  1. 杭州电子科技大学通信工程学院，浙江杭州 310018
  2. 杭州电子科技大学网安学院，浙江杭州 310018
- 作者简介：
  
  [ "徐海峰（1999- ），男，杭州电子科技大学通信工程学院硕士生，主要研究方向为感知视频编码" ]
  [ "王鸿奎（1990- ），男，博士，杭州电子科技大学通信工程学院讲师，主要研究方向为感知视频编码" ]
  [ "殷海兵（1974- ），男，博士，杭州电子科技大学通信工程学院教授，主要研究方向为数字视频编解码" ]
  [ "陈楚翘（1993- ），女，博士，杭州电子科技大学网安学院讲师，主要研究方向为智能信息处理" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(62202134);国家自然科学基金资助项目;The National Natural Science Foundation of China(62031009);国家自然科学基金资助项目;The National Natural Science Foundation of China(61972123);科技部重点研发课题资助项目;Ministry of Science and Technology Key Research and Development Project Funding Program(2023YFB4502800);浙江省“尖兵”“领雁”研发攻关计划项目;Zhejiang Provincial “Pioneer” and“Leading Goose” Research and Development Project(2023C01149);浙江省“尖兵”“领雁”研发攻关计划项目;Zhejiang Provincial “Pioneer” and“Leading Goose” Research and Development Project(2022C01068);浙江省自然科学基金资助项目;The Natural Science Foundation of Zhejiang Province(LDT23F01014F01);浙江省自然科学基金资助项目;The Natural Science Foundation of Zhejiang Province(LDT23F01011F01)
- DOI：10.11959/j.issn.1000-0801.2024015
  中图分类号： TN919
- 网络出版日期：2024-01，
  
  纸质出版日期：2024-01-20
- 稿件说明：
移动端阅览
徐海峰, 王鸿奎, 殷海兵, 等. 基于深度学习的多层级恰可察觉失真预测[J]. 电信科学, 2024,40(1):35-47.

Haifeng XU, Hongkui WANG, Haibing YIN, et al. Deep learning-based prediction of multi-level just noticeable distortion[J]. Telecommunications science, 2024, 40(1): 35-47.
徐海峰, 王鸿奎, 殷海兵, 等. 基于深度学习的多层级恰可察觉失真预测[J]. 电信科学, 2024,40(1):35-47. DOI： 10.11959/j.issn.1000-0801.2024015.

Haifeng XU, Hongkui WANG, Haibing YIN, et al. Deep learning-based prediction of multi-level just noticeable distortion[J]. Telecommunications science, 2024, 40(1): 35-47. DOI： 10.11959/j.issn.1000-0801.2024015.

摘要

视觉恰可察觉失真（just noticeable distortion，JND）直接反映人眼视觉系统对视觉信号噪声的敏感程度，广泛应用于图像和视频处理领域。针对视频 JND 阈值的多层级预测问题，将其转化为用户满意率（satisfied user ratio，SUR）曲线的预测问题，并提出一种基于特征融合的SUR曲线预测模型。该模型主要分为关键帧选择模块、特征提取和融合模块以及SUR分数回归模块。在关键帧选择模块，根据视觉感知机制，提出空时域感知复杂度并以此作为视频关键帧判决指标。在特征提取和融合模块，基于密集残差块（dense residual block，RDB）提出多尺度密集残差网络实现图像特征提取和多尺度融合。实验结果表明，所提出的SUR曲线预测模型在JND阈值预测精度方面整体优于现有模型，且在运行效率上平均降低8.1%的时间成本。同时，该模型还可以用于预测其他层级JND阈值，可直接应用于视频多层级感知编码优化。

Abstract

Visual just noticeable distortion (JND) directly reflects the sensitivity of the human visual system to visual signal noise

and is widely used in image and video processing.Aiming at the multilevel prediction problem of video JND threshold

it was transformed into the prediction problem of satisfied user ratio (SUR) curve

and a feature fusion-based SUR curve prediction model was proposed.The model was mainly divided into key frame extraction module

feature extraction and fusion module

and SUR score regression module.In the key frame extraction module

according to the visual perception mechanism

the spatial-temporal domain perception complexity was proposed and used as the video key frame judgment index.In the feature extraction and fusion module

a multi-scale dense residual network was proposed based on dense residual block (RDB) to realize image feature extraction and multi-scale fusion.The experimental results show that the proposed SUR curve prediction model is overall better than the existing models in terms of JND prediction accuracy and reduces the time cost by 8.1% on average in terms of operational efficiency.Meanwhile

the model can also be used to predict other layers of JND thresholds

which can be directly applied to video multilevel perceptual coding optimization.

关键词

Keywords

references

LIN W S , GHINEA G . Progress and opportunities in modelling just-noticeable difference (JND) for multimedia [J ] . IEEE Transactions on Multimedia , 2021 ( 24 ): 3706 - 3721 .

WU J , LI L , DONG W , et al . Enhanced just noticeable difference model for images with pattern complexity [J ] . IEEE Transactions on Image Processing , 2017 , 26 ( 6 ): 2682 - 2693 .

WANG H , YU L , LIANG J , et al . Hierarchical predictive coding-based JND estimation for image compression [J ] . IEEE Transactions on Image Processing , 2020 ( 30 ): 487 - 500 .

BAE S H , KIM M . A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images [J ] . IEEE Transactions on Image Processing , 2014 , 23 ( 8 ): 3227 - 3240 .

骆琼华 , 王鸿奎 , 殷海兵 , 等 . 基于熵掩蔽的 DCT 域恰可察觉失真模型 [J ] . 电信科学 , 2023 , 39 ( 2 ): 59 - 70 .

LUO Q H , WANG H K , YIN H B , et al . Just noticeable distortion model based on entropy masking in DCT domain [J ] . Telecommunications Science , 2023 , 39 ( 2 ): 59 - 70 .

邢亚芬 , 殷海兵 , 王鸿奎 , 等 . 基于视频时域感知特性的恰可察觉失真模型 [J ] . 电信科学 , 2022 , 38 ( 2 ): 92 - 102 .

XING Y F , YIN H B , WANG H K , et al . Video temporal perception characteristics based just noticeable difference model [J ] . Telecommunications Science , 2022 , 38 ( 2 ): 92 - 102 .

JIN L N , LIN J Y , HU S D , et al . Statistical study on perceived JPEG image quality via MCL-JCI dataset construction and analysis [J ] . Electronic Imaging , 2016 , 28 ( 13 ): 1 - 9 .

WANG H Q , GAN W H , HU S D , et al . MCL-JCV:a JND-based H.264/AVC video quality assessment dataset [C ] // Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP) . Piscataway:IEEE Press , 2016 : 1509 - 1513 .

WANG H Q , KATSAVOUNIDIS I , ZHOU J T , et al . VideoSet:a large-scale compressed video quality dataset based on JND measurement [J ] . Journal of Visual Communication and Image Representation , 2017 ( 46 ): 292 - 302 .

HUANG Q , WANG H Q , LIM S C , et al . Measure and prediction of HEVC perceptually lossy/lossless boundary QP values [C ] // Proceedings of the 2017 Data Compression Conference (DCC) . Piscataway:IEEE Press , 2017 : 42 - 51 .

ZHANG X , YANG C , WANG H , et al . Satisfied-user-ratio modeling for compressed video [J ] . IEEE Transactions on Image Processing , 2020 ( 29 ): 3777 - 3789 .

ZHANG Y , LIU H H , YANG Y , et al . Deep learning based just noticeable difference and perceptual quality prediction models for compressed video [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 3 ): 1197 - 1212 .

LIN J Y , JIN L N , HU S D , et al . Experimental design and analysis of JND test on coded image/video [C ] // SPIE Optical Engineering + Applications.Applications of Digital Image Processing XXXVIII .[S.l.:s.n. ] , 2015 : 324 - 334 .

FAN C L , ZHANG Y , HAMZAOUI R , et al . Learning-based satisfied user ratio prediction for symmetrically and asymmetrically compressed stereoscopic images [J ] . IEEE MultiMedia , 2021 , 28 ( 3 ): 8 - 20 .

LIU X H , CHEN Z H , WANG X , et al . JND-pano:database for just noticeable difference of JPEG compressed panoramic images [C ] // Pacific Rim Conference on Multimedia . Berlin:Springer , 2018 : 458 - 468 .

SHEN X L , NI Z K , YANG W H , et al . A JND dataset based on VVC compressed images [C ] // Proceedings of the 2020 IEEE International Conference on Multimedia ＆ Expo Workshops (ICMEW) . Piscataway:IEEE Press , 2020 : 1 - 6 .

LIN H H , CHEN G G , JENADELEH M , et al . Large-scale crowdsourced subjective assessment of picturewise just noticeable difference [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 9 ): 5859 - 5873 .

TIAN T , WANG H L , ZUO L X , et al . Just noticeable difference level prediction for perceptual image compression [J ] . IEEE Transactions on Broadcasting , 2020 , 66 ( 3 ): 690 - 700 .

LIU H H , ZHANG Y , ZHANG H , et al . Deep learning based picture-wise just noticeable distortion prediction model for image compression [J ] . IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society , 2019 ( 29 ): 641 - 656 .

ITU-T P . 910.Subjective video quality assessment methods for multimedia applications [S ] . Geneva:ITU-T Publications , 2022 .

ZHANG Y , TIAN Y , KONG Y , et al . Residual dense network for image super-resolution [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 2472 - 2481 .

PARK T , LIU M Y , WANG T C , et al . Semantic image synthesis with spatially-adaptive normalization [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 2337 - 2346 .

BAO L , YANG Z , WANG S , et al . Real image denoising based on multi-scale residual dense block and cascaded U-Net with block-connection [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Piscataway:IEEE Press , 2020 : 448 - 449 .

WANG Q , WU B , ZHU P , et al . ECA-Net:Efficient channel attention for deep convolutional neural networks [C ] // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Piscataway:IEEE Press , 2020 : 11534 - 11542 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

GMTBLC：基于深度学习的双模态网络流量分类

基于改进YOLOv5的天线下倾角识别方法研究

基于时序深度残差收缩网络的混叠信号调制识别方法

日志信息驱动的计算机网络节点故障预测研究

基于自注意力机制的大规模MIMO信道状态信息特征向量反馈方法