浏览全部资源
扫码关注微信
Video temporal perception characteristics based just noticeable difference model
- Telecommunications Science Vol. 38, Issue 2, Pages: 92-102(2022)
- 作者机构：
1. 杭州电子科技大学通信工程学院，浙江 杭州 310018
2. 华中科技大学电子信息与通信学院，湖北 武汉 430074
- 作者简介：
- 基金信息：
DOI：10.11959/j.issn.1000-0801.2022030
CLC： TN919-
Published Online：2022-02，
Published：20 February 2022
- 稿件说明：
移动端阅览
Yafen1 XING, Haibing YIN, Hongkui WANG, et al. Video temporal perception characteristics based just noticeable difference model[J]. Telecommunications science, 2022, 38(2): 92-102.
Yafen1 XING, Haibing YIN, Hongkui WANG, et al. Video temporal perception characteristics based just noticeable difference model[J]. Telecommunications science, 2022, 38(2): 92-102. DOI： 10.11959/j.issn.1000-0801.2022030.
摘要
现有的时域恰可察觉失真（just noticeable distortion，JND）模型对时域特征参量的作用刻画尚不够充分，导致空时域JND模型精度不够理想。针对此问题，提出能准确刻画视频时域特性的特征参量以及异质特征参量同质化融合方法，并基于此改进时域JND模型。关注前景/背景运动、时域持续时间、时域预测残差波动强度、帧间预测残差等特征参量，用来刻画视频内容的时域特征；基于人眼视觉系统（human visual system， HVS）特性探索感知概率密度函数，将异质特征参量统一映射到自信息和信息熵尺度上，实现同质化融合度量；从能量分配的角度探究视觉注意与掩蔽的耦合方法，并据此构建时域JND权重模型。在空域JND阈值的基础上，融合时域权重以得到更加准确的空时域JND模型。为了评估空时域JND模型的性能，进行了主观质量评估实验，与现有的JND模型相比，在感知质量接近的情况下，提出的空时域JND模型能够容忍更多失真，具有更强的掩藏噪声的能力。
Abstract
The existing temporal domain JND(just noticeable distortion) models are not sufficient to depict the interaction between temporal parameters and HVS characteristics
leading to insufficient accuracy of the spatial-temporal JND model.To solve this problem
feature parameters that can accurately describe the temporal characteristics of the video were explored and extracted
as well as a homogenization method for fusing heterogeneous feature parameters
and the temporal domain JND model based on this was improved.The feature parameters were investigated including foreground and background motion
temporal duration along the motion trajectory
residual fluctuation intensity along motion trajectory and adjacent inter-frame prediction residual
etc.
which were used to characterize the temporal characteristics.Probability density functions for these feature parameters in the perception sense according to the HVS(human visual system) characteristics were proposed
and uniformly mapping the heterogeneous feature parameters to the scales of self-information and information entropy to achieve a homogeneous fusion measurement.The coupling method of visual attention and masking was explored from the perspective of energy distribution
and the temporal-domain JND weight model was constructed accordingly.On the basis of the spatial JND threshold
the temporal domain weights was integrated to develop a more accurate spatial-temporal JND model.In order to evaluate the performance of the spatiotemporal JND model
a subjective quality evaluation experiment was conducted.Experimental results justify the effectiveness of the proposed model.
关键词
Keywords
references
YUAN D , ZHAO T S , XU Y W , et al . Visual JND:a perceptual measurement in video coding [J ] . IEEE Access , 2019 ( 7 ): 29014 - 29022 .
KORHONEN J . Two-level approach for no-reference consumer video quality assessment [J ] . IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society , 2019 , 28 ( 12 ): 5923 - 5938 .
CHOU C H , LI Y C . A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 1995 , 5 ( 6 ): 467 - 476 .
YANG X K , LING W S , LU Z K , et al . Just noticeable distortion model and its applications in video coding [J ] . Signal Processing:Image Communication , 2005 , 20 ( 7 ): 662 - 680 .
CHIN Y J , BERGER T . A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 1999 , 9 ( 3 ): 438 - 450 .
KELLY D H . Motion and vision.II.Stabilized spatio-temporal threshold surface [J ] . Journal of the Optical Society of America , 1979 , 69 ( 10 ): 1340 - 1349 .
DALY S . Engineering observations from spatiovelocity and spatiotemporal visual models [M ] . Vision Models and Applications to Image and Video Processing . Heidelberg : Springer , 2001 : 179 - 200 .
JIA Y , LIN W , KASSIM A A . Estimating just-noticeable distortion for video [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2006 , 16 ( 7 ): 820 - 829 .
WEI Z Y , NGAN K N . Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2009 , 19 ( 3 ): 337 - 346 .
BAE S H , KIM M . A DCT-based total JND profile for spatiotemporal and foveated masking effects [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2017 , 27 ( 6 ): 1196 - 1207 .
WANG H Q , GAN W H , HU S D , et al . MCL-JCV:a JND-based H.264/AVC video quality assessment dataset [C ] // Proceedings of 2016 IEEE International Conference on Image Processing . Piscataway:IEEE Press , 2016 : 1509 - 1513 .
WANG H Q , KATSAVOUNIDIS I , ZHOU J T , et al . VideoSet:a large-scale compressed video quality dataset based on JND measurement [J ] . Journal of Visual Communication and Image Representation , 2017 , 46 : 292 - 302 .
KI S , BAE S H , KIM M , et al . Learning-based just-noticeable-quantization- distortion modeling for perceptual video coding [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 7 ): 3178 - 3193 .
LIU H H , ZHANG Y , ZHANG H , et al . Deep learning-based picture-wise just noticeable distortion prediction model for image compression [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 641 - 656 .
ZHANG Y , LIU H H , YANG Y , et al . Deep learning based just noticeable difference and perceptual quality prediction models for compressed video [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 6224 ( 99 ): 1 .
WANG Z , LI Q . Video quality assessment using a statistical model of human visual speed perception [J ] . Journal of the Optical Society of America A,Optics,Image Science,and Vision , 2007 , 24 ( 12 ): B61 - B69 .
MACKNIK S L , LIVINGSTONE M S . Neuronal correlates of visibility and invisibility in the primate visual system [J ] . Nature Neuroscience , 1998 , 1 ( 2 ): 144 - 149 .
STOCKER A A , SIMONCELLI E P . Noise characteristics and prior expectations in human visual speed perception [J ] . Nature Neuroscience , 2006 , 9 ( 4 ): 578 - 585 .
PETERSON H A , AHUMADA A J J , WATSON A B . Improved detection model for DCT coefficient quantization [C ] // Proceedings of the Human Vision,Visual Processing,and Digital Dis play IV,F,1993 .[S.l.:s.n. ] , 1993 : 191 - 201 .
BARKOWSKY M , BIALKOWSKI J , ESKOFIER B , et al . Temporal trajectory aware video quality measure [J ] . IEEE Journal of Selected Topics in Signal Processing , 2009 , 3 ( 2 ): 266 - 279 .
HU S D , JIN L N , WANG H L , et al . Objective video quality assessment based on perceptually weighted mean squared error [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2017 , 27 ( 9 ): 1844 - 1855 .
CHEN Z , GUILLEMOT C . Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2010 , 20 ( 6 ): 806 - 819 .
WU J , QI F , SHI G , et al . Non-local spatial redundancy reduction for bottom-up saliency estimation [J ] . Journal of Visual Communication and Image Representation , 2012 , 23 ( 7 ): 1158 - 1166 .
崔帅南 , 彭宗举 , 邹文辉 , 等 . 多特征融合的合成视点立体图像质量评价 [J ] . 电信科学 , 2019 , 35 ( 5 ): 104 - 112 .
CUI S N , PENG Z J , ZOU W H , et al . Quality assessment of synthetic viewpoint stereo image with multi-feature fusion [J ] . Telecommunications Science , 2019 , 35 ( 5 ): 104 - 112 .
ZENG Z P , ZENG H Q , CHEN J , et al . Visual attention guide dpixel-wise just noticeable difference model [J ] . IEEE Access , 2019 , 7 : 132111 - 132119 .
0
Views
389
下载量
0
CSCD
- L-PDFDownload
- XMLDownload
- Meta-XMLDownload
- BibTeXDownload
- EndnoteDownload
- RefMan Download
- NoteExpressDownload
- NoteFirstDownload
Publicity Resources
Related Articles
Related Author
Related Institution
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
- Total Visits：70032 Visits Today：87