基于神经表达的动态场景重建综述

陆平; 孙俊杰; 李婉; 林家欣; 刘; 冯大权; 施文哲

doi:10.11959/j.issn.1000-0801.2025152

您当前的位置：

首页 >

文章列表页 >

基于神经表达的动态场景重建综述

研究与开发 | 更新时间：2025-06-28

- 基于神经表达的动态场景重建综述
- A review of dynamic scene reconstruction based on neural representation
- 电信科学 2025年41卷第6期页码：166-179
- 作者机构：
  
  1.移动网络和移动多媒体技术国家重点实验室，广东深圳 518057
  2.国家无线电监测中心检测中心，北京 100041
  3.广东省数字创意技术工程实验室，广东深圳 518060
  4.中国信息通信研究院泰尔终端实验室，北京 100083
  5.广东省哲学社会科学重点实验室（文化数字化与文化创新发展），广东深圳 518060
- 作者简介：
  
  [ "陆平（1971- ），男，博士，移动网络和移动多媒体技术国家重点实验室副主任、正高级工程师，东南大学、深圳大学博士生导师，中兴通讯股份有限公司副总裁，主要研究方向为云计算、增强现实、媒体大数据。" ]
  [ "孙俊杰（1992- ），男，博士，国家无线电监测中心检测中心工程师，主要研究方向为卫星互联网和数字媒体技术。" ]
  [ "李婉（1995- ），女，深圳大学电子与信息工程学院博士生，主要研究方向为计算机视觉、神经渲染和生成人工智能。" ]
  [ "林家欣（1996- ），男，深圳大学电子与信息工程学院博士生，主要研究方向为隐式表面表示和重建、几何计算和衣着人体。" ]
  [ "刘 \t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t（1973- ），女，现就职于中国信息通信研究院泰尔终端实验室，主要研究方向为人工智能、区块链、网络优化等。" ]
  [ "冯大权（1986- ），男，博士，深圳大学电子与信息工程学院副教授、博士生导师，广东省哲学社会科学重点实验室（文化数字化与文化创新发展）副主任，主要研究方向为生成式人工智能、沉浸式通信、VR/AR。" ]
  [ "施文哲（1989- ），男，移动网络和移动多媒技术国家重点实验室工程师，中兴通讯XRExplore产品规划经理，主要研究方向为室内视觉AR导航、三维重建、实时云渲染。" ]
- 基金信息：
  
  中兴通讯研究基金资助项目(2023ZTE03-04)
- DOI：10.11959/j.issn.1000-0801.2025152
  中图分类号： TP393
- 收稿：2025-02-25，
  
  修回：2025-05-29，
  
  录用：2025-06-03，
  
  纸质出版：2025-06-20
- 稿件说明：
移动端阅览
陆平,孙俊杰,李婉等.基于神经表达的动态场景重建综述[J].电信科学,2025,41(06):166-179.

LU Ping,SUN Junjie,LI Wan,et al.A review of dynamic scene reconstruction based on neural representation[J].Telecommunications Science,2025,41(06):166-179.
陆平,孙俊杰,李婉等.基于神经表达的动态场景重建综述[J].电信科学,2025,41(06):166-179. DOI： 10.11959/j.issn.1000-0801.2025152.

LU Ping,SUN Junjie,LI Wan,et al.A review of dynamic scene reconstruction based on neural representation[J].Telecommunications Science,2025,41(06):166-179. DOI： 10.11959/j.issn.1000-0801.2025152.

摘要

动态三维场景重建在计算机视觉和虚拟现实领域中具有重要的研究价值。近年来，得益于神经表达技术的快速发展，动态场景重建技术也取得了显著突破。过去4年中，基于神经辐射场和3D高斯泼溅的动态场景重建方法被相继提出，并取得了令人瞩目的效果。然而，由于相关文献的数量庞大，研究者很难全面跟踪所有相关工作。为了应对这一挑战，基于神经表达技术在动态场景重建中的发展历程，系统总结了该领域的代表性研究，总结归纳为基于神经辐射场和3D高斯泼溅两大类。此外，还介绍了当前动态三维场景重建的代表性数据集，并总结了常用的算法评估标准。最后，分析了当前动态三维场景重建技术面临的主要挑战，并展望了未来的发展趋势。

Abstract

Dynamic scene reconstruction holds significant research value in the fields of computer vision and virtual reality. Recent advancements in neural representation technologies have facilitated rapid progress in this task. Over the past four years

methods based on neural radiance fields and 3D Gaussian splatting have been proposed

achieving remarkable results. However

the large number of literature presents a challenge for individuals to comprehensively track comprehensive relevant works. To address this issue

typical work for dynamic scene reconstruction based on neural representation was summarized

categorizing them into methods based on neural radiance fields and 3D Gaussian splatting. Furthermore

representative datasets were highlighted and common evaluation metrics for algorithms were summarized. Finally

the persistent challenges in current methodologies were discussed and potential directions for future development trends were proposed.

关键词

Keywords

references

MILDENHALL B , SRINIVASAN P P , TANCIK M , et al . NeRF: representing scenes as neural radiance fields for view synthesis [C ] // Proceedings of the European Conference on Computer Vision (ECCV) 2020 . Cham : Springer , 2020 : 405 - 421 .

FANG J , YI T , WANG X , et al . Fast dynamic radiance fields with time-aware neural voxels [C ] // Proceedings of the SIGGRAPH Asia 2022 Conference Papers . New York : ACM Press , 2022 : 1 - 9 .

YU A , LI R , TANCIK M , et al . Plenoctrees for real-time rendering of neural radiance fields [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2021 : 5752 - 5761 .

SONG L C , CHEN A P , LI Z , et al . Nerfplayer: a streamable dynamic scene representation with decomposed neural radiance fields [J ] . IEEE Transactions on Visualization and Computer Graphics , 2023 , 29 ( 5 ): 2732 - 2742 .

WANG F , TAN S N , LI X H , et al . Mixed neural voxels for fast multi-view video synthesis [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2023 : 19706 - 19716 .

CAO A , JOHNSON J . Hexplane: a fast representation for dynamic scenes [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2023 : 130 - 141 .

SHAO R Z , ZHENG Z R , TU H Z , et al . Tensor4d: efficient neural 4D decomposition for high-fidelity dynamic reconstruction and rendering [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2023 : 16632 - 16642 .

KERBL B , KOPANAS G , LEIMKÜHLER T , et al . 3D Gaussian splatting for real-time radiance field rendering [J ] . ACM Transactions on Graphics , 2023 , 42 ( 4 ).

WU G J , YI T R , FANG J M , et al . 4D Gaussian splatting for real-time dynamic scene rendering [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 20310 - 20320 .

LIANG Y Q , KHAN N , LI Z Q , et al . GauFRe: Gaussian deformation fields for real-time dynamic novel view synthesis [C ] // Proceedings of the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE Press , 2025 : 2642 - 2652 .

LI T , SLAVCHEVA M , ZOLLHOEFER M , et al . Neural 3D video synthesis from multi-view video [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2022 : 5521 - 5531 .

LI W , PAN X , LIN J X , et al . FRPGS: fast, robust, and photorealistic monocular dynamic scene reconstruction with deformable 3D Gaussians [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2025 .

李吉洋 , 程乐超 , 何靖璇 , 等 . 神经辐射场的研究现状与展望 [J ] . 计算机辅助设计与图形学学报 , 2024 , 36 ( 7 ): 995 - 1013 .

LI J Y , CHENG L C , HE J X , et al . Current status and prospects of research on neural radiance fields [J ] . Journal of Computer-Aided Design & Computer Graphics , 2024 , 36 ( 7 ): 995 - 1013 .

WU T , YUAN Y J , ZHANG L X , et al . Recent advances in 3D Gaussian splatting [J ] . Computational Visual Media , 2024 , 10 ( 4 ): 613 - 642 .

LI Z , NIKLAUS S , SNAVELY N , et al . Neural scene flow fields for space-time view synthesis of dynamic scenes [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2021 : 6498 – 6508 .

PUMAROLA A , CORONA E , PONS-MOLL G , et al . D-NeRF: Neural radiance fields for dynamic scenes [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2021 : 10318 - 10327 .

PARK K , SINHA U , BARRON J T , et al . Nerfies: deformable neural radiance fields [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2021 : 5865 - 5874 .

PARK K , SINHA U , HEDMAN P , et al . HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields [J ] . ACM Transactions on Graphics , 2021 , 40 ( 6 ): 238 .

WANG L , ZHANG J , LIU X , et al . Fourier plenoctrees for dynamic radiance field rendering in real-time [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2022 : 13524 - 13534 .

LE CLEAC’H S , YU H X , GUO M , et al . Differentiable physics simulation of dynamics-augmented neural objects [J ] . IEEE Robotics and Automation Letters , 2023 , 8 ( 5 ): 2780 - 2787 .

LI Z Q , WANG Q Q , COLE F , et al . Dynibar: neural dynamic image-based rendering [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2023 : 4273 - 4284 .

YAN Z W , LI C , LEE G H . NeRF-DS: neural radiance fields for dynamic specular objects [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2023 : 8285 - 8295 .

BUI M Q , PARK J , OH J , et al . DyBluRF: dynamic neural radiance fields from blurry monocular video [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 7517 - 7527 .

LIN Y . Ced-NeRF: a compact and efficient method for dynamic neural radiance fields [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Los Altos : AAAI , 2024 , 38 ( 4 ): 3504 - 3512 .

ZHENG Z H , ZHONG H Q , HU Q , et al . HPC: hierarchical progressive coding framework for volumetric video [C ] // Proceedings of the 32nd ACM International Conference on Multimedia . New York : ACM Press , 2024 : 7937 - 7946 .

LUITEN J , KOPANAS G , LEIBE B , et al . Dynamic 3D gaussians: tracking by persistent dynamic view synthesis [C ] // 2024 International Conference on 3D Vision (3DV) . Piscataway : IEEE Press , 2024 : 800 - 809 .

SUN J K , JIAO H , LI G Y , et al . 3DGStream: on-the-fly training of 3D Gaussians for efficient streaming of photo-realistic free-viewpoint videos [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 20675 - 20685 .

YANG Z Y , GAO X Y , ZHOU W , et al . Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 20331 - 20341 .

YANG Z Y , YANG H Y , PAN Z J , et al . Real-time photorealistic dynamic scene representation and rendering with 4D Gaussian splatting [EB ] . 2024 .

DUAN Y X , WEI F Y , DAI Q Y , et al . 4D-Rotor Gaussian splatting: towards efficient novel view synthesis for dynamic scenes [C ] // Proceedings of the ACM Press SIGGRAPH 2024 Conference Papers . Denver : ACM Press , 2024 : 1 - 11 .

LIU Q M , LIU Y , WANG J P , et al . MoDGS: dynamic Gaussian splatting from casually-captured monocular videos [EB ] . 2024 .

LI Z , CHEN Z , LI Z , et al . Spacetime Gaussian feature splatting for real-time dynamic view synthesis [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 8508 - 8520 .

LU Z C , GUO X , HUI L , et al . 3D geometry-aware deformable Gaussian splatting for dynamic view synthesis [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 8900 - 8910 .

BAE J , KIM S , YUN Y , et al . Per-Gaussian embedding-based deformation for deformable 3D Gaussian splatting [C ] // Proceedings of the European Conference on Computer Vision . Cham : Springer Nature Switzerland , 2024 : 321 - 335 .

KATSUMATA K , VO D M , NAKAYAMA H . A compact dynamic 3D Gaussian reprsentation for real-time dynamic view synthesis [EB ] . 2023 .

LIN Y T , DAI Z Z , ZHU S Y , et al . Gaussian-flow: 4D reconstruction with dynamic 3D gaussian particle [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2024 : 21136 - 21145 .

LIANG H X , REN J W , MIRZAEI A , et al . Feed-forward bullet-time reconstruction of dynamic scenes from monocular videos [EB ] . 2024 .

KUNG P C , ZHANG X , SKINNER K A , et al . LiHi-GS: LiDAR-supervised gaussian splatting for highway driving scene reconstruction [EB ] . 2024 .

JAVED S , KHAN A J , DUMERY C , et al . Temporally compressed 3D Gaussian splatting for dynamic scenes [EB ] . 2024 .

GIRISH S , LI T Y , MAZUMDAR A , et al . QUEEN: quantized efficient encoding of dynamic Gaussians for streaming free-viewpoint videos [EB ] . 2024 .

GAO H , LI R , TULSIANI S , et al . Monocular dynamic view synthesis: a reality check [J ] . Advances in Neural Information Processing Systems , 2022 , 35 : 33768 - 33780 .

YOON J S , KIM K , GALLO O , et al . Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2020 : 5336 - 5345 .

BROXTON M , FLYNN J , OVERBECK R , et al . Immersive light field video with a layered mesh representation [J ] . ACM Transactions on Graphics (TOG) , 2020 , 39 ( 4 ): 1 - 86 .

SABATER N , BOISSON G , VANDAME B , et al . Dataset and pipeline for multi-view light-field video [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . Piscataway : IEEE Press , 2017 : 30 - 40 .

JOO H , LIU H , TAN L , et al . Panoptic studio: a massively multiview system for social motion capture [C ] // Proceedings of the IEEE International Conference on Computer Vision . Piscataway : IEEE Press , 2015 : 3334 - 3342 .

浏览量

1976

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据