2D/3D多模态医学图像配准算法研究

徐密; 诸葛斌; 袁非牛; 尹正虎; 董黎刚; 蒋献; 孙应绮; 宋嘉琦; 史晓彤; 苏雷; 周屹博; 林诗凡

doi:10.11959/j.issn.1000-0801.2024070

您当前的位置：

首页 >

文章列表页 >

2D/3D多模态医学图像配准算法研究

研究与开发 | 更新时间：2024-06-05

- 2D/3D多模态医学图像配准算法研究
- Research on 2D/3D multimodal medical image registration algorithm
- 电信科学 2024年40卷第3期页码：75-88
- 作者机构：
  
  1.浙江工商大学信息与电子工程学院，浙江杭州 310020
  2.上海师范大学信息与机电工程学院，上海 201418
  3.复旦大学附属儿科医院，上海 201102
- 作者简介：
  
  [ "徐密（1998- ），女，浙江工商大学信息与电子工程学院硕士生，主要研究方向为医学图像配准。" ]
  [ "诸葛斌（1976- ），男，博士，浙江工商大学信息与电子工程学院教授，主要研究方向为医学图像配准与模式识别、互联网技术和云计算。" ]
  [ "袁非牛（1976- ），男，博士，上海师范大学信息与机电工程学院教授，主要研究方向为图像处理、模式识别、人工智能、深度学习。" ]
  尹正虎（1997- ），男，浙江工商大学信息与电子工程学院硕士生，主要研究方向为医学图像处理、人工智能。
  董黎刚（1972- ），男，博士，浙江工商大学信息与电子工程学院教授，IEEE和IEEE-CS成员，中国电子学会高级会员，浙江省计算机学会理事，主要研究方向为智能网络、在线教育。
  蒋献（1988- ），男，浙江工商大学信息与电子工程学院讲师、实验员，主要研究方向为深度学习、在线教育。
  孙应绮，sunyingqiqq@163.com
  宋嘉琦（2002- ），男，浙江工商大学萨塞克斯人工智能学院在读，主要研究方向为医学图像处理。
  史晓彤（2004- ），女，浙江工商大学计算机科学与技术学院在读，主要研究方向为医学图像处理。
  苏雷（2003- ），男，浙江工商大学信息与电子工程学院在读，主要研究方向为医学图像处理。
  周屹博（2002- ），男，浙江工商大学计算机科学与技术学院在读，主要研究方向为数据科学。
  林诗凡（2000- ），女，浙江工商大学信息与电子工程学院硕士生，主要研究方向为医学图像配准。
- 基金信息：
  
  国家自然科学基金资助项目(62272308);浙江省新型网络标准与应用技术重点实验室项目(2013E10012);浙江省“尖兵”“领雁”研发攻关计划项目(2023C03202);浙江省研究生一般项目(1120XJ0622033);国家级大学生创新创业训练计划项目(202210353022;202310353040;202310353069);浙江省大学生科技创新活动计划—新苗人才计划项目(1120KZN0223068G)
- DOI：10.11959/j.issn.1000-0801.2024070
  中图分类号： P237
- 收稿日期：2023-12-12，
  
  修回日期：2024-01-20，
  
  纸质出版日期：2024-03-20
- 稿件说明：
移动端阅览
徐密,诸葛斌,袁非牛等.2D/3D多模态医学图像配准算法研究[J].电信科学,2024,40(03):75-88.

XU Mi,ZHUGE Bin,YUAN Feiniu,et al.Research on 2D/3D multimodal medical image registration algorithm[J].Telecommunications Science,2024,40(03):75-88.
徐密,诸葛斌,袁非牛等.2D/3D多模态医学图像配准算法研究[J].电信科学,2024,40(03):75-88. DOI： 10.11959/j.issn.1000-0801.2024070.

XU Mi,ZHUGE Bin,YUAN Feiniu,et al.Research on 2D/3D multimodal medical image registration algorithm[J].Telecommunications Science,2024,40(03):75-88. DOI： 10.11959/j.issn.1000-0801.2024070.

摘要

2D/3D多模态配准在医学影像导航手术中起着重要作用，主要用于提供术前三维图像和术中二维图像的实时信息，帮助医生精准定位病灶，规划手术路径，从而提高手术的安全性和效率。提出了一种2D/3D多模态医学图像配准算法，首先利用Swin Transformer优秀的特征提取能力，构建了初始姿态估计模型，实现姿态参数的快速预测；接着，为了提升整个配准方法的鲁棒性，引入基于Grangeat关系的粗配准方法；最后设计了基于梯度下降的精配准模块，以提升整个配准过程的精确性，且在该模块将Sobel微分算子与归一化互相关结合，提升了参数优化过程中的灵敏度。实验结果表明，所提配准方法在正位和侧位配准中的误差满足配准要求，配准成功率有显著提升。

Abstract

2D/3D multimodal alignment plays an important role in medical image navigation surgery

which is mainly used to provide real-time information of preoperative 3D images and intraoperative 2D images to help doctors accurately locate the lesions and plan the surgical paths

so as to improve the safety and efficiency of surgery. A 2D/3D multimodal medical image alignment algorithm was proposed

which firstly utilized the excellent feature extraction capability of Swin Transformer to construct an initial pose estimation model to realize the fast prediction of pose parameters. Then

in order to improve the robustness of the whole alignment method

a coarse alignment method based on the Grangeat relation was introduced. Finally

a fine alignment module based on gradient descent was designed. A fine alignment module based on gradient descent was designed to improve the accuracy of the whole alignment process

and the Sobel differential operator was combined with normalized correlation in this module to improve the sensitivity of the parameter optimization process. The experimental results show that the proposed alignment method meets the alignment requirements in the orthogonal and lateral alignment errors

and the alignment success rate is significantly improved.

关键词

Keywords

references

FAN X Q , ZHU Q Y , TU P X , et al . A review of advances in image-guided orthopedic surgery [J ] . Physics in Medicine and Biology , 2022 .

张峰峰 , 陈龙 , 杨诗怡 , 等 . 机器人辅助脊柱手术三维定位导航 [J ] . 哈尔滨工程大学学报 , 2020 , 41 ( 12 ): 1735 - 1741 .

ZHANG F F , CHEN L , YANG S Y , et al . Robot-assisted three-dimensional positioning and navigation in spinal surgery [J ] . Journal of Harbin Engineering University , 2020 , 41 ( 12 ): 1735 - 1741 .

莫晓盈 , 杨锋 , 尹梦晓 , 等 . 医学图像配准的深度学习方法综述 [J ] . 小型微型计算机系统 , 2021 , 42 ( 8 ): 1706 - 1714 .

MO X Y , YANG F , YIN M X , et al . Survey on deep learning in medical image registration [J ] . Journal of Chinese Computer Systems , 2021 , 42 ( 8 ): 1706 - 1714 .

VIERGEVER M A , MAINTZ J B A , KLEIN S , et al . A survey of medical image registration - under review [J ] . Medical Image Analysis , 2016 ( 33 ): 140 - 144 .

MARTÍNEZ-RÍO J , CARMONA E J , CANCELAS D , et al . Deformable registration of multimodal retinal images using a weakly supervised deep learning approach [J ] . Neural Computing and Applications , 2023 , 35 ( 20 ): 14779 - 14797 .

HE H Q , CHEN M , CHEN T , et al . Matching of remote sensing images with complex background variations via Siamese convolutional neural network [J ] . Remote Sensing , 2018 , 10 ( 2 ): 355 .

AKTER M , LAMBERT A J , PICKERING M R , et al . Improved robustness of 3D CT to 2D fluoroscopy image registration using log polar transforms [C ] // Proceedings of the 2016 2nd International Conference on Electrical , Computer & Telecommunication Engineering (ICECTE) . Piscataway : IEEE Press , 2016 : 1 - 5 .

HAMADEH A , LAVALLEE S , CINQUIN P . Automated 3-dimensional computed tomographic and fluoroscopic image registration [J ] . Computer Aided Surgery: Official Journal of the International Society for Computer Aided Surgery , 1998 , 3 ( 1 ): 11 - 19 .

GONG R H , STEWART J , ABOLMAESUMI P . Multiple-object 2-D-3-D registration for noninvasive pose identification of fracture fragments [J ] . IEEE Transactions on Bio-Medical Engineering , 2011 , 58 ( 6 ): 1592 - 1601 .

HIROTAKI K , MORIYA S , AKITA T , et al . Image preprocessing to improve the accuracy and robustness of mutual-information-based automatic image registration in proton therapy [J ] . Physica Medica: PM: an International Journal Devoted to the Applications of Physics to Medicine and Biology: Official Journal of the Italian Association of Biomedical Physics (AIFB) , 2022 ( 101 ): 95 - 103 .

HEBDA-SOBKOWICZ J , ZIMROZ R , PITERA M , et al . Informative frequency band selection in the presence of non-Gaussian noise - a novel approach based on the conditional variance statistic with application to bearing fault diagnosis [J ] . Mechanical Systems and Signal Processing , 2020 ( 145 ): 106971 .

SIDDIQI M H , ALI R , IDRIS M , et al . Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection [J ] . Multimedia Tools and Applications , 2016 , 75 ( 2 ): 935 - 959 .

VIEL C , VIOLLET S . Fast normalized cross-correlation for measuring distance to objects using optic flow, applied for helicopter obstacle detection [J ] . Measurement , 2020 ( 172 ): 108911 .

SENTHIL PANDI S , SENTHILSELVI A , MARAGATHARAJAN M , et al . An optimal self adaptive deep neural network and spine-kernelled chirplet transform for image registration [J ] . Concurrency and Computation: Practice and Experience , 2022 , 34 ( 27 ): e7297 .1-e7297. 16 .

ZHANG Z Y , LIU Y , TIAN J W , et al . Study on reconstruction and feature tracking of silicone heart 3D surface [J ] . Sensors , 2021 , 21 ( 22 ): 7570 .

PYATOV V A , SOROKIN D V . Affine registration of histological images using transformer-based feature matching [J ] . Pattern Recognition and Image Analysis , 2022 , 32 ( 3 ): 626 - 630 .

EL-GAMAL F E Z A , ELMOGY M , ATWAN A . Current trends in medical image registration and fusion [J ] . Egyptian Informatics Journal , 2016 , 17 ( 1 ): 99 - 124 .

EPPENHOF K A J , PLUIM J P W . Pulmonary CT registration through supervised learning with convolutional neural networks [J ] . IEEE Transactions on Medical Imaging , 2018 , 38 ( 5 ): 1097 - 1105 .

MIAO S , PIAT S , FISCHER P , et al . Dilated FCN for multi-agent 2D/3D medical image registration [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2018 , 32(1) .

孙涛 , 郭可 , 刘传耙 , 等 . 2D-3D配准中的初始位姿估计方法 [J ] . 天津大学学报(自然科学与工程技术版) , 2022 , 55 ( 2 ): 143 - 150 .

SUN T , GUO K , LIU C B , et al . Initial pose estimation method in 2D-3D registration [J ] . Journal of Tianjin University (Natural Science and Engineering Technology) , 2022 , 55 ( 2 ): 143 - 150 .

PICKERING M R , MUHIT A A , SCARVELL J M , et al . A new multi-modal similarity measure for fast gradient-based 2d-3d image registration [C ] // Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Piscataway : IEEE Press , 2009 : 5821 - 5824 .

AKTER M , LAMBERT A J , PICKERING M R , et al . Robust initialisation for single-plane 3D CT to 2D fluoroscopy image registration [J ] . Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization , 2015 , 3 ( 3 ): 147 - 171 .

GAO C , LIU X , GU W , et al . Generalizing spatial transformers to projective geometry with applications to 2D/3D registration [C ] // Medical Image Computing and Computer Assisted Intervention , 2020 : 329 - 339 .

LIU Z , LIN Y , CAO Y , et al . Swin transformer: hierarchical vision transformer using shifted windows [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2021 : 10012 - 10022 .

FRYSCH R , PFEIFFER T , ROSE G . A novel approach to 2D/3D registration of X-ray images using Grangeat’s relation [J ] . Medical Image Analysis , 2021 ( 67 ): 101815 .

KANG M S , KIM K T . Automatic SAR image registration via tsallis entropy and iterative search process [J ] . IEEE Sensors Journal , 2020 , 20 ( 14 ): 7711 - 7720 .

VINCENT O R , FOLORUNSO O . A descriptive algorithm for sobel image edge detection [C ] // Proceedings of informing science & IT education conference (InSITE) . 2009 , ( 40 ): 97 - 107 .

金顺楠 . 面向术中CT导航的脊柱自动分割与配准技术研究 [D ] . 杭州 : 杭州师范大学 , 2021 .

JIN S N . Research on automatic spinal segmentation and registration technology for intraoperative CT navigation [D ] . Hangzhou : Hangzhou Normal University , 2021 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据