Research on 2D/3D multimodal medical image registration algorithm

XU Mi; ZHUGE Bin; YUAN Feiniu; YIN Zhenghu; DONG Ligang; JIANG Xian; SUN Yingqi; SONG Jiaqi; SHI Xiaotong; SU Lei; ZHOU Yibo; LIN Shifan

doi:10.11959/j.issn.1000-0801.2024070

您当前的位置：

首页 >

文章列表页 >

Research on 2D/3D multimodal medical image registration algorithm

Research and Development | 更新时间：2024-06-05

- Research on 2D/3D multimodal medical image registration algorithm
- Telecommunications Science Vol. 40, Issue 3, Pages: 75-88(2024)
- 作者机构：
  
  1.浙江工商大学信息与电子工程学院，浙江杭州 310020
  2.上海师范大学信息与机电工程学院，上海 201418
  3.复旦大学附属儿科医院，上海 201102
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62272308);Zhejiang Key Laboratory of New Network Standards and Application Technology(2013E10012);Zhejiang Province “Spearhead” and “Leading Wild Goose” Research and Development Project(2023C03202);General Project for Postgraduates in Zhejiang Province(1120XJ0622033);National Innovation and Entrepreneurship Training Program for College Students(202210353022;202310353040;202310353069);Science and Technology Innovation Activity Program for College Students in Zhejiang Province Fresh Seedling Talent Program(1120KZN0223068G)
- DOI：10.11959/j.issn.1000-0801.2024070
  CLC： P237
- Received：12 December 2023，
  
  Revised：20 January 2024，
  
  Published：20 March 2024
- 稿件说明：
移动端阅览
徐密,诸葛斌,袁非牛等.2D/3D多模态医学图像配准算法研究[J].电信科学,2024,40(03):75-88.

XU Mi,ZHUGE Bin,YUAN Feiniu,et al.Research on 2D/3D multimodal medical image registration algorithm[J].Telecommunications Science,2024,40(03):75-88.
徐密,诸葛斌,袁非牛等.2D/3D多模态医学图像配准算法研究[J].电信科学,2024,40(03):75-88. DOI： 10.11959/j.issn.1000-0801.2024070.

XU Mi,ZHUGE Bin,YUAN Feiniu,et al.Research on 2D/3D multimodal medical image registration algorithm[J].Telecommunications Science,2024,40(03):75-88. DOI： 10.11959/j.issn.1000-0801.2024070.

摘要

2D/3D多模态配准在医学影像导航手术中起着重要作用，主要用于提供术前三维图像和术中二维图像的实时信息，帮助医生精准定位病灶，规划手术路径，从而提高手术的安全性和效率。提出了一种2D/3D多模态医学图像配准算法，首先利用Swin Transformer优秀的特征提取能力，构建了初始姿态估计模型，实现姿态参数的快速预测；接着，为了提升整个配准方法的鲁棒性，引入基于Grangeat关系的粗配准方法；最后设计了基于梯度下降的精配准模块，以提升整个配准过程的精确性，且在该模块将Sobel微分算子与归一化互相关结合，提升了参数优化过程中的灵敏度。实验结果表明，所提配准方法在正位和侧位配准中的误差满足配准要求，配准成功率有显著提升。

Abstract

2D/3D multimodal alignment plays an important role in medical image navigation surgery

which is mainly used to provide real-time information of preoperative 3D images and intraoperative 2D images to help doctors accurately locate the lesions and plan the surgical paths

so as to improve the safety and efficiency of surgery. A 2D/3D multimodal medical image alignment algorithm was proposed

which firstly utilized the excellent feature extraction capability of Swin Transformer to construct an initial pose estimation model to realize the fast prediction of pose parameters. Then

in order to improve the robustness of the whole alignment method

a coarse alignment method based on the Grangeat relation was introduced. Finally

a fine alignment module based on gradient descent was designed. A fine alignment module based on gradient descent was designed to improve the accuracy of the whole alignment process

and the Sobel differential operator was combined with normalized correlation in this module to improve the sensitivity of the parameter optimization process. The experimental results show that the proposed alignment method meets the alignment requirements in the orthogonal and lateral alignment errors

and the alignment success rate is significantly improved.

关键词

Keywords

references

FAN X Q , ZHU Q Y , TU P X , et al . A review of advances in image-guided orthopedic surgery [J ] . Physics in Medicine and Biology , 2022 .

张峰峰 , 陈龙 , 杨诗怡 , 等 . 机器人辅助脊柱手术三维定位导航 [J ] . 哈尔滨工程大学学报 , 2020 , 41 ( 12 ): 1735 - 1741 .

ZHANG F F , CHEN L , YANG S Y , et al . Robot-assisted three-dimensional positioning and navigation in spinal surgery [J ] . Journal of Harbin Engineering University , 2020 , 41 ( 12 ): 1735 - 1741 .

莫晓盈 , 杨锋 , 尹梦晓 , 等 . 医学图像配准的深度学习方法综述 [J ] . 小型微型计算机系统 , 2021 , 42 ( 8 ): 1706 - 1714 .

MO X Y , YANG F , YIN M X , et al . Survey on deep learning in medical image registration [J ] . Journal of Chinese Computer Systems , 2021 , 42 ( 8 ): 1706 - 1714 .

VIERGEVER M A , MAINTZ J B A , KLEIN S , et al . A survey of medical image registration - under review [J ] . Medical Image Analysis , 2016 ( 33 ): 140 - 144 .

MARTÍNEZ-RÍO J , CARMONA E J , CANCELAS D , et al . Deformable registration of multimodal retinal images using a weakly supervised deep learning approach [J ] . Neural Computing and Applications , 2023 , 35 ( 20 ): 14779 - 14797 .

HE H Q , CHEN M , CHEN T , et al . Matching of remote sensing images with complex background variations via Siamese convolutional neural network [J ] . Remote Sensing , 2018 , 10 ( 2 ): 355 .

AKTER M , LAMBERT A J , PICKERING M R , et al . Improved robustness of 3D CT to 2D fluoroscopy image registration using log polar transforms [C ] // Proceedings of the 2016 2nd International Conference on Electrical , Computer & Telecommunication Engineering (ICECTE) . Piscataway : IEEE Press , 2016 : 1 - 5 .

HAMADEH A , LAVALLEE S , CINQUIN P . Automated 3-dimensional computed tomographic and fluoroscopic image registration [J ] . Computer Aided Surgery: Official Journal of the International Society for Computer Aided Surgery , 1998 , 3 ( 1 ): 11 - 19 .

GONG R H , STEWART J , ABOLMAESUMI P . Multiple-object 2-D-3-D registration for noninvasive pose identification of fracture fragments [J ] . IEEE Transactions on Bio-Medical Engineering , 2011 , 58 ( 6 ): 1592 - 1601 .

HIROTAKI K , MORIYA S , AKITA T , et al . Image preprocessing to improve the accuracy and robustness of mutual-information-based automatic image registration in proton therapy [J ] . Physica Medica: PM: an International Journal Devoted to the Applications of Physics to Medicine and Biology: Official Journal of the Italian Association of Biomedical Physics (AIFB) , 2022 ( 101 ): 95 - 103 .

HEBDA-SOBKOWICZ J , ZIMROZ R , PITERA M , et al . Informative frequency band selection in the presence of non-Gaussian noise - a novel approach based on the conditional variance statistic with application to bearing fault diagnosis [J ] . Mechanical Systems and Signal Processing , 2020 ( 145 ): 106971 .

SIDDIQI M H , ALI R , IDRIS M , et al . Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection [J ] . Multimedia Tools and Applications , 2016 , 75 ( 2 ): 935 - 959 .

VIEL C , VIOLLET S . Fast normalized cross-correlation for measuring distance to objects using optic flow, applied for helicopter obstacle detection [J ] . Measurement , 2020 ( 172 ): 108911 .

SENTHIL PANDI S , SENTHILSELVI A , MARAGATHARAJAN M , et al . An optimal self adaptive deep neural network and spine-kernelled chirplet transform for image registration [J ] . Concurrency and Computation: Practice and Experience , 2022 , 34 ( 27 ): e7297 .1-e7297. 16 .

ZHANG Z Y , LIU Y , TIAN J W , et al . Study on reconstruction and feature tracking of silicone heart 3D surface [J ] . Sensors , 2021 , 21 ( 22 ): 7570 .

PYATOV V A , SOROKIN D V . Affine registration of histological images using transformer-based feature matching [J ] . Pattern Recognition and Image Analysis , 2022 , 32 ( 3 ): 626 - 630 .

EL-GAMAL F E Z A , ELMOGY M , ATWAN A . Current trends in medical image registration and fusion [J ] . Egyptian Informatics Journal , 2016 , 17 ( 1 ): 99 - 124 .

EPPENHOF K A J , PLUIM J P W . Pulmonary CT registration through supervised learning with convolutional neural networks [J ] . IEEE Transactions on Medical Imaging , 2018 , 38 ( 5 ): 1097 - 1105 .

MIAO S , PIAT S , FISCHER P , et al . Dilated FCN for multi-agent 2D/3D medical image registration [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . 2018 , 32(1) .

孙涛 , 郭可 , 刘传耙 , 等 . 2D-3D配准中的初始位姿估计方法 [J ] . 天津大学学报(自然科学与工程技术版) , 2022 , 55 ( 2 ): 143 - 150 .

SUN T , GUO K , LIU C B , et al . Initial pose estimation method in 2D-3D registration [J ] . Journal of Tianjin University (Natural Science and Engineering Technology) , 2022 , 55 ( 2 ): 143 - 150 .

PICKERING M R , MUHIT A A , SCARVELL J M , et al . A new multi-modal similarity measure for fast gradient-based 2d-3d image registration [C ] // Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Piscataway : IEEE Press , 2009 : 5821 - 5824 .

AKTER M , LAMBERT A J , PICKERING M R , et al . Robust initialisation for single-plane 3D CT to 2D fluoroscopy image registration [J ] . Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization , 2015 , 3 ( 3 ): 147 - 171 .

GAO C , LIU X , GU W , et al . Generalizing spatial transformers to projective geometry with applications to 2D/3D registration [C ] // Medical Image Computing and Computer Assisted Intervention , 2020 : 329 - 339 .

LIU Z , LIN Y , CAO Y , et al . Swin transformer: hierarchical vision transformer using shifted windows [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2021 : 10012 - 10022 .

FRYSCH R , PFEIFFER T , ROSE G . A novel approach to 2D/3D registration of X-ray images using Grangeat’s relation [J ] . Medical Image Analysis , 2021 ( 67 ): 101815 .

KANG M S , KIM K T . Automatic SAR image registration via tsallis entropy and iterative search process [J ] . IEEE Sensors Journal , 2020 , 20 ( 14 ): 7711 - 7720 .

VINCENT O R , FOLORUNSO O . A descriptive algorithm for sobel image edge detection [C ] // Proceedings of informing science & IT education conference (InSITE) . 2009 , ( 40 ): 97 - 107 .

金顺楠 . 面向术中CT导航的脊柱自动分割与配准技术研究 [D ] . 杭州 : 杭州师范大学 , 2021 .

JIN S N . Research on automatic spinal segmentation and registration technology for intraoperative CT navigation [D ] . Hangzhou : Hangzhou Normal University , 2021 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰