Project: Low-Cost Tracked 3D Ultrasound Reconstruction with GE Vscan Air and NousNav

[amirali-azimi96]

Draft Status

Draft - team will hold off on page creation

Category

IGT and Training

Key Investigators

• Amirali Azimi (Brigham and Women's Hospital, USA)
• Alexandra Golby (Brigham and Women's Hospital, USA)
• Colin Galvin (Brigham and Women's Hospital, USA)
• Tina Kapur (Brigham and Women's Hospital, USA)

Project Description

Intraoperative brain shift reduces the accuracy of neuronavigation systems that rely on preoperative imaging. Updated intraoperative imaging can help restore navigation accuracy, but technologies such as intraoperative MRI and conventional cart-based ultrasound systems are not available in many surgical environments.

We are developing a lower-cost workflow for tracked three-dimensional ultrasound reconstruction using the wireless GE Vscan Air CL ultrasound probe and NousNav, an open-source neuronavigation platform implemented in 3D Slicer. During an ultrasound sweep, NousNav uses an optical tracking system to record the position and orientation of both the ultrasound probe and a patient reference frame. The goal is to associate each two-dimensional ultrasound frame with the corresponding tracking transform, position the frames correctly in three-dimensional space, and reconstruct them as a navigable three-dimensional ultrasound volume.

A proof-of-concept reconstruction has already been performed using ultrasound images acquired through the Vscan Air mobile application and tracking data transmitted using the PLUS Toolkit and SlicerIGT. However, the current workflow relies on screen-recorded ultrasound video, which reduces image quality and makes precise synchronization between ultrasound frames and tracking data difficult. Several additional components remain unresolved, including calibration of the ultrasound image plane relative to the tracked probe, registration of the reconstructed ultrasound volume to the patient coordinate system using the tracked reference frame, and integration of the current Python-based NIfTI reconstruction process directly into the 3D Slicer environment.

During Project Week, we aim to develop and evaluate a more reliable and integrated workflow for ultrasound image acquisition, timestamp synchronization, probe calibration, patient registration, three-dimensional reconstruction, and visualization within 3D Slicer and the broader open-source image-guided therapy ecosystem.

Objective

1. Implement and validate ultrasound probe calibration and patient-coordinate registration, enabling each ultrasound frame to be placed accurately within the navigation coordinate system.
2. Integrate the current Python-based 3D reconstruction pipeline into 3D Slicer to generate and visualize a navigable three-dimensional ultrasound volume without relying on a separate offline NIfTI-generation process.

Approach and Plan

1. Review and organize the existing proof-of-concept pipeline, including the Vscan Air screen-recorded ultrasound data, PLUS Toolkit tracking stream, probe and reference-frame transforms, and the current Python-based NIfTI reconstruction code.
2. Develop a frame-extraction and timestamping workflow for the ultrasound video, and evaluate methods for matching each ultrasound frame with the closest corresponding probe and reference-frame tracking transform.
3. Implement ultrasound probe calibration to determine the spatial relationship between the tracked probe marker and the ultrasound image plane, and evaluate the calibration using a phantom or known target geometry.
4. Establish the complete transformation chain between the ultrasound image, tracked probe, reference frame, patient, and 3D Slicer coordinate systems so that each ultrasound frame can be positioned correctly in the navigation space.
5. Transfer the current Python reconstruction process into the 3D Slicer environment, either as a Python scripted module or an integrated processing workflow, to reduce dependence on external offline processing.
6. Reconstruct the synchronized ultrasound frames into a 3D volume and visualize the resulting volume together with the patient imaging and tracked instruments in 3D Slicer.
7. Evaluate the reconstructed volume for spatial consistency, image orientation, dimensions, frame placement, and alignment with the phantom or patient reference coordinate system.
8. Document the complete acquisition, synchronization, calibration, registration, reconstruction, and visualization workflow, including remaining limitations and requirements for future real-time integration with the GE Vscan Air system.

Progress and Next Steps

1. Developed an initial proof-of-concept application for recording Vscan Air ultrasound video and NousNav tracking data during the same acquisition.
2. Implemented collection of ultrasound probe and reference-frame tracking transforms through PLUS and OpenIGTLink.
3. Created an initial workflow for extracting ultrasound frames from the recorded video and approximately matching them with tracking timestamps.
4. Developed a preliminary Python-based reconstruction method for placing ultrasound frames in 3D space and exporting the result as a NIfTI volume.
5. Generated early proof-of-concept reconstructions that can be viewed in 3D Slicer, but the spatial accuracy and reliability of the workflow have not yet been validated.

Illustrations

No response

Background and References

No response