Feat: Axis-based Image Stretching

[arul-trenser]

Context

• PR for the Feature Request: Axis-based Image Stretching [Feature Request] Support for Axis-Based Image Stretching #2287
• This PR has been incorporated by FlyWheel.io.

Changes & Results

This PR introduces axis-based image stretching using camera projection matrix manipulation.

Usage Example

Stretch along the X-axis:

const { aspectRatio }  = viewport.getCamera()
viewport.setCamera({
    acpectRatio: [2, aspectRatio[1]]
})

Stretch along the X-axis:

const { aspectRatio }  = viewport.getCamera()
viewport.setCamera({
    acpectRatio: [aspectRatio[0], 2]
})

Testing

Basic Stretching along X and Y axis:

axialBasedStretching.mp4

Segmentation Brush Tool Behavior in Stretched Viewport:

brushWorkingWithStretch.mp4

Circle ROI Tool Behavior in Stretched Viewport

circleAnnotationInStretch.mp4

Checklist

PR

• My Pull Request title is descriptive, accurate and follows the
  semantic-release format and guidelines.

Code

• My code has been well-documented (function documentation, inline comments,
  etc.)

Public Documentation Updates

• [] The documentation page has been updated as necessary for any public API
  additions or removals.

Tested Environment

• "OS: Windows 11
• "Node version: 22.19.0
• "Browser: Chrome 141.0.7390.123

[arul-trenser]

@sedghi when you have a moment, could you kindly review this PR at your earliest convenience?
Thanks!

[sedghi]

how we can test it?

[arul-trenser]

how we can test it?

@sedghi Added an example demonstrating the stretch behavior in axialBasedImageStretching
Also attached videos in the PR description

[sedghi]

Are you aware of other viewers how they do this feature? i'm trying to evaluate the result at least to some other library or product that has this feature

[jmannau]

Hi,
We work with systems that have different vertical/horizontal scales. For example, with Optical Coherence Topography (https://eyewiki.org/Optical_Coherence_Tomography) scans, it is common to display the image at 2:1 (vertical/horizontal) aspect ration. The scans often have 2x->4x vertical resolution compared to the horizontal scale. This allows better visualisation of pathology in the vertical/depth axis. We also allow users to change the aspect ratio between 1:1 and 4:1 to account for different vendor devices and scan types.
Below are 2 images that demonstrate this.

1:1 Aspect Ratio

the same scan at 2:1 Aspect Ratio

[sedghi]

@jmannau lovely, would you be able to test this PR against the viewer you have?

[jmannau]

Happy to. But we’re very short on time until the new year. I’ll see what we can do

[sen-trenser]

@jmannau We are aiming to include this change along with the CS3D dependent version of the upcoming OHIF 3.12 release, which is expected to be released later this year.
When you have the opportunity, could you please prioritize reviewing this MR and share any feedback? Your input would be greatly appreciated and help us to timely include it in the next release.

Thank you!

[wayfarer3130]

We need to define some requirements as part of this change for what it is that the circle, sphere and elliptical tools are doing.
The sphere I believe should always be a sphere in world coordinates, as otherwise it does weird things to the shape. That means it should show as an ellipse if displaying world coordinates does not result in an aspect ratio of 1:1 for the rows/columns.

Circle and circle brushes are harder because of the differences in how things get saved. The circle brush is never saved in image coordinates, so I think we could fairly easily display a circle in the current transformed coordinates.
Circle annotations are saved as both MPR in world coordinates and as index coordinates.
This needs some documentation in a requirements section for exactly what it is that is meant by "sphere" "circle brush" and "circle tool" under MPR/co-planar modes with index, canvas and world coordinates at different aspect ratios.

[wayfarer3130]

For the volume (mpr and 3d) use cases, we should define what the aspect ratio means and how it applies in terms of the transform. In general, one can have 3 vectors that the transform is applied along, and those 3 are orthonormal, with a scaling factor defining them. I think this is something we need to figure out how to incorporate into the transform directly rather than trying to figure out which axial/sagittal/coronal it is closest to. That may actually be easier if we used a generic canvas stretch direction vector and orthogonal scale vector rather than x,y scaling. That way the transform for rotation simply rotates the stretch vector.

[arul-trenser]

@wayfarer3130
I have addressed all the review comments.

The implementation has been updated to use canvas-space stretching, ensuring that the stretch is applied relative to the canvas axes regardless of the MPR view rotation. This maintains consistent stretching behavior even when the image is rotated.

Could you please re-review the PR and share your feedback?

[sen-trenser]

@wayfarer3130 Could you please take a look at this PR and provide your feedbacks?

[wayfarer3130]

It is looking close - I added the ability to test with CPU rendering, and to have different viewport types, with the crosshairs bound to ctrl+click to allow orientation changes.

I am still noticing the changing the aspect ratio does not fit to viewport. I suspect you need a reset camera operation that has a don't change aspect ratio and that it should get called when you change the aspect ratio. Then, the fit to viewport needs to take into account the updated aspect ratio size.

[sen-trenser]

@wayfarer3130

I am still noticing the changing the aspect ratio does not fit to viewport.

Then, the fit to viewport needs to take into account the updated aspect ratio size.

Keeping the image fitted to the viewport after applying aspect ratio changes would cause the image to shrink rather than stretch. The intended behavior of the stretch feature is to keep the image centered and expand it outward along the selected axis while preserving the original size of the non-stretched axis. This allows users to view and annotate details more accurately in parallel to the stretched direction.

Could you share the use case for refitting the image to the viewport after stretching? Understanding this use case will help us assess whether supporting this behavior as the default would be beneficial to end users.

Thanks!

cc: @kgoettler

[arul-trenser]

changing the aspect ratio does not fit to viewport.

Fixed the above scenario by adding a configuration option to control whether the viewport should fit after applying the stretch. The current default behavior is to fit the viewport after stretch.

@wayfarer3130 Could you please take a look at this PR and provide your feedbacks?

[wayfarer3130]

@arul-trenser - could you fix the OHIF downstream tests? This should be passing for those.

[wayfarer3130]

@wayfarer3130

I am still noticing the changing the aspect ratio does not fit to viewport.

Then, the fit to viewport needs to take into account the updated aspect ratio size.

Keeping the image fitted to the viewport after applying aspect ratio changes would cause the image to shrink rather than stretch. The intended behavior of the stretch feature is to keep the image centered and expand it outward along the selected axis while preserving the original size of the non-stretched axis. This allows users to view and annotate details more accurately in parallel to the stretched direction.

Could you share the use case for refitting the image to the viewport after stretching? Understanding this use case will help us assess whether supporting this behavior as the default would be beneficial to end users.

Thanks!

cc: @kgoettler

The point is to be able to visually see fractures or other features that are narrow in the direction of stretch and wide orthogonal For example, rib fractures. If the zoom factor is reset so that the image stays in view, then the user can then use the zoom and pan tools to go through the entire image to look for such features. If they have adjusted it and don't initially see the full image, it is possible that a small portion of the image will not be visible AND will not be noticed to be offscreen. So, in terms of safety I would prefer to start with displaying the full image after changing the aspect ratio.
If one really wants a single-click tool, then probably the best option would be a way to draw a line that automatically orients itself up-down on the screen or left/right depending on the original orientation, AND stretches the line to 2:1 or something like that. That way the clinician would know exactly what they are seeing, and if they wanted a full-image view, then the line would start out full-range. For now, I'd like the scale to fit to work to prevent issues.