Skip to content

Conversation

@SeanCurtis-TRI
Copy link
Contributor

@SeanCurtis-TRI SeanCurtis-TRI commented Nov 20, 2025

Fix rendering energy loss in RenderEngineVtk

Energy loss is due to three issues.

  1. Drake introduced an erroneous patch which bled energy badly.
  2. Set the skybox to use gamma correction, even for hdr images.
  3. Introduce a new patch to change the VTK brdf model from a single-scatter
    approximation to a multi-scatter approximation.

This will have an immediate change on rendered outputs -- images will be brighter.

Based on brightness changes:

  1. The lighting configuration tutorial has been updated accordingly.
  2. The unit tests for RenderEngineVtk have had reference values (subject to PBR) brightened accordingly.

Current camera configurations will need to be updated, especially with respect to their exposure values and lighting levels. Renderings for which high exposure values seemed necessary will now seem oversaturated. Furthermore, with more energy coming from the environment map, lights may need to be brighter to have a comparable effect as before.

Resolves #23772


The following will let you see the effect:

  1. Download an environment map (Drake doesn't store big expensive maps in its repo).
curl -o test.hdr https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/2k/thatch_chapel_2k.hdr
  1. Use meshcat to exercise RenderEngineVtk:
bazel run //tools:model_visualizer_private  -- --show_rgbd_sensor --no_lights \
package://drake_models/iiwa_description/sdf/iiwa7_with_box_collision.sdf --environment_map test.hdr

When we run against master, we get the following:

image

This PR gives us:

image

In addition to the image just generally being brighter, the skybox with the environment map handles the dynamic lighting range better; details are more discernible through the window.


This change is Reviewable

@SeanCurtis-TRI SeanCurtis-TRI added the release notes: fix This pull request contains fixes (no new features) label Nov 20, 2025
@SeanCurtis-TRI
Copy link
Contributor Author

+(release notes: fix) +a:@jwnimmer-tri for feature review, please.

Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@jwnimmer-tri reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: 2 unresolved discussions, needs at least two assigned reviewers


a discussion (no related file):
Some unit test(s?) are complaining about the environment map.


tutorials/configuring_rendering_lighting.ipynb line 2645 at r1 (raw file):

    "    \"shadow_map_size\": 1024,\n",
    "    # With additional lights, we reduce exposure.\n",
    "    \"exposure\": 1, # more lights --> reduced exposure.\n",

BTW Doesn't this EOL comment duplicate the line immediately above?

@SeanCurtis-TRI
Copy link
Contributor Author

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

Some unit test(s?) are complaining about the environment map.

Yeah...unit tests should be complaining. My oversight.

I've got a replacement patch (see #23772). I'm going to refresh this PR to not just remove the old, bad patch, but to include the new good patch.

Would you want to see that in a separate PR?

Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 unresolved discussions, needs at least two assigned reviewers


a discussion (no related file):

Previously, SeanCurtis-TRI (Sean Curtis) wrote…

Yeah...unit tests should be complaining. My oversight.

I've got a replacement patch (see #23772). I'm going to refresh this PR to not just remove the old, bad patch, but to include the new good patch.

Would you want to see that in a separate PR?

I'm OK with any approach.

Copy link
Contributor Author

@SeanCurtis-TRI SeanCurtis-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 unresolved discussion, needs at least two assigned reviewers


a discussion (no related file):

Previously, SeanCurtis-TRI (Sean Curtis) wrote…

Yeah...unit tests should be complaining. My oversight.

I've got a replacement patch (see #23772). I'm going to refresh this PR to not just remove the old, bad patch, but to include the new good patch.

Would you want to see that in a separate PR?

I'll push it here and you can choose if you'd like it reverted. I like them both together because the two together solve the problem.

If you want to run the furnace test locally:

  1. Download the following files:
curl -o material_quadrant.bin https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/material_quadrant.bin
curl -o material_quadrant.gltf https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/material_quadrant.gltf
curl -o furnace.hdr https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/furnace.hdr
  1. Run the visualizer
bazel run //tools:model_visualizer_private  -- --show_rgbd_sensor /absolute/path/to/the/files/you/downloaded/material_quadrant.gltf  --environment_map furnace.hdr --no_light

Running that on master and this PR will illustrate the lighting change (in ways a robot in a noisy environment can't).

@SeanCurtis-TRI SeanCurtis-TRI changed the title Fix rendering energy loss via correcting VTK usage Fix rendering energy loss in RenderEngineVtk Nov 21, 2025
Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-:lgtm:

@jwnimmer-tri reviewed 5 of 5 files at r2, all commit messages.
Reviewable status: 1 unresolved discussion, needs at least two assigned reviewers


a discussion (no related file):

Previously, SeanCurtis-TRI (Sean Curtis) wrote…

I'll push it here and you can choose if you'd like it reverted. I like them both together because the two together solve the problem.

If you want to run the furnace test locally:

  1. Download the following files:
curl -o material_quadrant.bin https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/material_quadrant.bin
curl -o material_quadrant.gltf https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/material_quadrant.gltf
curl -o furnace.hdr https://raw.githubusercontent.com/SeanCurtis-TRI/drake/refs/heads/PR_multiscatter_with_furnace/geometry/test/furnace.hdr
  1. Run the visualizer
bazel run //tools:model_visualizer_private  -- --show_rgbd_sensor /absolute/path/to/the/files/you/downloaded/material_quadrant.gltf  --environment_map furnace.hdr --no_light

Running that on master and this PR will illustrate the lighting change (in ways a robot in a noisy environment can't).

I agree together is best.

In any case, the CI is fixed so I'll close this thread.


tools/workspace/vtk_internal/repository.bzl line 192 at r3 (raw file):

            #   will be upstreamed into VTK itself, so they should be the first
            #   changes applied to reduce merge conflict churn.
            # - Patch file names should begin with the name of the module being

BTW Maybe rename the patch file to match this convention?

(Also I wonder if "add" is the right verb for it. Aren't we replacing single-scatter with multi-?)

Code quote:

Patch file names should begin with the name of the module

Energy loss is due to three issues.

1. Drake introduced an erroneous patch which bled energy badly.
2. Set the skybox to use gamma correction, even for hdr images.
3. Introduce a *new* patch to change the VTK brdf model from a single-scatter
   approximation to a multi-scatter approximation.

This will have an immediate change on rendered outputs -- images will be
brighter.

Based on brightness changes:
  1. The lighting configuration tutorial has been updated accordingly.
  2. The unit tests for RenderEngineVtk have had reference values (subject
     to PBR) brightened accordingly.

Current camera configurations will need to be updated, especially with
respect to their exposure values and lighting levels. Renderings for which
high exposure values seemed necessary will now seem oversaturated.
Furthermore, with more energy coming from the environment map, lights may
need to be brighter to have a comparable effect as before.
Copy link
Contributor Author

@SeanCurtis-TRI SeanCurtis-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW This is the PR I had in mind for using the release notes: announce tag (#23812).

+a:@sammy-tri for platform review, please.

Reviewable status: LGTM missing from assignee sammy-tri(platform)

Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jwnimmer-tri reviewed 2 of 2 files at r4, all commit messages.
Reviewable status: LGTM missing from assignee sammy-tri(platform)

Copy link
Contributor

@sammy-tri sammy-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@sammy-tri reviewed 2 of 4 files at r1, 3 of 5 files at r2, 2 of 2 files at r4, all commit messages.
Reviewable status: :shipit: complete! all discussions resolved, LGTM from assignees sammy-tri(platform),jwnimmer-tri(platform)

@sammy-tri sammy-tri merged commit 3e7bc19 into RobotLocomotion:master Dec 1, 2025
9 checks passed
@SeanCurtis-TRI SeanCurtis-TRI deleted the PR_vtk_pbr_restoration branch December 1, 2025 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release notes: fix This pull request contains fixes (no new features)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RenderEngineVtk suffers significant energy loss

3 participants