What happened?

Physically-based rendering is supposed to be energy conserving. The mathematical formulation certainly is. However, compromises made in implementing for real time can introduce energy losses. Various implementations attempt to minimize the energy loss. RenderEngineVtk's implementation has non-trivial energy loss.

Take this image for example:

1. Four spheres varying between being metallic (bottom row) and dielectric (top row) and with different roughness (100% on the left-hand column and 0 on the right). Each sphere as a base color of white.
2. An HDR environment map of a sunlit studio.

Rendered in Meshcat, it looks like this:

Figure 1: Meshcat rendering

The same setup, rendered in RenderEngineVtk looks like this:

Figure 2: RenderEngineVtk rendering

Of particular note:

• The background environment map renders much darker than in meshcat.
• Similarly, the foreground spheres are likewise darker.
• The two renderers have been configured with similar tonemapping (see model_visualizer environment map gets forwarded to render engine #23771). So, the differences are not due to tone mapping or exposure.

We can apply a so-called "furnace test" to the two renderers. By applying an environment that is a uniform white color, we can assess energy loss. When the light is uniform in all directions, the perfectly reflecting objects should disappear into the background.

Figure 3: Meshcat in the furnace test.

In Figure 3, we've applied a uniform white environment map to the scene. In this case, we've left the grid visible, because, otherwise, you would not be able to see the spheres at all. With the grid, you can see the partial silhouettes of the lower row of spheres. There is no energy loss; the environment is reflected perfectly.

Figure 4: RenderEngineVtk in the furnace test

In Figure 4, there is clear energy loss on three of the four spheres. Only the smooth, metallic sphere is reflecting its whole environment. Roughness leads to energy loss (as well as being dielectric).

This is a known phenomenon (see this page's section on Single Scattering Results) for a detailed explanation. The short version is that VTK has implemented a simpler, more out-dated implementation of PBR.

The solution would be to update VTK's implementation to use multi-scattering, recapturing the light that is lost via the single-scatter model it has implemented.

Side note:

Even though it's not the focus of this issue, there is a further apparent disparity between VTK and Meshcat. In the Figure 2, the rough, metallic sphere (lower left) exhibits far higher contrast than the corresponding sphere in the Meshcat rendering. That could be attributable to exposure; the Meshcat rendering is brighter, so those illumination features may simply have been saturated to indiscernibility. However, if we reduce the exposure of meshcat (to 0.3), we can create an image that has the same qualitative illumination levels as RenderEngineVtk (see figure below).

However, the rough, metallic sphere still doesn't have the same contrast in the Meshcat renderer as it does in RenderEngineVtk. In fact, the RenderEngineVtk sphere simply doesn't seem as rough. At 100% roughness, I wouldn't expect to see environment-correlated highlights as clearly as we do.

Version

No response

What operating system are you using?

No response

What installation option are you using?

No response

Relevant log output