feat: Capture SceneColor at transition start into a UTextureRenderTarget2D (Plan C – SVE Tonemap hook)

[EmbarrassingMoment]

1. Summary

This feature introduces the ability to capture the viewport's SceneColor at the exact frame a transition begins, write it into a UTextureRenderTarget2D, and automatically expose that render target to transition materials via a named texture parameter. This unlocks an entirely new class of transition effects that operate on a frozen still of the previous scene, such as screen shatters, page turns, slide-aways, glitch freezes, persona-style tile-spins, or dissolve-on-captured-frame effects, while maintaining the plugin's single Blueprint node workflow.

2. Motivation

The existing PostProcess architecture inherently processes the live scene every frame, meaning it cannot "freeze" the previous frame's image to manipulate it over time without external capture mechanisms.

• Dismissed SceneCapture2D: Requires a separate view re-render which is too costly, and it suffers from diverging TAA and autoexposure history compared to the main viewport.
• Dismissed DrawMaterialToRenderTarget: Does not have access to a valid SceneTexture:PostProcessInput0 context outside of the render pipeline, making it impossible to capture the post-processed frame accurately.

3. Design overview

The adopted solution (Plan C) uses a custom FSceneViewExtensionBase subclass that hooks into the render pipeline by subscribing to EPostProcessingPass::Tonemap. In the AfterPass callback, it issues an RDG AddCopyTexturePass to copy the tonemapped SceneColor into a pooled UTextureRenderTarget2D. Tonemap-after is chosen because the frame is TAA-resolved, tonemapped, in LDR, and pre-UI—matching the plugin's existing PP rendering layer semantics.

sequenceDiagram
    participant GT as Game Thread (Subsystem)
    participant SVE as Frame Capture SVE
    participant RT as Render Thread (RDG)

    GT->>SVE: StartTransition (Detect bCaptureFrameOnStart)
    GT->>SVE: RequestCapture(FrameCaptureRT)
    GT-->>GT: Set bPendingStartAfterCapture = true (Defer start)
    
    note over SVE,RT: Render pipeline executes normally
    
    SVE->>RT: SubscribeToPostProcessingPass (After Tonemap)
    RT->>SVE: Execute AfterPass callback
    SVE->>RT: AddCopyTexturePass (SceneColor -> FrameCaptureRT)
    SVE-->>SVE: Set bCaptureDone = true (Atomic)
    
    GT->>GT: Tick() checks IsCaptureFinished()
    GT->>GT: Inject captured RT into OverrideParams
    GT->>GT: Clear deferred flag, call StartTransition_Internal()

Loading

4. New types and member additions

// SVE Declaration
class FTransitionFrameCaptureSVE : public FSceneViewExtensionBase
{
public:
    FTransitionFrameCaptureSVE(const FAutoRegister& AutoReg);

    // Called on Game Thread
    void RequestCapture(UTextureRenderTarget2D* InTargetRT);
    
    // Called on Game Thread
    bool IsCaptureFinished() const;

    // FSceneViewExtensionBase Interface
    virtual void SubscribeToPostProcessingPass(EPostProcessingPass PassId, FAfterPassCallbackDelegateArray& InOutPassCallbacks, bool bIsPassEnabled) override;
    
    // The callback itself. (Signature requires verification against local engine source for UE 5.5/5.6)
    FScreenPassTexture PostProcessPassAfterTonemap_RenderThread(FRDGBuilder& GraphBuilder, const FSceneView& View, const FPostProcessMaterialInputs& Inputs);

    virtual int32 GetPriority() const override { return 10; }

private:
    // Written on GT, read on RT
    std::atomic<bool> bCaptureRequested{false};
    
    // Written on RT, read on GT
    std::atomic<bool> bCaptureDone{false};
    
    // Only touched on GT
    TWeakObjectPtr<UTextureRenderTarget2D> TargetRT;
};

// Subsystem additions
USTRUCT()
struct FPendingStartArgs
{
    GENERATED_BODY()

    UPROPERTY()
    TObjectPtr<UTransitionPreset> Preset;

    ETransitionMode Mode;
    float PlaySpeed;
    bool bInvert;
    bool bHoldAtMax;
    FTransitionParameters OverrideParams;
};

// In UTransitionManagerSubsystem:
UPROPERTY()
TObjectPtr<UTextureRenderTarget2D> FrameCaptureRT;

TSharedPtr<FTransitionFrameCaptureSVE, ESPMode::ThreadSafe> CaptureSVE;

bool bPendingStartAfterCapture = false;
FPendingStartArgs PendingStartArgs;


// Preset additions (In UTransitionPreset)
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "TransitionFX")
bool bCaptureFrameOnStart = false;

UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "TransitionFX", meta = (EditCondition = "bCaptureFrameOnStart"))
FName CapturedFrameParamName = TEXT("CapturedFrame");

// 0,0 means viewport-follow
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "TransitionFX", meta = (EditCondition = "bCaptureFrameOnStart"))
FIntPoint CaptureResolution = FIntPoint(0, 0);

5. Implementation plan, file by file

• NEW Plugins/TransitionFX/Source/TransitionFX/Public/TransitionFrameCaptureSVE.h

  • Purpose: Declares the custom scene view extension for frame capture.
  • Added/Modified: FTransitionFrameCaptureSVE class.
  • Code Excerpt:

    std::atomic<bool> bCaptureRequested{false};
    std::atomic<bool> bCaptureDone{false};
    TWeakObjectPtr<UTextureRenderTarget2D> TargetRT;

• NEW Plugins/TransitionFX/Source/TransitionFX/Private/TransitionFrameCaptureSVE.cpp

  • Purpose: Implements the RDG hook and SVE lifetime. Creation via FSceneViewExtensions::NewExtension<FTransitionFrameCaptureSVE>().
  • Added/Modified: FTransitionFrameCaptureSVE implementation.
  • Code Excerpt:

    FScreenPassTexture FTransitionFrameCaptureSVE::PostProcessPassAfterTonemap_RenderThread(FRDGBuilder& GraphBuilder, const FSceneView& View, const FPostProcessMaterialInputs& Inputs)
    {
        // ... Check atomics, get RT ...
        // Verify FPostProcessMaterialInputs and FScreenPassTexture signatures against local engine source
        FScreenPassTexture SceneColor = Inputs.ReturnUntouchedSceneColorForPostProcessing(GraphBuilder);
        // Register external RT
        FRDGTextureRef DestTexture = GraphBuilder.RegisterExternalTexture(CreateRenderTarget(TargetRT->GetRenderTargetResource(), TEXT("CapturedFrameRT")));
        // Issue copy
        AddCopyTexturePass(GraphBuilder, SceneColor.Texture, DestTexture);
        bCaptureDone = true;
        bCaptureRequested = false;
        return SceneColor;
    }

• MODIFY Plugins/TransitionFX/Source/TransitionFX/Public/TransitionPreset.h and Plugins/TransitionFX/Source/TransitionFX/Private/TransitionPreset.cpp

  • Purpose: Expose capture configuration to the user.
  • Added/Modified: bCaptureFrameOnStart, CapturedFrameParamName, CaptureResolution to UTransitionPreset.

• MODIFY Plugins/TransitionFX/Source/TransitionFX/Public/TransitionManagerSubsystem.h and Plugins/TransitionFX/Source/TransitionFX/Private/TransitionManagerSubsystem.cpp

  • Purpose: SVE lifecycle, RT pooling, and deferred transition start logic.
  • Added/Modified: CaptureSVE, FrameCaptureRT, bPendingStartAfterCapture, FPendingStartArgs, StartTransition shim, StartTransition_Internal.
  • Code Excerpt (Subsystem Tick):

    if (bPendingStartAfterCapture && CaptureSVE->IsCaptureFinished())
    {
        bPendingStartAfterCapture = false;
        PendingStartArgs.OverrideParams.TextureParams.Add(PendingStartArgs.Preset->CapturedFrameParamName, FrameCaptureRT);
        StartTransition_Internal(PendingStartArgs.Preset, PendingStartArgs.Mode, PendingStartArgs.PlaySpeed, PendingStartArgs.bInvert, PendingStartArgs.bHoldAtMax, PendingStartArgs.OverrideParams);
    }

  • Create and store the SVE in Initialize, release in Deinitialize.
  • Lazy-allocate FrameCaptureRT and resize when the viewport size changes.
  • Rework StartTransition(...) into a public shim that detects bCaptureFrameOnStart, caches args, calls CaptureSVE->RequestCapture(FrameCaptureRT), sets bPendingStartAfterCapture = true, and returns. Extract the current body into a private StartTransition_Internal(...).
  • Extend ForceClear / Deinitialize / world-teardown paths to cancel a pending capture safely.

• MODIFY Plugins/TransitionFX/TransitionFX.Build.cs

  • Purpose: Ensure necessary renderer and RDG modules are available.
  • Verify and, if missing, add RenderCore, Renderer, and Projects to PublicDependencyModuleNames / PrivateDependencyModuleNames as appropriate. Do not blindly add them; check first.

• Reference Material: A reference material (e.g., M_Transition_CapturedShatter) that samples the CapturedFrame texture parameter should be created, though the asset creation itself is out of scope for this issue.

6. StartTransition flow change

Before:

Blueprint -> StartTransition()
   -> Stop existing transition
   -> Create/reuse PostProcessTransitionEffect
   -> Initialize effect and apply override params
   -> Set bIsTransitionActive = true
   -> Immediately begin ticking progress

After:

Blueprint -> StartTransition()
   -> Detect Preset->bCaptureFrameOnStart == true
      -> YES: 
         -> Cache arguments in PendingStartArgs
         -> CaptureSVE->RequestCapture(FrameCaptureRT)
         -> Set bPendingStartAfterCapture = true
         -> Return (Deferred start. Looks like a 1-2 frame wait in Blueprint/latent actions)
      -> NO:
         -> StartTransition_Internal() (Original behavior)

Tick()
   -> If bPendingStartAfterCapture && CaptureSVE->IsCaptureFinished()
      -> Inject FrameCaptureRT into PendingStartArgs.OverrideParams
      -> bPendingStartAfterCapture = false
      -> StartTransition_Internal()

Blueprint callers and the PlayTransitionAndWait latent action are entirely unaffected: the deferred-start simply looks like a 1–2 frame delay inside the existing "waiting for transition" state.

7. Edge cases and risks

• Dynamic resolution / upscaler output mismatch: Source scene color and the destination UTextureRenderTarget2D extents may disagree. If so, AddCopyTexturePass will fail. Fallback to AddDrawScreenPass with UV remap when source and destination extents disagree.
• Split screen and multi-view: SVE callback will trigger for all views. The callback must filter the SVE capture logic to only the primary player's FSceneView.
• RT lifetime: Use a single pooled UTextureRenderTarget2D, resized on viewport changes, and explicitly released in Deinitialize.
• Pause / slomo: Tonemapper still runs even when paused, so the capture pass will remain valid and execute correctly.
• World teardown while a capture is in flight: Ensure an atomic cancel path exists in ForceClear and Deinitialize to abort the pending start.
• UI layer not captured: This suffers from the same limitation as the plugin's existing PostProcess architecture. The UMG UI layer sits outside the main render pipeline. Refer to the plugin README "Limitations & Notes" regarding UMG.
• UE minor-version drift: The signatures for FPostProcessMaterialInputs and FScreenPassTexture drift between UE 5.5 and 5.6. Verify against local engine source before finalizing the SVE AfterPass code.
• Thread safety: The atomic flags (bCaptureRequested, bCaptureDone) manage the thread-boundary state. TWeakObjectPtr is game-thread-only and must be dereferenced and converted to a raw pointer on the GT, passing the required RT/resource to the render thread safely.

8. Testing plan

• Manual Verification: Use an in-editor debug widget or UKismetRenderingLibrary::ExportRenderTarget2D to dump the captured frame and verify visually.
• Compatibility Check: Ensure all 22+ existing presets continue to function properly with bCaptureFrameOnStart = false (the default).
• Performance Profiling: Gather a before/after stat unit sample confirming there is no steady-state performance cost when the SVE is idle / capture is inactive.

9. Blueprint / BP API impact

None. PlayTransitionAndWait, PlayRandomTransitionAndWait, OpenLevelWithTransitionAndWait, and QuickFadeToBlack/FromBlack all retain their current signatures and behavior.

10. Out of scope

• UMG-layer capture (to be handled in a future separate issue).
• HDR / pre-tonemap capture path.
• Authoring of more than one reference material.
• Mobile / console validation.

11. Acceptance criteria

• Builds successfully on UE 5.5 and UE 5.6.
• Default bCaptureFrameOnStart = false preserves all existing behavior.
• Example material correctly receives a valid captured frame at progress ≈ 0.
• No new warnings in LogTransitionFX.
• stat unit delta is under ~0.05 ms on a 1080p desktop reference when the capture is active for a single frame.

12. References

• Runtime/Renderer/Public/PostProcess/PostProcessMaterialInputs.h (Verify path in local engine source)
• Runtime/Renderer/Public/SceneViewExtension.h (Verify path in local engine source)
• RDG headers for AddCopyTexturePass and AddDrawScreenPass (Verify path in local engine source)