Skip to content

Use native flow prediction for DMD critic loss - #57

Open
zjn20030811 wants to merge 2 commits into
thu-ml:mainfrom
zjn20030811:fix/dmd-native-flow-prediction
Open

Use native flow prediction for DMD critic loss#57
zjn20030811 wants to merge 2 commits into
thu-ml:mainfrom
zjn20030811:fix/dmd-native-flow-prediction

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

  • use the fake critic's native flow prediction for flow-matching loss
  • avoid a flow-to-x0-to-flow round trip through two different scheduler grids
  • preserve the existing non-flow prediction path

Why

WanDiffusionWrapper already returns both the model's flow prediction and its derived x0 prediction. The fake critic uses the wrapper's default shift of 8, while the generator scheduler in the default DMD configurations uses shift 5. Reconstructing flow from x0 with the generator scheduler therefore selects sigma from a different discretized grid and is not an identity operation.

For example, at the valid critic timestep 20 the old round trip scales the native flow prediction by about 1.194. Passing the native prediction directly to FlowPredLoss keeps it in the parameterization produced by the critic and matches the noise - x training target.

This addresses the mismatch discussed in #35.

Tests

  • verifies that flow critic loss receives the native flow tensor with the expected flattened batch/frame layout
  • verifies gradients propagate to the native flow prediction
  • verifies the non-flow branch still converts x0 to a noise prediction
2 passed

@zjn20030811

Copy link
Copy Markdown
Author

Follow-up in 1d1c610: the repository has a parallel long_video/model/dmd.py critic path that still reconstructed flow from the derived x0 prediction. It now consumes the native flow tensor returned by fake_score, matching the main DMD path and preserving the correct parameterization/gradient. Added a focused long-video regression case; python -m pytest -q tests/test_dmd_flow_prediction.py passes (3 tests).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant