Skip to content

Conversation

@AnMakc
Copy link

@AnMakc AnMakc commented Nov 7, 2025

This PR disables unconditional rounding of the loss value to 4 digits in logged values. Effectively reverting part of #9491.

Closes #38032 (stale)

Motivation:

  • Loss value may have very low absolute magnitude depending on model/data/objective function.
  • No other values are rounded (e.g. eval/loss or grad_norm) which may result in confusion.
  • Rounding is a display/formatting detail in this case and should not be done silently in non-UI code

Examples of undesired behavior that would be fixed with this PR:

Notebook outputs:
Screenshot 2025-11-07 at 16 06 21

Wandb loss chart:
Screenshot 2025-11-07 at 15 42 45

@AnMakc AnMakc force-pushed the disable-loss-rounding branch from 686b9df to 45fa1e7 Compare November 7, 2025 19:36
@Rocketknight1
Copy link
Member

cc @SunMarc, but maybe formatting using something like %.4f so that it always displays 4 significant figures with an optional exponent would preserve the clean rendering but still work in cases when the loss is very small?

@AnMakc
Copy link
Author

AnMakc commented Nov 10, 2025

@Rocketknight1 Any rounding at this point introduces inconsistency between validation and training loss. We could not apply formatting here as we log numeric value, not string.

The main point is that any formatting is a representation detail that should be handled by the display-related code and we should not make assumptions on how it works as in most cases it is some external handler (wandb, trackio, ipython NB widget, etc.).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Removing the modification of loss value due to rounding off to 4 digits

2 participants