Skip to content

feat: adding stochastic positional embeddings#1963

Open
kir-7 wants to merge 2 commits into
lightly-ai:masterfrom
kir-7:stop_emb
Open

feat: adding stochastic positional embeddings#1963
kir-7 wants to merge 2 commits into
lightly-ai:masterfrom
kir-7:stop_emb

Conversation

@kir-7

@kir-7 kir-7 commented Jul 5, 2026

Copy link
Copy Markdown

Part of #1900

Summary

  • Adds add_stochastic_positional_noise as utility function to lightly/models/utils.py .
  • Adds optinal use of StoP to:
    * lightly/models/modules/ijepa.py and
    * ightly/models/modules/ijepa_timm.py
  • Keeps StoP disabled by default to preserve existing behaviour.
  • Adds a new test tests/utils/test_stochastic_positional_embedding.py to test the behaviour of StoP function while disabled and enabled.

Additional details:

  • StoP can be used for masked image modelling (MIM) models and this PR uses it for IJEPA. It can be used for MAE as well, but currently MAEDecoder in lightly/models/modules does not provide functionality to modify the masks directly, these are modified while creating the MAE class (examples/pytorch/mae.py). These examples can be modified to evaluate StoP.

@gabrielfruet

Copy link
Copy Markdown
Contributor

/review

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.23077% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.33%. Comparing base (9c0346f) to head (9c27f5f).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
lightly/models/modules/ijepa.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1963      +/-   ##
==========================================
- Coverage   86.36%   86.33%   -0.04%     
==========================================
  Files         172      172              
  Lines        7358     7371      +13     
==========================================
+ Hits         6355     6364       +9     
- Misses       1003     1007       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kir-7

kir-7 commented Jul 7, 2026

Copy link
Copy Markdown
Author

@gabrielfruet The failing check seems to be regarding test coverage for ijepa.py for from_vit_encoder, should I add tests for this? And is there anything else I need to modify?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants