Skip to content

feat: convert output to float in distill#320

Open
stephantul wants to merge 1 commit intomainfrom
cast-to-float
Open

feat: convert output to float in distill#320
stephantul wants to merge 1 commit intomainfrom
cast-to-float

Conversation

@stephantul
Copy link
Copy Markdown
Contributor

This PR pro-actively converts transformer outputs, both pooler and hidden, to float. We pro-actively checked for bfloat16, but we also see problems with upcasting with float16.

Tested with: https://huggingface.co/Alibaba-NLP/gte-modernbert-base

@stephantul stephantul requested a review from Pringled April 16, 2026 13:31
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
model2vec/distill/inference.py 98.31% <100.00%> (+1.65%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants