Feature request
I would like to propose adding FRoD as a new tuner to huggingface/peft.
FRoD (Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence) is a parameter-efficient fine-tuning method that uses a shared full-rank basis together with sparse learnable rotational updates. The goal is to preserve the expressive capacity of full-rank adaptation while keeping the number of trainable parameters small.
I believe FRoD is a good fit for PEFT because:
- it targets standard linear layers in Transformer blocks,
- it follows the adapter-style fine-tuning workflow used by other PEFT tuners,
- it can be integrated through
get_peft_model(...),
- and it provides a structured full-rank alternative to low-rank adaptation methods.
Paper:
Code:
If this proposal is accepted, I plan to adapt FRoD to the latest upstream PEFT code structure and submit the implementation as a PR.
Your contribution
Yes. I can prepare and submit a PR for the core FRoD tuner.
My planned contribution includes:
- adding
PeftType.FROD,
- adding
src/peft/tuners/frod/ with config.py, layer.py, model.py, and __init__.py,
- registering the tuner in the current PEFT registration flow,
- adding save/load support,
- adding tests for integration with PEFT model wrapping,
- and adding documentation for the new method.
I will target the latest peft main branch rather than submitting an old fork directly. If preferred, I can keep the first PR focused on the core tuner and submit runnable task examples in a follow-up PR.
Feature request
I would like to propose adding FRoD as a new tuner to
huggingface/peft.FRoD (Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence) is a parameter-efficient fine-tuning method that uses a shared full-rank basis together with sparse learnable rotational updates. The goal is to preserve the expressive capacity of full-rank adaptation while keeping the number of trainable parameters small.
I believe FRoD is a good fit for PEFT because:
get_peft_model(...),Paper:
Code:
If this proposal is accepted, I plan to adapt FRoD to the latest upstream PEFT code structure and submit the implementation as a PR.
Your contribution
Yes. I can prepare and submit a PR for the core FRoD tuner.
My planned contribution includes:
PeftType.FROD,src/peft/tuners/frod/withconfig.py,layer.py,model.py, and__init__.py,I will target the latest
peftmain branch rather than submitting an old fork directly. If preferred, I can keep the first PR focused on the core tuner and submit runnable task examples in a follow-up PR.