Skip to content

Returned xlora_scalings should be copied to the same device as the model #35

Description

@data-hound

https://github.com/EricLBuehler/xlora/blob/8217992c8dcf5a1c025749a48e665c59cc0b6350/src/xlora/xlora_insertion.py#L65C16-L65C30

The xlora_scalings generated as:

xlora_scalings: Tensor = self.model.internal_xlora_scalings[:, :, self.layer_number, :]

should be put on the same device as where the model exists to enable return x * scalings in https://github.com/EricLBuehler/xlora/blob/8217992c8dcf5a1c025749a48e665c59cc0b6350/src/xlora/xlora_insertion.py#L40C9-L40C28

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions