Skip to content

Minimax PDD Distillation #512

Description

@simlang

Great job on the LoRA! The best out so far for sure!

I also played around with PDD a bit but never managed to reproduce the results from the paper. Do you have any insights into the algorithm, which hyperparameters matter the most etc?

Sharing your setup would be amazing!

Thanks!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions