The Fine-Tuning Index / PEFT & LoRA / #149
BorealisAI/flora-opt
by BorealisAI · PEFT & LoRA · updated 1y ago
This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.
23
momentum
107
stars
5
forks
#149
rank
deep-learningflaxjaxlarge-language-modelsloramemory-efficient-tuningoptaxpeftrandom-projectiontransformers
View on GitHub →