The Fine-Tuning Index / PEFT & LoRA / #114
Guitaricet/relora
by Guitaricet · PEFT & LoRA · updated 2y ago
Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
30
momentum
474
stars
42
forks
#114
rank
deep-learningdistributed-trainingllamanlppefttransformer
View on GitHub →