The Fine-Tuning Index / PEFT & LoRA / #117
Guitaricet/relora
by Guitaricet · PEFT & LoRA · updated 2y ago
Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
30
momentum
475
stars
42
forks
#117
rank
deep-learningdistributed-trainingllamanlppefttransformer
View on GitHub →