Fine tune a 70B language model at home
603 by jph00 | 146 comments
Jeremy from Answer.AI here. This is our first project since launching our new R&D lab at the start of this year. It's the #1 most requested thing I've been hearing from open source model builders: the ability to use multiple GPUs with QLoRA training. So that's why we decided to make it our first project. Huge thanks to Tim Dettmers for helping us get started to this -- and of course for creating QLoRA in the first place! Let me know if you have any questions or thoughts.
Subscribe to:
Post Comments (Atom)
New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy
New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy 379 by weinzierl | 72 comments
-
Boards are dangerous to founder/CEOs 574 by tosh | 264 comments
-
Samsung plans $17B chip plant in Taylor, Texas 515 by kungfudoi | 370 comments
-
Stepping Back from Speaking 502 by alfredbez | 124 comments
No comments:
Post a Comment