Daniel Han
@danielhanchen
Building @UnslothAI. Finetune train LLMs faster. LLMs bug hunter. OSS package http://github.com/unslothai/unsloth. YC S24. Prev ML at NVIDIA. Hyperlearn used by NASA.
We managed to fit Llama 3.1 8B < 15GB with GRPO! Experience the R1 "aha moment" for free on Colab! Phi-4 14B also works with @UnslothAI & vLLM is now integrated allowing 20x faster inference! LoRA with GRPO also just work! 1. We removed double memory usage during vLLM serving…
You can now reproduce DeepSeek-R1's reasoning on your own local device! Experience the "Aha" moment with just 7GB VRAM. Unsloth reduces GRPO training memory use by 80%. 15GB VRAM can transform Llama-3.1 (8B) & Phi-4 (14B) into reasoning models. Blog: unsloth.ai/blog/r1-reason…
this pod was all about optimizations, torch dot compile, benchmaxxing and beyond. @danielhanchen has some exciting news too coming up for unsloth⚡️ pod coming soon.
Thank you to @Kimi_Moonshot for quickly addressing my queries on the correct system prompt for Kimi K2! We'll be re-uploading all BF16 + dynamic @unslothai GGUFs with fixed tool calling & the new sys prompt! Sys prompt = "You are Kimi, an AI assistant created by Moonshot AI."
We’ve updated Kimi K2’s chat template to make tool calls more robust. What’s changed: - updated default system prompt - always use model-returned tool_id in multi-turn tool calls, which is more reliable. - If `arguments` in tool call is already a string, don't apply `tojson` to…
a complete guide to fine-tuning LLMs in 15 minutes. this covers how to use Unsloth to fine-tune models in notebooks, how to create your custom chat templates, datasets, and more. this guy deserves much more attention. here is also the full 1:20 hour video going in detail:
coming up @groundzero_ai talks x @danielhanchen (ceo, unsloth) soon. drop your questions/thoughts/insights around training, fine-tuning, RL, scaling or llms in general. (dms open) stay tuned⚡
Highly recommend this Stanford lecture video with @_jasonwei and @hwchung27 :) It's one of my favorites on scaling laws and the bitter lesson! Also Hyung's "Don't teach. Incentivize" video: youtube.com/watch?v=kYWUEV… youtube.com/watch?v=3gb-Zk…
LOVE ITT! You can run Kimi K2 (1T token MoE) on a single M4 Max 128GB VRAM (w/ offloading) or a single M3 Ultra (512GB) 🔥 The model was released less than 72 hours ago - love how fast the community optimises open weights - kudos to @UnslothAI 🤗 huggingface.co/unsloth/Kimi-K…
You can utilize our Gemma 3n multimodal and fine-tuning Kaggle notebook for any submission to the $150,000 challenge! The $10,000 is specifically for the Unsloth track - but you can submit it for the main track as well! Kaggle notebook: kaggle.com/code/danielhan…
We’ve teamed up with @GoogleDeepMind for a challenge with a $10,000 Unsloth prize! 🦥 Show off your best fine-tuned Gemma 3n model using Unsloth, optimized for an impactful task. The entire hackathon has $150,000 prizes to be won! Kaggle notebook: kaggle.com/code/danielhan…
🦥 Fine-tuning with @UnslothAI now supports Gemma 3n! ✨ Friendly reminder: the Gemma 3n models can understand not just text and code, but also images, audio, video, and a whole lot more.
You can now fine-tune Gemma 3n for free with our notebook! Unsloth makes Google Gemma training 1.5x faster with 50% less VRAM and 5x longer context lengths - with no accuracy loss. Guide: docs.unsloth.ai/basics/gemma-3… GitHub: github.com/unslothai/unsl… Colab: colab.research.google.com/github/unsloth…
Gemma 3N quirks! 1. Vision NaNs on float16 2. Conv2D weights are large FP16 overflows to infinity 3. Large activations fixed vs Gemma 3 4. 6-7 training losses: normal for multimodal? 5. Large nums in msfa_ffn_pw_proj 6. NaNs fixed in @UnslothAI Details: docs.unsloth.ai/basics/gemma-3…
You can now fine-tune Gemma 3n for free with our notebook! Unsloth makes Google Gemma training 1.5x faster with 50% less VRAM and 5x longer context lengths - with no accuracy loss. Guide: docs.unsloth.ai/basics/gemma-3… GitHub: github.com/unslothai/unsl… Colab: colab.research.google.com/github/unsloth…
Huge thanks to everyone who attended our @Google & @UnslothAI Gemma developer meetup yesterday! 🦥 Was amazing meeting you all & thank you to @blueviggen for hosting the event with us. Thank you to the Google speakers: @DynamicWebPaige, Doug Reid, @imayank42, @GrmCameron and of…
💎 Celebrating the official release of Gemma 3n with the inaugural Gemma Community meetup at @Google San Francisco, cohosted with @Unsloth! Great presentations from the Unsloth founders on agents, the Gemma team on architectural internals, and how to craft effective evals.
We’re fully releasing Gemma 3n, which brings powerful multimodal AI capabilities to edge devices. 🛠️ Here’s a snapshot of its innovations 🧵
Excited to see you all tomorrow for our Google Gemma & Unsloth developer meetup! 🦥 We'll be having @Grmcameron from @ArtificialAnlys and @DynamicWebPaige & more amazing talks! Location has been updated so please check & if you need help please DM me! lu.ma/gemma-unsloth
r/LocalLlama is back!! reddit.com/r/LocalLLaMA/c…
We need r/LocalLlama back :( Hopefully a good neutral moderator takes the reins asap!
We need r/LocalLlama back :( Hopefully a good neutral moderator takes the reins asap!
Managed to mostly fix Mistral 3.2 tool calling for GGUF / transformers! 1. 3.2 tool calling is different from 3.1 2. timedelta(days=1) (yesterday) changed with a if-else - supports 2024 to 2028 dates - so now word for word same sys prompt! 3. Made experimental FP8 quant as well!
Mistral releases Small 3.2 (24B), a new update to their 3.1 model. 🔥 The model performs much better on 5-shot MMLU (CoT), instruction following and function/tool calling! Run locally with FP8 or 16GB RAM using our Dynamic GGUFs with fixed chat template: huggingface.co/unsloth/Mistra…
New tutorial: how to build a synthetic dataset with recent information and use it to fine tune with @UnslothAI Check out the collab: colab.research.google.com/drive/1JK04IBE… Steps in the 🧵