EXO Labs
@exolabs
AI on any device. 12 Days of EXO: https://blog.exolabs.net We're hiring: https://exolabs.net
The future of AI is open source and decentralized
"Exo's use of Llama 405B and consumer-grade devices to run inference at scale on the edge shows that the future of AI is open source and decentralized." - @mo_baioumy x.com/ac_crypto/stat…
‘The number of Macs that can train together coherently doubles every 2 months’; I'll call this 'Cheema's Law'. And it might sound a joke, but it has been remarkable how much progress we've made on this problem in such a short time. When you're working in a space that is mostly…
We're doubling the number of Apple Silicon macs that can train together coherently every 2 months. Our new KPOP optimizer was designed specifically for the hardware constraints of Apple Silicon and implemented using mlx.distributed.
We're doubling the number of Apple Silicon macs that can train together coherently every 2 months. Our new KPOP optimizer was designed specifically for the hardware constraints of Apple Silicon and implemented using mlx.distributed.
New research from Exo done (in part) with MLX on Apple silicon: An algorithm for distributed training that leverages higher RAM capacity of Apple silicon relative to FLOPs and inter-machine bandwidth.
EXO 💛 MLX
New research from Exo done (in part) with MLX on Apple silicon: An algorithm for distributed training that leverages higher RAM capacity of Apple silicon relative to FLOPs and inter-machine bandwidth.
New research from Exo done (in part) with MLX on Apple silicon: An algorithm for distributed training that leverages higher RAM capacity of Apple silicon relative to FLOPs and inter-machine bandwidth.
Paper is out. Link: openreview.net/pdf?id=TJjP8d5…
Paper is out. Link: openreview.net/pdf?id=TJjP8d5…
KPOP is a new DL optimizer designed for large scale distributed training on Apple Silicon. KPOP uses a lot more memory but is more efficient per FLOP than AdamW, so it's a better fit for hardware with a high memory:flops ratio. Some hardware numbers: H100: 80GB, 1000TFLOPS…
This past spring, I spent time with the @exolabs team to work on a new DL optimizer and wiring up clusters of Macs for distributed TRAINING on Apple Silicon. If you’re at ICML, be sure to come by the @ESFoMo workshop (posters 1-2:30pm) this Saturday. I’ll be there to share some…
I’m going to be in Vancouver next week for ICML! Would love to meet anyone involved with distributed training, infrastructure, inference engines, open source AI. I'll be presenting two papers: - EXO Gym - an open source framework for simulating distributed training algorithms…
EXO isn't just for inference.
A new approach to efficient large scale distributed training on Apple Silicon. Most AI research today is focused on traditional GPUs. These GPUs have a LOT of FLOPS but not much memory. They have a low memory:flops ratio. Apple Silicon has a lot more memory available for the GPU…
Apple considered building its own AWS competitor, spending resources evaluating the complexity and cost of cloud infrastructure. Meanwhile, @exolabs has already proved the power of distributed in-house hardware, turning idle Macs into an active, decentralized data center.
Report: Apple looked into building its own AWS competitor 9to5mac.com/2025/07/03/rep… by @mvcmendes
Mac Minis are the ultimate store of value.
"Mac Minis for example are a very good fit" - @karpathy @karpathy shouted out my work on @exolabs in his keynote at @ycombinator AI SUS! Here's the breakdown: Right now most AI workloads run in the cloud where requests from different users are continuously batched together.…
Cost to run DeepSeek R1 (fp8) on Apple Silicon: $20,000 Cost to run DeepSeek R1 (fp8) on H100s: $300,000
.@karpathy shouted out my work on @exolabs at @ycombinator AI SUS! “we use LLMs similarly to mainframes in the ‘70s - compute is timeshared by having a slice in the batch dimension. models will compress over time, and with this we’ll be able to run more on-device”
How long before this gets into the training data?
Just got sent these news articles reporting that @exolabs is founded by @karpathy. wtf??