Akshat Bubna
@akshat_b
building @modal_labs
You either die making fun of SF billboards, or live long enough to be on one of them.
This graphic is, quite literally, the "money figure"
ICYMI, open models for transcription are very good now. In just the last few months, we've gotten @NVIDIA Parakeet and Canary, @kyutai_labs STT, and @MistralAI Voxtral. Running your own transcription at scale is now 100x faster and 100x cheaper than using a proprietary API.
My talk for @aiDotEngineer on what I think every person working with language models needs to know about GPUs is now available! - Latency lags bandwidth. - GPUs embrace bandwidth. - Don't be scared of N squared. - Use the Tensor Cores, Luke! youtube.com/watch?v=y-UGrY…
Tokasaurus, the "little LLM engine that could" by @jordanjuravsky and @EyubogluSabri of @HazyResearch/@ScalingIntelLab, is capable of some pretty impressive perf. We replicated their report of >80k tok/s for 16bit LLaMA 3.1 8B on Large Language Monkeys GSM8K - and you can too!
the story was, in June, I was so excited about a "Marimo cloud" that I went to their office & we hacked together the first prototype of a Modal Sandbox backend for Marimo in a couple hours. just a month later, guess they have a whole new cloud-based product, still on Modal :D
Announcing molab: a cloud-hosted marimo notebook workspace with link-based sharing. Experiment on AI, ML and data using the world’s best Python (and SQL!) notebook. Launching with examples from @huggingface, @weights_biases, and using @PyTorch marimo.io/blog/announcin…
The holy trinity of serverless GPU has been achieved internally at @modal_labs: ✅ Custom filesystem optimized for container cold start ✅ CPU snapshot+restore ✅ GPU snapshot+restore Last one is working internally but not released yet – stay tuned!
Lovable would not have been nearly as lovable without @modal_labs @Cloudflare @supabase @github @vite_js powering pooled VMs, hosting, backend, IDE sync, visual edits, and a lot of other great things. Thank you to all less known devs out there building stellar tech.
I was really bullish on execution environments for AI when we first built Sandboxes more than 2 years ago. Still had no idea how quickly companies like @lovable_dev would take off. This is just the beginning.
@lovable_dev uses Modal Sandboxes to power 20,000 concurrent code execution sessions in production 🩷💚 Every time you create an app in Lovable, that AI-generated code runs in a secure Modal Sandbox!
Two years ago, I built my first Modal app -- a diffusion-based QR code generator. The results were sometimes good, sometimes terrible. It's a common story: a cool AI demo that's not robust enough to be useful. Here's how we engineered our way from the left image to the right.
Big thank you to stellar engineering team at @modal_labs, who provided some critical infrastructure for this effort. They are a joy to work with.
We’re excited to introduce Chai-2, a major breakthrough in molecular design. Chai-2 enables zero-shot antibody discovery in a 24-well plate, exceeding previous SOTA by >100x. Thread👇
We got Flux generations to be 3x faster! Lots of little gems in here, e.g. on tensor memory layouts, first-block caching, why fused QKV works, etc.
You can go FLUX yourself! In this blog post, we walk through some optimizations that speed up @bfl_ml FLUX.1-dev enough that you can run it on @modal_labs at a price and latency competitive with API providers.
True story
Our biggest test of scale yet: @lovable_dev’s used 1M+ Modal Sandboxes, with 20k concurrently at peak, for their AI Showdown this past weekend 😎 That’s millions of secure, isolated executions of LLM-generated code in the last 48 hours.
The most valuable thing I got from my time at @scale_AI was the opportunity to work with incredible people. This is why I tell everyone early in their career to think hardest about who they want to be working long hours in the trenches with. It matters more than anything else,…
Linear programming for fun, profit and _reliability_: modal.com/blog/resource-…
💚
Brian Wilson of Beach Boys passed away yesterday sadly, so please remember you wouldn’t be “vibecoding” without him
Sad because I really wanted to end up on the shame list at 0ver.org
After many years of being on 0.x, we've finally shipped the 1.0 of the Modal Python SDK! It took a lot of client changes to get here, but going forward it means a way more stable client interface.