Andrej Karpathy
@karpathy
Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
The hottest new programming language is English
Love this! Supercharger, diner, … but really a kind of exhibit for the future. Plotting a road trip SF -> LA to charge Shadowfax
Tesla Diner & Supercharger in Hollywood, LA Open 24/7, starting now
Hi @gmail does the "report phishing" button do anything
"Using a better model for analysis" 🤨 I didn't realize I was using haiku all this time, no idea when claude code snuck this one in rofl.
Diffusion video models but now - **realtime**! Simple video filters are real-time but can only do basic re-coloring and styles. Video diffusion models (Veo and friends) are magic, but they take many seconds/minutes to generate. MirageLSD is real-time magic. Unlike simple video…
Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):
I always learn a lot more from in-depth analysis of few random cases over dashboards of aggregate statistics across all cases. Both projections can be helpful but the latter is disproportionately pervasive.
Scaling up RL is all the rage right now, I had a chat with a friend about it yesterday. I'm fairly certain RL will continue to yield more intermediate gains, but I also don't expect it to be the full story. RL is basically "hey this happened to go well (/poorly), let me slightly…
I often rant about how 99% of attention is about to be LLM attention instead of human attention. What does a research paper look like for an LLM instead of a human? It’s definitely not a pdf. There is huge space for an extremely valuable “research app” that figures this out.
I'm constantly irritated that I don't have time to read the torrent of cool papers coming faster and faster from amazing people in relevant fields. Other scientists have the same issue and have no time to read most of my lengthy conceptual papers either. So whom are we writing…
This is what the ideal grocery store looks like. Minimally processed (NOVA Group 1) food only (no "edible food-like substances"), organic, local, fresh. Food should not be more complex than this, yet I don't believe this exists.
my weekend project to learn about bluetooth mesh networks, relays and store and forward models, message encryption models, and a few other things. bitchat: bluetooth mesh chat...IRC vibes. TestFlight: testflight.apple.com/join/QwkyFq6z GitHub: github.com/jackjackbits/b…
Knowledge makes the world so much more beautiful.
Test-based certification is the only way forward in food, eager to see more over time. Food is not simple anymore - it is a complex, industrial product with global supply and processing chains. Contamination can be introduced in many stages along the way from farming to harvest,…
Something new and exciting is here Dog and cat food toxin testing + fund your pet's food + if brand claims results, your money comes back + fund more tests Together we can rapidly test all US dog and cat food. Initial results Blueprint Quantified tested 22 mass-market…
Love this project: nanoGPT -> recursive self-improvement benchmark. Good old nanoGPT keeps on giving and surprising :) - First I wrote it as a small little repo to teach people the basics of training GPTs. - Then it became a target and baseline for my port to direct C/CUDA…
Recently, there has been a lot of talk of LLM agents automating ML research itself. If Llama 5 can create Llama 6, then surely the singularity is just around the corner. How can we get a pulse check on whether current LLMs are capable of driving this kind of total…
May your regularizer be strong, lest you RLHF to slop.
+1 for "context engineering" over "prompt engineering". People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window…
I really like the term “context engineering” over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.
Mildly obsessed with what the "highest grade" pretraining data stream looks like for LLM training, if 100% of the focus was on quality, putting aside any quantity considerations. Guessing something textbook-like content, in markdown? Or possibly samples from a really giant model?…