Tanishq Kumar

trained a nanoGPT? feeling behind before o4-mini? 🚨🚨i'm open-sourcing beyond-nanoGPT, an internal codebase to help people go from LLM basics to research-level understanding. 🚨🚨 it contains thousands of lines of from-scratch, annotated pytorch implementing advanced…

i find it entertaining that under the hood, most open source "GRPO" implementations (eg. trl) by default actually implement REINFORCE with monte carlo group-advantages (by not reusing rollouts & making clipping/ratios redundant)

discussing classic literature with frontier models really exposes their overconfidence. presumably because Dickens appears often in high-quality pretraining corpora, gpt-4o believes it can respond to all my questions without searching, resulting in near-constant hallucination.

hello friends, i will be in SF for july/aug/some of sept - if you know any summer sublets/rentals in the city i should look at on short notice, dm me :)