Brendan Hogan
@brendanh0gan
AI/ML Research @morganstanley || PhD in CS @cornell 2024 || Abingdon Elementary 2005 https://neuron-by-neuron.ghost.io
first preliminary plot looking at all outputs
last two runs of the biggest scale project ive ever done 🥲 training 1.5b, 3b, 7b, 14b, 32b models - pretraining + rejection sampling to build a ds + supervised finetuning + reinforcement learning now time to write
last two runs of the biggest scale project ive ever done 🥲 training 1.5b, 3b, 7b, 14b, 32b models - pretraining + rejection sampling to build a ds + supervised finetuning + reinforcement learning now time to write
i hate beginning a paper with “ai is rapidly advancing” or whatever but I also can never think of another way to start that doesnt sound like it’s trying too hard or feels awkward
there should be an o3 powered thrift store app, take a picture of something, it asks for clarifying details/pictures - then gets you an estimated resell value then if you purchase it auto makes the listing
training 1.5b, 3b, 7b, 14b and 32b models for the same task really gives you a good feel for when certain properties emerge
recent studies have shown that ancient studies have been confirmed
advice id love to be able give myself 3 weeks ago: 1. get the evaluation perfect before doing any training 2. Use vllm for everything possible (and parallel calls) 3. Have fun :)
it felt like any dumb idea I had grpo could magically learn but now that I have a verifiable coding environment with test cases it can't learn at all
intuition people --> see too many patterns --> cant explain to others reasoning people --> see less, but more precisely --> good explanations former perceives latter as blind, and latter perceives former as delusional
i have the perfect benchmark but cant tell you what it is or even the results (too risky)
richard linklater made before sunrise just to have cash for his art film school of rock
the trial of tim heidecker has got to be one of the best pieces of media ever produced