Cody Blakeney
@code_star
Data Dawg @datologyai | Formerly Data Research Lead @DbrxMosaicAI | Visiting Researcher @ Facebook | Ph.D | #TXSTFOOTBALL fan | http://linktr.ee/code_star
We are looking for a post-training lead at @datologyai we have gpus, you can make them go brrrr
I have never read a truer sentence than “every group of researchers has their own opinion on how configs should be done”
Ours was forked from torchtitan to start but has since been heavily modified (although some of the core remains intact!) For example, every group of researchers has their own opinion on how configs should be done.
I bet pretty soon a Chinese research org drops a LLM scaling laws for RL paper. Closed frontier labs have definitely done this and wont share it, academics havent mastered the data + infra tweaks yet.
I guess my wife is putting the best memes by my coworkers on our fridge. Gotta turn it into a leaderboard
Guys you're still trying to replace active users on this site. I'm telling you we can't do it. But we can recreate them in the aggregate.
If you believe in AGI you would never invest in healthy activities. The technology will come. Vegetables are fundamentally a decelerationist decision.
I could probably make a killing with AI slop work decorations on Etsy.
The exotic optimizers / MoE / better data memes on-page numbers where 200k views, the "do you ever think about tokenizers" was 50k. What we are looking for is a volume of memes that averages 250k a week.
This is what everyone at @datologyai looks like at work btw
everybody want AGI, nobody wanna INSPECT THE TRAINING DATA. ILL DO IT THOUGH!!!!
I understood this reference
and don't tell me there is no green curve.
The dream of the 90s
Car ride thoughts: a ska band should really come out with a brand of dog poop bags and call it ‘Pick it up. Pick it up. Pick it up.’
Car ride thoughts: a ska band should really come out with a brand of dog poop bags and call it ‘Pick it up. Pick it up. Pick it up.’
Extrapolating this trend to robotics, i believe if one is doing sim2real they should prefer Autoregressive > Diffusion (compute bottleneck). But if they are doing real world training then Autoregressive < Diffusion (data bottleneck).. We don't empirically validate this for…
🚨 The era of infinite internet data is ending, So we ask: 👉 What’s the right generative modelling objective when data—not compute—is the bottleneck? TL;DR: ▶️Compute-constrained? Train Autoregressive models ▶️Data-constrained? Train Diffusion models Get ready for 🤿 1/n
zoomers for some reason
Kind of losing it over how the Zoomers have assigned a name to hating robots ("robophobia") and invented a new slur ("clanker")
this
now look at the evals then look at the model outputs on those evals each one 🏋️♂️🏋️♂️
just deeply disappointing to see More Perfect Union blatantly lying that building more market rate housing *causes* homelessness instead of preventing it. Austin is one of the few places in America where rents actually *fell* because it built so much housing!
Billionaires like Elon Musk and Peter Thiel are trying to destroy affordable housing in Austin and replace it with luxury condos. In a city where evictions are at record highs and homelessness is skyrocketing, residents have had enough. Now, they're fighting back.