tokenbender
@tokenbender
playing reward lottery • child of oss • experimentalist
e/xperiments philosophy: * No muh-favourite-architecture * Stay GPU-poor, stay foolish (literally) * Forever behind SoTA, always learning * Everyone sleeps on smol models * Data curation/evaluation is the MOAT * Synthetic dataset creation is art
“if i tell you which one, it won’t be accepted”
Anyone knows adam?
Spent the last 4 hours investigating an implementation plan. I have a WIP mock-up ready, which uses the official runfiles provided by NVIDIA. I looked at conda first, and it used a disgustingly terrible and complicated form of package management for large groups like cuda. I…
It would be incredibly useful if uv could handle CUDA installations, like conda does. This is the only thing preventing uv from being perfect for me cc @charliermarsh I'm willing to help w/ the implementation if you'd like this feature but don't have the bandwidth to implement.
it's been probably 5 months now since I've bookmarked anything. it's great.
Japanese scientists successfully removed the extra chromosome causing Down syndrome in lab cells.
i probably haven't endured any other ai product as much as claude code. ~ 1 tok/s output :(
just a reference for myself - diffusion, data augmentation/morphosis and RL feel meant for each other.
probably the most beautiful thing about RL in gen ai is being able to learn from the same thing over and over. it is high sample efficiency though trades data with compute.
i am prone to chasing mental butterflies and finding myself far away from home. always have been.
signatures to look for in ai writing - > "it isn't just x, it is y" > narrative-philosophical-poetic section headings "The XYZ - A Journey of ABC" > overuse of symbolism and lofty adjectives - "stands as a testament", "plays a vital role", "underscores its importance" >…
once you see it you can't unsee it. and it's everywhere
how can you not love these guys? putting so much care in enriching the paper with more insights. be blessed.
K2 report is out now. I guess some guys are also waiting for the Kimi-Dev report.👀 We're still working on adding more insights to the report so we can deliver a paper that's more valuable for everyone.😎 github.com/MoonshotAI/Kim…