Collin McMillan

It's exciting that LLMs can describe code for us, but now the problem is knowing if that output is any good. This paper makes great observations, to which I would add that in my experience models get saturated after a million examples or so. @vitaleantonio7 , @AntonioMastro2 ,…

Federal debt to SP500 market cap ratio is somewhat stable and actually relatively low at the moment. What's recent is that SP500 and debt are greater than GDP. Made this out of curiosity, numbers approx. I have no idea what this means, but it's interesting! @1ArmedEconomist

Evidence is continuing to accumulate questioning "the prevailing assumption that increasing model parameters is always the optimal path forward." Instead, blend classic and neural ideas. Velasco, Garryyeva, Palacio, @AntonioMastro2 , @Denys19674549 arxiv.org/abs/2502.01806

🚀 Our paper "Explaining GitHub Actions Failures with LLMs" has been accepted at @icpcconf! We explore how LLMs help diagnose GitHub Actions failures, revealing key challenges & insights. 🔗 Read it: lnkd.in/dWcB4uJB Huge thanks to all co-authors! #GitHubActions #LLM

More evidence of LLMs struggling to demonstrate an understanding of code even though they can write it. Interesting read by Jain and Purandare on concurrent code and LLMs. arxiv.org/abs/2501.14326

The blending of LLMs with older rule-based approaches continues. DeepSeek etc. get all the attention but a lot of the front line really looks like this. A great example in converting C to Rust. @vikramnitin9 , @r_krsn, Valle, @baishakhir arxiv.org/abs/2501.14257

A useful taxonomy of the types of things people say to AI during software development. @ctreude , @gerosa_marco arxiv.org/abs/2501.08774

Local models work just as well on code tasks like comment generation as zilla-sized models, without the need to send private data to a third party. (And it's just fun running your own LLMs!) @v_maze_ , Shah, Kulkarni arxiv.org/abs/2501.07857

Compilers sometimes generate error messages that are red herrings. So instead just have GPT make a new message from the code only, *without* giving it the original one. Simple and creative idea! Salmon, Hammer, @_eddieantonio , @brettabecker arxiv.org/abs/2501.05706

No but they could be programmed to do all these things with some simple fine tuning. Cyber security is going to be interesting the next few years.

ICPC Vaclav Rajlich Early Career Achievement Award: the call for nominations of young programming comprehension researchers is open till February 12 conf.researchr.org/track/icpc-202…

My favorite LLM use case is asking it to help me find a song I vaguely remember from years past, with the emotions it evoked and maybe a lyric word or two.

Many security bugs can be fixed with simple code changes, just like many building security issues boil down to unlocked doors, weak windows, or lost keys. Code LLMs can fix those simple bugs. Fakih, Dharmaji, @HalimaBouzidi1 , Araya, Ogundare, Faruque arxiv.org/abs/2501.03446

I imagined being able to do things like this after seeing Penny's book in Inspector Gadget as a kid. Just draw the program you need and the computer figures it out. We really do live in the future. @luisfgomes24 , @VHellendoorn , @JAldrichPL , Abreu arxiv.org/abs/2412.13386