Derek Lim
@dereklim_lzh
Post-Training @OpenAI
Our new paper covers: Neural nets on eigenvectors / eigenspaces, transformers on graphs, universal approximation of invariant functions, graph positional encodings, generalizing spectral graph neural networks, and more!! Thread about SignNet and BasisNet: 1/9
I joined the post-training team at OpenAI and moved to SF a month ago! excited to be working on improving frontier models
When and why are neural network solutions connected by low-loss paths? In our #ICML2025 paper, we show that mode connectivity often arises from symmetries—transformations of parameters that leave the network’s output unchanged. Paper: arxiv.org/abs/2505.23681 (1/6)
Check out our new paper on learning from LLM output signatures: the |tokens| x |vocab|+1 matrix of predicted next-token probs and actual next-token prob. It provably generalizes several existing approaches and is great at hallucination / data contamination detection tasks!
📢 Introducing: Learning on LLM Output Signatures for Gray-box LLM Behavior Analysis [arxiv.org/pdf/2503.14043] A joint work with @ffabffrasca (co-first author) and our amazing collaborators: @dereklim_lzh @yoav_gelberg @YftahZ @el_yaniv @GalChechik @HaggaiMaron 🧵Thread
What if models could be the data🤔Find out at @iclr_conf #ICLR2025 Join the 1st workshop on Model Weights as a New Data Modality. We're training networks on model weights for a wide variety of tasks. Featuring an amazing lineup of papers & speakers🚀 🗓️Sunday 9-17 📍Topaz 220-225
Boston Symmetry Day is happening TODAY at Northeastern University’s Columbus Place and Alumni Center (716 Columbus Ave, 6th floor)! Breakfast starts at 9 AM, but talks are happening throughout the day, followed by a social. We’ll see you there!
Speakers are confirmed and registration is open for the third Boston Symmetry Day! Come increase the order of the group!
Registration is now open for Boston Symmetry Day on March 31! Sign up by March 21st at docs.google.com/forms/d/e/1FAI… We have an exciting lineup of speakers (see our website: bostonsymmetry.github.io ) Also featuring a poster session so you have a chance to present your awesome work!
Save the date -- Boston Symmetry Day 2025 will be held on March 31st, at Northeastern University! Speakers and sponsors to be announced in the coming weeks, but you can expect another great lineup of talks, networking, and posters. We'll see you there!
did you know you've been doing test-time learning this whole time? transformers, SSMs, RNNs, are all test-time regressors but with different design choices we present a unifying framework that derives sequence layers (and higher-order attention👀) from a *single* equation 🧵
Excited to share our new 7B LLM @LiquidAI_ . Strong evals on diverse tasks (including several evals from the synthetic arena that I lead), long context strength at low memory cost, and edge-device / on-prem deployment options for customers. Great work from the team :).
Introducing LFM-7B, our new best-in-class language model in English, Arabic, and Japanese optimized to be the substrate for private enterprise chat, code, fast instruction following, and agentic workflows. 1/
Tune in to GLOW next week for my talk on metanetworks!
🌟 GLOW 2025 kicks off with a super session in January! 🎙️ Hear from our amazing speakers Clayton Sanford and @dereklim_lzh. 🗓️ Jan 15th, 17 CET on Zoom. 🌐 Details & sign-up: sites.google.com/view/graph-lea…
We raised a $250M Series A led by @AMD Ventures to scale Liquid Foundation Models and accelerate their deployment on-device and at enterprises liquid.ai/blog/we-raised…
Presenting our paper today (Thursday) at NeurIPS at 11am! East Exhibit Hall A-C #4402 Stop by if you want to learn about our insights on weight space geometry, loss landscapes, model merging etc. Reach out to me if you want to chat about anything else at NeurIPS too!
New version + code for our NeurIPS paper is now out: “The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof.” We study how symmetries in weight-space impact optimization and loss landscape geometry of neural nets, via "counterfactual" NNs w/o symmetries. 1/n
Are you interested in symmetry and geometric deep learning? Join us on November 25th for a Boston Symmetry Group social + poster session at MIT’s Stata Center! This event is open to the public, but fill out this quick form to attend: forms.gle/bjykpjjzsycQ74…
We are organizing the second learning meets geometry, graphs, and networks meetup (Nov 21-22) in Jersey City @LoGNYCMeet! Last time we had roughly 100 attendees and 15 talks! Jointly organized with Ali Parviz, @yingheng_wang, @Abel0828 and @dereklim_lzh #NYC