Scientific Methods for Understanding Deep Learning

One epoch is not all you need! Our paper, Emergent properties with repeated examples, with @KempeLab, won the NeurIPS24 Debunking Challenge, organized by the Science for Deep Learning workshop, @scifordl arxiv.org/abs/2410.07041

Congratulation to the winners of our debunking challenge @f_charton and @KempeLab for "Emergent properties with repeated examples" 🥳🎉

What a great panel @scifordl moderated by @ZKadkhodaie and @FlorentinGuth ! #NeurIPS2024

It's time for our panel session! @SuryaGanguli @EeroSimoncelli @andrewgwils @yasamanbb and Mikhail Belkin discuss science, understanding and deep learning. 👀 Hosted by our amazing organizers @ZKadkhodaie and @FlorentinGuth.

Last talk of the day (before our panel session), Mikhail Belkin shares his personal experience and his learnings from experiments (in an alchemist way? 😎)

Hopfield models are hot again to initialize memorization and generalization in diffusion models in @baophamhq contributed talk! 🧐

Poster session still going on in West meeting rooms 205-207! Come say hi and check your favorite posters (also there is decaf)

Moving onto David Krueger's talk about Input Space Mode Connectivity in Deep Neural Networks! Come learn about how you can interpolate between loss minimizers.

We are back with a contributed talk from @ccperivol about some intriguing properties of sotfmax in Transformer architectures!

Absolutely!

Can LLMs plan? The answer might be in @HanieSedghi's talk. Only one way to know, join us in West meeting rooms 205-207

Excited for the NeurIPS workshops today! I'll be speaking at: (1) Science for DL (panel, 3:10-4:10, scienceofdlworkshop.github.io/schedule/) (2) "Time Series in the Age of Large Models" (talk, 4:39-5:14, neurips-time-series-workshop.github.io).

We need more of *Science of Deep Learning* in the major ML conferences. This year’s @NeurIPSConf workshop @scifordl on this topic is just starting, and I hope it is NOT the last edition!!!

Now onto @SuryaGanguli talk, investigating the creativity in diffusion models, i.e. why do diffusion models fail to learn the score function and why it's a good thing.