Marco Ciccone @ ICML 🇨🇦
@mciccone_AI
Postdoctoral Fellow @VectorInst - Collaborative, Decentralized, Modular ML - Competition chair @NeurIPSConf 2021, 2022, 2023 - PhD @polimi ex @NVIDIA @NNAISENSE
🚨 Life update 🚨 I moved to Toronto 🇨🇦and joined @VectorInst as a Postdoctoral Fellow to work with @colinraffel and his lab on collaborative, decentralized, and modular machine learning to democratize ML model development. Exciting times ahead! 🪿
Nice parallel!
ML labs seem to have quite the parallels with F1 teams. Over the top of my mind:
Dan’s talk was a masterclass — Go watch the recording. Super clear, packed with results, and genuinely one of the most well-delivered talks I’ve seen in a while.
Happening in 30 minutes in West Ballroom A - looking forward to sharing our work on Distillation Scaling Laws!
6/ @JoshSouthern13 and I be at #ICML2025, poster session Tuesday — stop by and chat if you're around! ... I would also be happy to meet up and chat about graphs, (graphs and) LLMs, and how to detect their hallucinations 😳 Feel free to reach out!
I'll discuss distributed learning on Saturday, July 12. First, I'll cover current methods needing high bandwidth, then next-generation methods for decentralized learning
Distributed Training in Machine Learning🌍 Join us on July 12th as @Ar_Douillard explores key methods like FSDP, Pipeline & Expert Parallelism, plus emerging approaches like DiLoCo and SWARM—pushing the limits of global, distributed training. Learn more: tinyurl.com/9ts5bj7y
Even crazier is that the rmsprop algorithm was introduced by @geoffreyhinton in his neural networks course, and we cite his slides - so it's fine!
even crazy part is there's no muon paper
Fantastic work from @allen_ai - asynchronous training of MoEs on private datasets and a domain-aware router. Akin to cross-silo FL, but no synchronization and no fear of heterogeneity anymore on LLMs. Nice, clean, and of course modular!
The bottleneck in AI isn't just compute - it's access to diverse, high-quality data, much of which is locked away due to privacy, legal, or competitive concerns. What if there was a way to train better models collaboratively, without actually sharing your data? Introducing…
The Department of Statistical Sciences in the Faculty of Arts and Science at the University of Toronto invites applications for a full-time teaching stream position in the area of Statistical Sciences. The appointment will be at the rank of Assistant Professor, Teaching Stream…
This is probably the most compelling result of the paper. Clear improvements at scale — 1K, 10K, and 40K clients, beyond synthetic benchmarks like CIFAR. You know what that means.
Faster local optimization by recycling past outer gradients! Incredibly proud of this work and collaboration. We have a bunch of ideas in the pipeline, looking forward to the next one with @RickZack96!
Do you feel FL research is stuck with methods that do not work well in realistic scenarios? 🤔 🫵We got you! Introducing 🚀Generalized Heavy-Ball Momentum (GHBM)🚀, accepted at TMLR: the FL algorithm with both SOTA theoretical guarantees and much better empirical results. 🧵1/9
The videos of the workshop are now online! youtube.com/playlist?list=…
Workshop alert 🚨 We'll host in ICLR 2025 a workshop on modularity, encompassing collaborative + decentralized + continual learning. Those topics are on the critical path to building better AIs. Interested? submit a paper and join us in Singapore! sites.google.com/corp/view/mcdc…
Despite the impressive quality of video generation, what I don't understand about #Veo3 is how it is possible to create such models without infringing on copyright. Did they make a deal with the movie industry?
starting soon, in hall 4 #3 see you all there and follow that thread :)
the workshop is tomorrow! and it will be livestreamed at iclr.cc/virtual/2025/w… for those not in Singapore
Sadly, I won't be able to travel and attend our Modularity workshop :( I'm sure I'll miss many amazing discussions and talks. Luckily @Ar_Douillard and @derylucio will be there to hold down the fort! Proud to be part of the organizing team - can't wait to meet you all next time!
the workshop is tomorrow! and it will be livestreamed at iclr.cc/virtual/2025/w… for those not in Singapore