Charlie Hou
@hou_char
AI Research Scientist @ Meta | synthetic data, reinforcement learning, ranking/retrieval, privacy
like everyone else i am hopping on the blog post trend gene.ttic.edu/blog/incomplet…
Come by the poster if you want recommendations on cool restaurants to try in Vancouver 😃!
[#ICML2025] Have you ever wanted to train LLMs on distributed private data but were blocked by model size or privacy constraints 😔? Here’s a solution: Introducing 🌸POPri (Policy Optimization for Private Data)! Poster 🗓️ today at 4:30pm PT, 📍East Exhibition Hall A-B E-1006
There are many algorithms for constructing pre-training data mixtures—which one should we use? Turns out: many of them fall under one framework, have similar issues, and can be improved with a straightforward modification. Introducing Aioli! 🧄 1/9
With Chameleon we showed that early fusion mixed modal LLMs can deliver strong improvements over unimodal and late fusion alternatives, however with this paradigm shift how do we rethink our core model architecture to optimize for native multimodality and efficiency? We…
1/n Introducing MoMa 🖼, our new sparse early-fusion architecture for mixed-modal language modeling that significantly boosts pre-training efficiency 🚀 (arxiv.org/pdf/2407.21770). MoMa employs a mixture-of-expert (MoE) framework with modality-specific expert groups. Given any…
Hey everyone! I'll be presenting PrE-Text (arxiv.org/abs/2406.02958) at ICML 2024 on Tuesday July 23 😊! Oral is 5:15pm-5:30pm in Hall A2, and poster is 1:30pm-3pm Hall C 4-9 #2307.