Polina Kirichenko
@polkirichenko
Research Scientist at FAIR @AIatMeta & visiting researcher at Princeton @VisualAILab prev. PhD at New York University 🇺🇦
Excited to release AbstentionBench -- our paper and benchmark on evaluating LLMs’ *abstention*: the skill of knowing when NOT to answer! Key finding: reasoning LLMs struggle with unanswerable questions and hallucinate! Details and links to paper & open source code below! 🧵1/9
The code and model weights for this paper are finally open! Despite being a little late for releasing them, I hope you will find them useful! Code: github.com/facebookresear… Models: - (ViT-G): huggingface.co/lavoies/llip-v… - (ViT-B): huggingface.co/lavoies/llip-v…
Should we account for the diverse ways that an image can be captioned? In our #ICML2024 paper. We propose Llip — a Vision Language Pretraining method that models the diverse ways in which an image can be captioned! 📜arxiv.org/abs/2405.00740 🧵👄
Check out @yucenlily's ICML poster tomorrow 11 am - 1:30 pm, East Exhibition Hall A-B # E-603, on "Out-of-Distribution Detection Methods Answer the Wrong Questions"!
In our new ICML paper, we show that popular families of OOD detection procedures, such as feature and logit based methods, are fundamentally misspecified, answering a different question than “is this point from a different distribution?” arxiv.org/abs/2507.01831 [1/7]
STAMP offers strong protection, can successfully detect membership of content that appears only once in the training data & constitutes < 0.001% of the total tokens. Work led by Saksham Rastogi, in collab w/ @pratyushmaini. Paper: arxiv.org/abs/2504.13416
In our new ICML paper, we show that popular families of OOD detection procedures, such as feature and logit based methods, are fundamentally misspecified, answering a different question than “is this point from a different distribution?” arxiv.org/abs/2507.01831 [1/7]
Are you a graduate student in #Ukraine interested in machine learning and neuroscience? My research lab at #UofT is now accepting applications for remote thesis supervision. (1/3) #neuroAI #compneuro @VectorInst @UofT @UofTCompSci @UHN
nice to see (QA)^2 included here, false presupposition Qs still going strong! every now and then i sit down and see how long it takes to come up with a Q that leads to fun invalid accommodations (my fav recent example in 🧵for the nerds), and the answer is always not very long...
Excited to release AbstentionBench -- our paper and benchmark on evaluating LLMs’ *abstention*: the skill of knowing when NOT to answer! Key finding: reasoning LLMs struggle with unanswerable questions and hallucinate! Details and links to paper & open source code below! 🧵1/9
I had a great time last week at #CVPR2025's DemoDiv workshop (slides below)! I shared an overview of geographic biases in modern datasets and the vast consequences across ML tasks. I argued that existing tools won't be sufficient to solve this problem, and gave some initial ideas…
Very excited this exists. A hill to climb on one of the traits I listed as super needed for next-gen models :)
Excited to release AbstentionBench -- our paper and benchmark on evaluating LLMs’ *abstention*: the skill of knowing when NOT to answer! Key finding: reasoning LLMs struggle with unanswerable questions and hallucinate! Details and links to paper & open source code below! 🧵1/9
Thanks for sharing our work as well! We totally agree! We also tried to mitigate this issue by mixing a small amount of data during training. But definitely more can be explored in this direction!
Our results also align with concurrent work from UCLA @linxins2 @taiwei_shi @jieyuzhao11 which also observed reasoning LLMs hallucinate on unanswerable math problems! x.com/linxins2/statu… More evidence to argue that hallucination and failure to abstain is a big challenge in…
Had a fantastic time at #CVPR2025 and my spotlight talk on culturally representative T2I models at the DemoDiv workshop was definitely the highlight Thanks @polkirichenko and all the organizers, attendees and panelists for a super engaging & thought provoking workshop!
🚨 I'll be giving a short contributed talk about CuRe at CVPR 's DemoDiv workshop today (06/11) at 10 AM! Please drop by, I'd love to chat 😁 x.com/polkirichenko/…
So impressed by Professor @orussakovsky starting her keynote talk by highlighting her research team. We need more of this in our community @WiCVworkshop @CVPR @Princeton @PrincetonCS #wicv #CVPR2025