Monica Agrawal @ ICML 2025
@MonicaNAgrawal
Asst prof at @DukeU. Co-founder at @LayerHealth. ML and NLP for healthcare. PhD @mit_csail, BS/MS @stanford. She/her.
General LLMs perform well on clinical NLP tasks, even though they never see EHR data. How? Our CHIL 2025 paper, led by @Flora__Jia, investigates by probing the clinical content in pre-training datasets behind popular LLMs. 🔗 arxiv.org/abs/2505.15024
I am recruiting PhD students at Duke! Please apply to @dukecompsci or Duke CBB if you are interested in developing new methods and paradigms for NLP/LLMs in healthcare. For details, see here: monicaagrawal.com/home/research-…. Feel free to retweet!
Interested in using LLMs to automate writing clinical notes? Remember stories like this, which happen all the time. Writing = thinking. LLMs can and do help us think, but not if they are implemented thoughtlessly.
Sent a patient home last week without evaluating a concern. Decided it didn't warrant evaluation. An hour later, while typing note, I had second thoughts. Called the pt and scheduled a test. Got the results today. Very glad I did the test. A close call.
When you walk into the ER, you could get a doc: 1. Fresh from a week of not working 2. Tired from working too many shifts I’ve been both and believe me, they're different! And we can tell the difference, just from notes they write. Paper @NatureComms nature.com/articles/s4146…
🩺 Open-source large language models now perform well across various clinical natural language processing tasks, even though they never see electronic health records. Where do they pick up that clinical knowledge? 🚀 We are excited to share our CHIL 2025 paper “Diagnosing our…
The Machine Learning for Healthcare (MLHC) 2025 Call for Papers is out! mlforhc.org/paper-submissi… Pre-submission intent deadline: April 4th, 2025 Full submission deadline (Research and Clinical Abstracts Tracks): April 11th, 2025 Submit via OpenReview: openreview.net/group?id=mlfor…
This is joint work with @dmshanmugam, @rajivmovva, Jon Kleinberg, @MonicaNAgrawal, @mdredze, @KadijaFerryman, @judywawira, @jurafsky, @PangWeiKoh, @karen_ec_levy, @m_sendhil, @oziadias, @harini824, and @keyonV! Freely accessible link here: ai.nejm.org/stoken/default…
Our article on using LLMs to improve health equity is out in @NEJM_AI! 85% of equity-related LLM papers focus on *harms*. But equally vital are the equity-related *opportunities* LLMs create: detecting bias, extracting structured data, and improving access to health info.
We also consider what new challenges generative AI in medicine might raise: what does patient consent look like with LLMs? Who is liable for algorithmic medical errors? These are open questions; there’s some work to answer this but lots more to be done!
We have a new review on generative AI in medicine, to appear in the Annual Review of Biomedical Data Science! We cover *over 250 papers* in the recent literature to provide an updated overview of use cases and challenges for generative AI in medicine.
Medicine thrives on knowledge, yet clinical vocabularies are fragmented. AI struggles to unify this knowledge, creating a gap in precision medicine. Excited to share unified clinical vocabulary embeddings, a project led by stellar @ruthie_johnson Half of healthcare foundation AI…
I'm recruiting PhD students for Fall 2025! CS PhD Deadline: Dec. 15th. I work on safe/reliable ML and causal inference, motivated by healthcare applications. Beyond myself, Johns Hopkins has a rich community of folks doing similar work! Come join us!
🧵LLMs are great at synthesizing info, but unreliable at citing sources. Search engines are the opposite. What lies between them? Our new paper runs human evals on 7 systems across the✨extractive-abstractive spectrum✨for utility, citation quality, time-to-verify, & fluency!
My lab at Duke has multiple Ph.D. openings! Our mission is to augment human decision-making by advancing the reasoning, comprehension, and autonomy of modern AI systems. I am attending #emnlp2024, happy to chat about PhD applications, LLM agents, evaluation etc etc!
The internet has been progressively diluted with AI-generated slop. Are medical records headed for the same fate? 🧵 I just published a perspective in @NEJM with @AdamRodmanMD and Arjun Manrai on why rushing AI into medical documentation could be a mistake.
🧵When should LLMs trust external contexts in RAG? New paper from @YukunHuang9 and @sanxing_chen enhances LLMs’ *situated faithfulness* to external contexts -- even when they are wrong!👇
We recently updated our paper on augmenting clinical notes with LLMs to make them more readable. This paper is a great read for thinking how LLMs can improve patient experience (and potential risks!). arxiv.org/abs/2401.09637
🚀Announcing the 6th Annual Conference on Health, Inference, and Learning (CHIL 2025) to be held in Berkeley, CA! 🌁🎉 Led by General Chairs @MattBMcDermott & @irenetrampoline, and Program Chairs @drjessilyn & @RoxanaDaneshjou, this year's event is set to be incredible! 🧵(1/2)