Prateek Mittal
@prateekmittal_
Professor at Princeton. Focused on privacy, cybersecurity, AI and machine learning, public interest technologies.
I am honored to receive the 2023 ACM Grace Murray Hopper Award for enhancing Internet privacy and security! @TheOfficialACM @Princeton engineering.princeton.edu/news/2024/06/1…
This is a sad statistic #NeurIPS2025 I hope reviewers recalibrate, perhaps after reading other reviews and author responses!
Back in grad school, when I realized how the “marketplace of ideas” actually works, it felt like I’d found the cheat codes to a research career. Today, this is the most important stuff I teach students, more than anything related to the substance of our research. A quick…
Check out our new blogpost and policy brief on our recently updated lab website! ❓Are we actually capturing the bubble of risk for cybersecurity evals? Not really! Adversaries can modify agents by a small amount and get massive gains.
Professor Andrew Houck '00 has been appointed dean of @eprinceton, effective August 1: bit.ly/44CPh4S Houck co-directs the Princeton Quantum Initiative and leads the federally funded national quantum research center, @C2QAdvantage.
AI risk is real. Paper from Princeton AI Lab shows it’s shockingly easy to jailbreak genome-focused LLMs—opening doors to dangerous misuse. We must build strong safeguards now. Check out our call on Nature Biotech that maps out the AI guardrail technologies needed to mitigate…
🧬 GeneBreaker: Unveiling the Biosecurity Risks of DNA Language Models DNA models can be "jailbroken" to generate sequences resembling human pathogens like SARS-CoV-2 and HIV-1. With up to a 60% success rate! 🔗 Read more: arxiv.org/abs/2505.23839
Princeton engineers have identified a universal weakness in AI chatbots that allows users to bypass safety guardrails and elicit directions for malicious uses, from creating nerve gas to hacking government databases. bit.ly/3SzRto7
Last week, I shared two #ICLR2025 papers that were recognized by their Award committee. Reflecting on the outcome, I thought it might be interesting to share that both papers were previously rejected by #NeurIPS2024. I found the dramatic difference in reviewer perception of…
Delighted to share that two papers from our group @EPrinceton got recognized by the @iclr_conf award committee. Our paper, "Safety Alignment Should be Made More Than Just a Few Tokens Deep", received the ICLR 2025 Outstanding Paper Award. This paper showcases that many AI…
Delighted to share that two papers from our group @EPrinceton got recognized by the @iclr_conf award committee. Our paper, "Safety Alignment Should be Made More Than Just a Few Tokens Deep", received the ICLR 2025 Outstanding Paper Award. This paper showcases that many AI…
Outstanding Papers Safety Alignment Should be Made More Than Just a Few Tokens Deep. Xiangyu Qi, et al. Learning Dynamics of LLM Finetuning. Yi Ren and Danica J. Sutherland. AlphaEdit: Null-Space Constrained Model Editing for Language Models. Junfeng Fang, et al.
Dennis Ritchie, the man who invented C, co-created Unix, and is largely regarded as effectively influencing every software system we use on a daily basis. His death was largely ignored, overshadowed by Steve Jobs' death, one week before.
Thinking intervention is a new paradigm for controlling LLMs. The idea is deceptively simple: we can do thought engineering in model’s reasoning space. This has nicely applications to safety alignment, instruction hierarchy, instruction following, and more 👇
🛠️ Still doing prompt engineering for R1 reasoning models? 🧩 Why not do some "engineering" in reasoning as well? Introducing our new paper, Effectively Controlling Reasoning Models through Thinking Intervention. 🧵[1/n]
Until now, pricing structure on rideshare apps has been opaque for both drivers and riders. 🚗 To help fix this, the @the_wao_ and researchers from @Princeton created the FairFare app to crowdsource payment info from drivers. Now, a new law in Colorado mandates transparency.
I didn't know this! #Competition #Monopolies
The fear that private wealth could lead to a system of private *rule* has been in America since its founding, when Benjamin Franklin tried (unsuccessfully) to put a ban on monopolies into the US Constitution. 32/
Ok, look people: Signal as a *protocol* is excellent. As a service it’s excellent. But as an application running on your phone, it’s… an application running on your consumer-grade phone. The targeted attacks people use on those devices are well known.
Reasons why people who work w/computers seem to have a lot of free time: bit.ly/2IlCrNu Image v/Evil Jaymz
How can we enhance trustworthiness of AI summaries? This could be a good use case for applying AI robustness techniques that were developed in the context of data poisoning. The idea is that LLM outputs should not depend too much on any single data source, say one user’s…
Example #4317 of why you should never trust AI summaries: Dante, famously, despite being Florentine, is *not* buried in Florence, because they EXILED him. He's buried in Ravenna; this comment is picked up from a TripAdvisor post by a Sri Lankan tourist. (A monument, not tomb.)
This is a great idea!
This teacher uses AI-generated images of her students to motivate them to achieve their dreams
Fantastic work by @PandaAshwinee, @XinyuTang7 and @GoogleDeepMind collaborators on the first practical approach for privacy auditing of LLMs. 🧵👇
we show for the first time ever how to privacy audit LLM training. we give new SOTA methods that show how much models can memorize. by using our methods, you can know beforehand whether your model is going to memorize its training data, and how much, and when, and why! (1/n 🧵)
Congratulations to @prateekmittal_, professor in @Princeton Electrical & Computer Engineering, for being named a 2024 @TheOfficialACM Distinguished Member🎉These are awarded for technical & professional achievements & contributions in computer science & information technology