Tianyi Zhou
@tianyi_zhou12
CS PhD Student @USC
Billion-parameter LLMs still struggle with simple arithmetic? 📞 FoNE (Fourier Number Embedding) tackles this problem. By mapping numbers directly into Fourier space, it bypasses tokenization and significantly improves numerical accuracy with better efficiency and accuracy.
If an LLM’s hallucinated claim contradicts its own knowledge, it should be able to retract the claim. Yet, it often reaffirms the claim instead. Why? @yyqcode dives deep to show that faulty model internal beliefs (representations of “truthfulness”) drive retraction failures!
🧐When do LLMs admit their mistakes when they should know better? In our new paper, we define this behavior as retraction: the model indicates that its generated answer was wrong. LLMs can retract—but they rarely do.🤯 arxiv.org/abs/2505.16170 👇🧵
Textual steering vectors can improve visual understanding in multimodal LLMs! You can extract steering vectors via any interpretability toolkit you like -- SAEs, MeanShift, Probes -- and apply them to image or text tokens (or both) of Multimodal LLMs. And They Steer!
Great to see others discovering similar findings as we did in our Neurips2024 paper (arxiv.org/abs/2406.03445). We call these Fourier features instead of helix. How are these features useful for representing numbers? Stay tuned for our new number embedding paper coming soon!
(1/N) LLMs represent numbers on a helix? And use trigonometry to do addition? Answers below 🧵