Huihan Li
@huihan_li
Amazon Fellow, PhD @nlp_usc, Research Intern @meta GenAI | MSE '22 @Princeton @SiebelScholars '22 | @Wellesley '20 | Prev. @ai2_mosaic @Apple AI/ML
Accepted @iclr_conf馃ぉWe build a pretraining corpora attribution framework that determines whether an entity is associated with a culture through memorization or other driving factors, and analyze whether such associations are related to pretraining data distribution. #ICLR2025
1+1=3 2+2=5 3+3=? Many language models (e.g., Llama 3 8B, Mistral v0.1 7B) will answer 7. But why? We dig into the model internals, uncover a function induction mechanism, and find that it鈥檚 broadly reused when models encounter surprises during in-context learning. 馃У
I will be presenting our @iclr_conf paper on attributing culture conditioned generations to memorization of pretraining data, on Fri April. 25, Hall 3 + Hall 2B #255! DM me if you want to chat about memorization, culture, or anything else! #ICLR2025 #iclr #ICLR25
Accepted @iclr_conf馃ぉWe build a pretraining corpora attribution framework that determines whether an entity is associated with a culture through memorization or other driving factors, and analyze whether such associations are related to pretraining data distribution. #ICLR2025
Arrived at @COLM_conf ! Excited to present about some new insights we found on LLM culture bias. Drop by our poster Wed 11am, or DM me if you want to chat馃コ
馃ぉTo appear in COLM 2024 arxiv.org/abs/2404.10199 How does LLM鈥檚 perception of global cultures show from its generations? A: LLMs have abundant cultural knowledge, but inadequate for some cultures; can highlight cultural distinctions, but implicitly exoticize many cultures 1/9