TREC RAG @ 2025 (@TREC_RAG)

Pinned

T

TREC RAG @ 2025@TREC_RAG · Nov 19

#TREC2024 Retrieval Augmented Generation (RAG) track is happening live today at TREC! Exciting findings (Retrieval, Nuggets, Support & Fluency) from the organizers and ongoing participants' submissions! 😎 Thanks to all participants, we had lots of runs to evaluate! 😃🙏🏼

TREC_RAG's tweet image. #TREC2024 Retrieval Augmented Generation (RAG) track is happening live today at TREC!

Exciting findings (Retrieval, Nuggets, Support &amp; Fluency) from the organizers and ongoing participants' submissions! 😎

Thanks to all participants, we had lots of runs to evaluate! 😃🙏🏼

0

1

13

1

3.0K

T

TREC RAG @ 2025@TREC_RAG · 22 h

TREC RAG 2025 official retrieval baselines are available now! 💥💥💥 Time to start generating those answers and submit them to eval base before August 17th! 🗓️ Let the games begin, well you have less than a month remaining to submit! 🍻

TTREC RAG @ 2025@TREC_RAG · 23 h

🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here: trec-rag.github.io/annoucements/2…

0

3

16

4

1.0K

T

TREC RAG @ 2025@TREC_RAG · 22 h

🚨 TREC RAG 2025 update: Official baselines are now available! 📅 Mark your calendars: submissions due August 17

TTREC RAG @ 2025@TREC_RAG · 23 h

🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here: trec-rag.github.io/annoucements/2…

0

2

5

0

405

T

TREC RAG @ 2025@TREC_RAG · 23 h

We are out with the official baselines for @TREC_RAG this year: github.com/castorini/ragn… @Ushivani3 and I had fun putting together a strong Retrieve (Pyserini) -> Rerank (RankLLM) -> Augmented Gen (Ragnarök) baseline and we hope to see you all beat it!

TTREC RAG @ 2025@TREC_RAG · 23 h

🚀 The official baselines and validation scripts for TREC RAG 2025 are now available! These include both retrieval results (for the AG task) and the corresponding end-to-end augmented generation outputs. Access the baselines and necessary scripts here: trec-rag.github.io/annoucements/2…

0

1

5

0

248

T

TREC RAG @ 2025@TREC_RAG · Jul 15

TREC RAG TOPICS ARE OFFICIALLY OUT NOW!!! 🔥🔥🔥 LET THE GAMES BEGIN! 🥶

SShivani Upadhyay@Ushivani3 · Jul 11

📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): trec-rag.github.io/annoucements/2… Let the games begin! #TREC2025 #RAG

0

5

13

0

2.0K

T

TREC RAG @ 2025@TREC_RAG · Jul 11

We have released the @TREC_RAG 2025 topics and will be out with strong baselines soon. But for those who are eager, go right ahead!

SShivani Upadhyay@Ushivani3 · Jul 11

📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): trec-rag.github.io/annoucements/2… Let the games begin! #TREC2025 #RAG

0

2

0

647

TREC RAG @ 2025 Retweeted

S

Shivani Upadhyay@Ushivani3 · Jul 11

📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): trec-rag.github.io/annoucements/2… Let the games begin! #TREC2025 #RAG

0

3

18

9

3.0K

T

TREC RAG @ 2025@TREC_RAG · Jun 28

We had a successful participation of over 45+ teams & 150+ runs last year in TREC RAG 2024! 🔥🔥 We are back with @TREC_RAG 2025 this year! Make sure you participate in one of the RAG tracks this year to help accelerate RAG evaluation! ⚡⚡ Signup here: docs.google.com/forms/d/e/1FAI…

TTREC RAG @ 2025@TREC_RAG · Jun 27

📝 The tentative schedule for the TREC 2025 #RAG is released now! 😎 Details in 🧵: 1️⃣ Test topics & baselines: Mid July 2025 2️⃣ Submission deadline: Mid August 2025 3️⃣ Results & judgments returned: October 2025 4️⃣ TREC 2025: November 2025

1

3

8

4

878

T

TREC RAG @ 2025@TREC_RAG · Jun 27

Hello world! I'm back for TREC RAG 2025. Did you miss us?

0

1

6

1

180

T

TREC RAG @ 2025@TREC_RAG · Jun 25

Now we know RAG is alive (again). I'll be talking about evals this, evals that, evals!

BBen Clavié@bclavie · Jun 25

Phew, thank God my boring talk is over, and everyone now knows RAG is alive. Now we can all focus on the good stuff. Next week, @beirmug will finally answer one of the most asked questions: how do I, in 2025, know which models I should trust to be good on unseen data?

4

24

6

7.0K

TREC RAG @ 2025 Retweeted

N

Nandan Thakur@beirmug · Nov 24

2024 has been a fun year! Initial collab with @vectara on multilingual RAG benchmarking to coordinating a first-ever RAG track (@TREC_RAG) in TREC 2024 and lastly getting to intern at @DbrxMosaicAI! I predict 2025 will be the year of diversity & realistic evals in IR & RAG!

0

4

11

1

1.0K

TREC RAG @ 2025 Retweeted

J

Jimmy Lin@lintool · Nov 18

As @ian_soboroff notes kicking off TREC 2024, the 33rd iteration of the series, this is the 30th anniversary of BM25, introduced at TREC-3: trec.nist.gov/pubs/trec3/t3_…

1

9

40

9

8.0K

T

TREC RAG @ 2025@TREC_RAG · Nov 17

🎙️ We bring to you UMBRELA ⛱️ relevance judgments for the TREC 2024 RAG topics! It should be a good proxy for evaluating your retrieval pipelines with the TREC 2024 RAG test topics. New topics to assess your systems! In time, other judgments shall be made public too. 🧙‍♂️

SShivani Upadhyay@Ushivani3 · Nov 17

TREC RAG 2024 fully automated qrel (using UMBRELA) is officially now released! Check out the website for the qrel (with 301 topics): trec-rag.github.io/annoucements/u… #RAG #IR #TREC2024

0

2

12

1

2.0K

T

TREC RAG @ 2025@TREC_RAG · Nov 17

⚗️ UMBRELA qrels for the TREC 2024 RAG Track have been posted! arxiv.org/abs/2411.08275

SShivani Upadhyay@Ushivani3 · Nov 17

TREC RAG 2024 fully automated qrel (using UMBRELA) is officially now released! Check out the website for the qrel (with 301 topics): trec-rag.github.io/annoucements/u… #RAG #IR #TREC2024

0

1

12

3

2.0K