Caia Costello

@CaiaCostello

MS CS Stanford https://github.com/caiacostello

Joined January 2019

13Following

95Followers

Pinned

Caia Costello@CaiaCostello · May 13

1/5 Can small models learn to reason without RL or large datasets? Success of LLM post-training with synthetic data hinges on: 1. Generating Model Size 2. Synthetic Data Volume 3. Pruning Strategy 4. Number of Fine-Tuning Rounds We found a simple recipe: Think, Prune, Train (TPT)

CaiaCostello's tweet image. 1/5 Can small models learn to reason without RL or large datasets? Success of LLM post-training with synthetic data hinges on:
1. Generating Model Size
2. Synthetic Data Volume
3. Pruning Strategy
4. Number of Fine-Tuning Rounds
We found a simple recipe: Think, Prune, Train (TPT)

132

102

32.0K

Caia Costello@CaiaCostello · Jun 27

So excited to speak tomorrow about Think Prune Train at LAD'25 session on Reasoning and Self Improvement! iclad.ai

2.0K