CS234

Turning research papers into Manim animations

Fine-tuning a small model to generate ManimThe library 3blue1brown uses to generate its videos. See manim.community. animations straight from a research paper.

View poster GitHub

The idea

Input a paper and get back a set of clear animations explaining it. Larger models were able to generate some good looking animations, but every video could end up costing $2-3 given how expensive tokens were. So the plan was to distill that ability into a small model I could run cheaply.

Where the instructions for the animations come from

To fine-tune a model to make animations, I first needed prompt-and-result pairs: an instruction describing an animation, and the ManimThe library 3blue1brown uses to generate its videos. See manim.community. code that actually produces it. I built a whole system to generate those pairs.

It starts from a database of papers I crawled from AI/ML conferences. For each paper, I asked Gemini 3 Flash (the teacher) to break it into about four animation “scripts”: a voice-over plus instructions for what to draw. Those scripts are the prompts.

Pipeline: a crawled paper database feeds one paper at a time into Gemini 3 Flash, which produces four animation scripts per paper. — Step one: turn each paper into a handful of self-contained animation scripts (the prompts).

To get the results, Gemini then turned each script into actual ManimThe library 3blue1brown uses to generate its videos. See manim.community. code and I compiled it. Roughly 75% of Gemini's code compiled into a video, and those (script → compiling code) pairs became my SFT dataset for Qwen2.5-Coder-3B.

Pipeline: an animation script goes into Gemini 3 Flash, which emits Manim code that either renders a video or throws a compilation error, at roughly a 75% compile rate. — Step two: Gemini writes the Manim code. The scripts and the code that actually compiled became the prompt-and-result pairs.

The small model was terrible on its own

I tried feeding those same scripts to Qwen2.5-Coder-3B directly, and it produced almost nothing but ManimThe library 3blue1brown uses to generate its videos. See manim.community. compilation errors. So I was forced to first do SFT on the Gemini pairs, and then a round of DPO on top.

Pipeline: an animation script goes into Qwen-2.5-Coder-3B, which emits Manim code that practically always ends in a compilation error rather than a video. — Before any fine-tuning, the small model produced practically only compilation errors.

Building preference pairs for DPO

For the DPO stage I sampled the SFT model eight times per prompt. Some attempts compiled, most didn't, and I paired each failed attempt with a compiling one to build preference pairs. Prompting, SFT, and DPO together moved pass@1 from 2.5% to 25%.

Pipeline: the SFT model is sampled many times for one script; some outputs compile into a video and some throw errors, and these are paired up as DPO preference data. — Step three: sample the SFT model many times per prompt, then pair compiling and non-compiling outputs into DPO preference data.

Reward hacking

Pass@1 went up, but the animations themselves weren't good. A compilation-based reward is useful, but over-optimizing it just makes Manim code that compiles into a video, not something that looks good. You can see it below: next to a Gemini training example, my fine-tuned model's output is far simpler and emptier.

What the course actually taught me

CS234 taught me the math behind reinforcement learning: policy optimization, DPO/GRPO, reward design. In this project I therefore struggled with the "crafty" part of doing SFT/DPO well: building a good and diverse dataset and a high quality set of evals.

Teacher vs. student

A training example generated by Gemini 3 Flash (the teacher). This is the kind of animation I was trying to learn from.

What my fine-tuned model produces after finetuning. It is relatively simple.

What I learned

My first time doing SFT / DPO on an LLM. I learned the shape of the data necessary for doing this, and got first experience with using cloud compute (in this case Google Collab).
The real bottleneck in finetuning small models is high quality data and the evals, not the adapter hyperparameters. I spent a bunch of time changing hyperparams with practically no results.

What I'd do differently

Build a real benchmark first: a fixed set of ~50 animation scripts with reference videos from a large model, and score every model against that same set. My only signal was compile rate, measured on inconsistent sets, so my evaluation was a mess.
Choose models from that benchmark: I'd compare small candidates (Liquid, Qwen, Gemma) to pick the base model, and also measure the teacher (Gemini 3 Flash) as the target to beat.
Start from Manim datasets that already exist on Hugging Face, like bespoke-manim or manim_python. My hand-generated Gemini data was lower quality, partly because Gemini itself only compiled ~75% of the time and the subset of papers used to generate the animation scripts was limited.
Try GRPO, not just DPO: GRPO does online learning over the same kind of samples I generated for this project and might fit this task better.
Don't over-sample the same prompt: Building 7+ pairs from a single prompt overweighted a few papers and likely pushed the model to overfit them.

← Back to all projects