← Anson Chu

Learning RL with Claude

I have a scrolling problem. Same as everyone.

At some point I noticed that when I fell into a research hole, I would end up on my phone anyway, except reading garbage about it. So I started doing something different. Whenever I hit a concept I didn't understand — GRPO, RLVR, whatever the vortex served up that week — I would ask Claude to write me a lesson about it. Sized for a phone. Then I'd read that on the couch instead of the feed.

It sounds small but it changed how I learn. The lessons are generated from my questions, so the curriculum is exactly the shape of my curiosity, not the shape of someone's syllabus. And unlike a textbook, when something doesn't click I just ask the next question and get the next lesson. Zero to Hero taught me to learn by building. This is learning by asking.

Most of these lived and died in my phone, which is fine — the reading was the point, not the artifact. Here are two, regenerated and kept:

lesson · claude-generated PPO vs GRPO, side by side what PPO keeps in memory, what GRPO throws away, and why group-relative advantages fit verifiable rewards lesson · claude-generated RLVR, the unlock swap the learned reward model for a checker, and suddenly you can push as hard as you want

These pages are Claude-generated, obviously. That's the point. I direct, it writes, I read. Then I ask better questions.