Neural Networks: Zero to Hero
I have been on parental leave over the past month and I happen to have a bit of time on my hands.
I decided to spend this time going through Andrej Karpathy's Zero-to-Hero course to learn more about neural nets and especially GPTs, which is something I have not worked with before.
Andrej is an absolute legend and this is by far the best hands on tutorial and explanation of language models I have found. I am really glad I spent the time going through it!
All of my code is at github.com/kumikoda/zerohero.
Lessons
- 2023-10Makemore Part 1 — Bigram Model
- 2023-10Makemore Part 2 — Multilayer Perceptron
- 2023-10Makemore Part 3 — Activations & Gradients, BatchNorm
- 2023-10Makemore Part 4 — Becoming a Backprop Ninja
- 2023-11Makemore Part 5 — Building a WaveNet
- 2023-11Let's Build GPT: from scratch, in code, spelled out
Backprop, by hand
Side quests from Part 4 — deriving the backward passes on paper until they agreed with PyTorch.