I have been on parental leave over the past month and I happen to have a bit of time on my hands.
I decided to spend this time going through Andrei Karpathy’s Zero-to-Hero course to learn more about neural nets and especially GPTs, which is something I have not worked with before.
Andrei is a absolute legend and this is by far the best hands on tutorial and explanation of language models I have found. I am really glad I spent the time going through it!
Makemore Part 1 - Bigram Model
Makemore Part 2 - Multilayer Perceptron
Makemore Part 3 - Activations & Gradients, BatchNorm
Makemore Part 4 - Becoming a Backprop Ninja
Makemore Part 5 - Building a WaveNet
Let’s Build GPT: from scratch, in code, spelled out
You can also find all of my code here
