lecture
Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy · 14 recommenders · 8 months
Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.
Why Hacker News readers recommend it: It is the go-to recommendation for learning how transformers and LLMs actually work, praised for its clarity and Karpathy's teaching.
LLMs & generative AIMachine learning & deep learning#gpt#transformer#pytorch#nanogpt#attention#from scratch
“the best video I have seen on the topic”
View the original HN discussion
