Update README.md
Browse files
README.md
CHANGED
@@ -1,57 +1,27 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
- **Jupyter Notebooks**: Step-by-step implementation and exploration of the concepts.
|
29 |
-
- **README.md**: Overview and guide for this repository.
|
30 |
-
- **names.txt**: Supplementary data file used in training the model.
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
### **📄Instructions**
|
35 |
-
|
36 |
-
To get the best understanding:
|
37 |
-
|
38 |
-
- The format and structure of this particular section of the project will be different from what I've implemented so far, as Andrej Karpathy himself had quoted- "I recommend you work through the exercise yourself but work with it in tandem and whenever you are stuck unpause the video and see me give away the answer. This video is not super intended to be simply watched."
|
39 |
-
|
40 |
-
- So keeping this in mind, we will be focusing more on the notebook itself and only making notes whenever absolutely necessary.
|
41 |
-
|
42 |
-
- You will find my notes/key points as comments in the code cells (Apart from the time stamps with necessary headers which will be in their normal format ofcourse)
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
### **⭐Documentation**
|
47 |
-
|
48 |
-
For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/)**.
|
49 |
-
|
50 |
-
> **💡Pro Tip**: This site provides an interactive and visually rich explanation of the notes and code. It is highly recommended you view this project from there.
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
### **✍🏻Acknowledgments**
|
55 |
-
Notes and implementations inspired by the **Makemore - Part 4** video by [Andrej Karpathy](https://karpathy.ai/).
|
56 |
-
|
57 |
For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
datasets:
|
4 |
+
- MuzzammilShah/people-names
|
5 |
+
language:
|
6 |
+
- en
|
7 |
+
model_name: Manual Backpropagation through BatchNorm
|
8 |
+
library_name: pytorch
|
9 |
+
tags:
|
10 |
+
- makemore
|
11 |
+
- backpropagation
|
12 |
+
- batchnorm
|
13 |
+
- neural-networks
|
14 |
+
- andrej-karpathy
|
15 |
+
---
|
16 |
+
|
17 |
+
# Manual Backpropagation through BatchNorm: Makemore (Part 4)
|
18 |
+
|
19 |
+
This repository explores manual backpropagation through a 2-layer MLP (with BatchNorm) without using PyTorch autograd's loss.backward(). It involves manually backpropagating through the cross entropy loss, 2nd linear layer, tanh, batchnorm, 1st linear layer, and the embedding table.
|
20 |
+
|
21 |
+
## Documentation
|
22 |
+
For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part4/)**.
|
23 |
+
|
24 |
+
## Acknowledgments
|
25 |
+
Notes and implementations inspired by the **Makemore - Part 4** video by [Andrej Karpathy](https://karpathy.ai/).
|
26 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).
|