MuzzammilShah commited on
Commit
2a0226b
·
verified ·
1 Parent(s): a74ead5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -56
README.md CHANGED
@@ -1,57 +1,27 @@
1
- ## SET 1 - MAKEMORE (PART 4) 🔗
2
-
3
- [![Documentation](https://img.shields.io/badge/Documentation-Available-blue)](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part4/)
4
- ![Number of Commits](https://img.shields.io/github/commit-activity/m/MuzzammilShah/NeuralNetworks-LanguageModels-4?label=Commits)
5
- [![Last Commit](https://img.shields.io/github/last-commit/MuzzammilShah/NeuralNetworks-LanguageModels-4.svg?style=flat)](https://github.com/MuzzammilShah/NeuralNetworks-LanguageModels-4/commits/main)
6
- ![Project Status](https://img.shields.io/badge/Status-Done-success)
7
-
8
-  
9
-
10
- ### **Overview**
11
- In this repository, we take the 2-layer MLP (with BatchNorm) from the previous video/lecture and **backpropagate through it manually without using PyTorch autograd's loss.backward()**. So we will be manually backpropagating through the cross entropy loss, 2nd linear layer, tanh, batchnorm, 1st linear layer, and the embedding table.
12
-
13
-  
14
-
15
- ### **🗂️Repository Structure**
16
-
17
- ```plaintext
18
- ├── .gitignore
19
- ├── starter-code.ipynb
20
- ├── exercise-1.ipynb
21
- ├── exercise-2.ipynb
22
- ├── exercise-3.ipynb
23
- ├── final-code.ipynb
24
- ├── README.md
25
- └── names.txt
26
- ```
27
-
28
- - **Jupyter Notebooks**: Step-by-step implementation and exploration of the concepts.
29
- - **README.md**: Overview and guide for this repository.
30
- - **names.txt**: Supplementary data file used in training the model.
31
-
32
-  
33
-
34
- ### **📄Instructions**
35
-
36
- To get the best understanding:
37
-
38
- - The format and structure of this particular section of the project will be different from what I've implemented so far, as Andrej Karpathy himself had quoted- "I recommend you work through the exercise yourself but work with it in tandem and whenever you are stuck unpause the video and see me give away the answer. This video is not super intended to be simply watched."
39
-
40
- - So keeping this in mind, we will be focusing more on the notebook itself and only making notes whenever absolutely necessary.
41
-
42
- - You will find my notes/key points as comments in the code cells (Apart from the time stamps with necessary headers which will be in their normal format ofcourse)
43
-
44
-  
45
-
46
- ### **⭐Documentation**
47
-
48
- For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/)**.
49
-
50
- > **💡Pro Tip**: This site provides an interactive and visually rich explanation of the notes and code. It is highly recommended you view this project from there.
51
-
52
-  
53
-
54
- ### **✍🏻Acknowledgments**
55
- Notes and implementations inspired by the **Makemore - Part 4** video by [Andrej Karpathy](https://karpathy.ai/).
56
-
57
  For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - MuzzammilShah/people-names
5
+ language:
6
+ - en
7
+ model_name: Manual Backpropagation through BatchNorm
8
+ library_name: pytorch
9
+ tags:
10
+ - makemore
11
+ - backpropagation
12
+ - batchnorm
13
+ - neural-networks
14
+ - andrej-karpathy
15
+ ---
16
+
17
+ # Manual Backpropagation through BatchNorm: Makemore (Part 4)
18
+
19
+ This repository explores manual backpropagation through a 2-layer MLP (with BatchNorm) without using PyTorch autograd's loss.backward(). It involves manually backpropagating through the cross entropy loss, 2nd linear layer, tanh, batchnorm, 1st linear layer, and the embedding table.
20
+
21
+ ## Documentation
22
+ For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part4/)**.
23
+
24
+ ## Acknowledgments
25
+ Notes and implementations inspired by the **Makemore - Part 4** video by [Andrej Karpathy](https://karpathy.ai/).
26
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).