MuzzammilShah commited on
Commit
d6312dd
·
verified ·
1 Parent(s): 9470d95

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -58
README.md CHANGED
@@ -1,59 +1,34 @@
1
- ## SET 1 - MAKEMORE (PART 3) 🔗
2
-
3
- [![Documentation](https://img.shields.io/badge/Documentation-Available-blue)](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part3/)
4
- ![Number of Commits](https://img.shields.io/github/commit-activity/m/MuzzammilShah/NeuralNetworks-LanguageModels-3?label=Commits)
5
- [![Last Commit](https://img.shields.io/github/last-commit/MuzzammilShah/NeuralNetworks-LanguageModels-3.svg?style=flat)](https://github.com/MuzzammilShah/NeuralNetworks-LanguageModels-3/commits/main)
6
- ![Project Status](https://img.shields.io/badge/Status-Done-success)
7
-
8
-  
9
-
10
- ### **Overview**
11
- In this repository, I implemented **Batch Normalization** within a neural network framework to enhance training stability and performance, following Andrej Karpathy's approach in the **Makemore - Part 3** video.
12
-
13
- This implementation focuses on **normalizing activations and gradients, addressing initialization issues, and utilizing Kaiming initialization to prevent saturation of activation functions**. Additionally, **visualization graphs** were created at the end to analyze the effects of these techniques on the training process and model performance.
14
-
15
-  
16
-
17
- ### **🗂️Repository Structure**
18
-
19
- ```plaintext
20
- ├── .gitignore
21
- ├── A-Main-Notebook.ipynb
22
- ├── StarterCode.ipynb
23
- ├── VisualizationTools.ipynb
24
- ├── README.md
25
- ├── notes/
26
- │ ├── A-main-makemore-part3.md
27
- │ └── README.md
28
- └── names.txt
29
- ```
30
-
31
- - **Notes Directory**: Contains detailed notes corresponding to each notebook section.
32
- - **Jupyter Notebooks**: Step-by-step implementation and exploration of the concepts.
33
- - **README.md**: Overview and guide for this repository.
34
- - **names.txt**: Supplementary data file used in training the model.
35
-
36
-  
37
-
38
- ### **📄Instructions**
39
-
40
- To get the best understanding:
41
-
42
- 1. Start by reading the notes in the `notes/` directory. Each section corresponds to a notebook for step-by-step explanations.
43
- 2. Open the corresponding Jupyter Notebook (e.g., `A-Main-Notebook.ipynb` for `A-main-makemore-part3.md`).
44
- 3. Follow the code and comments for a deeper dive into the implementation details.
45
-
46
-  
47
-
48
- ### **⭐Documentation**
49
-
50
- For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/)**.
51
-
52
- > **💡Pro Tip**: This site provides an interactive and visually rich explanation of the notes and code. It is highly recommended you view this project from there.
53
-
54
-  
55
-
56
- ### **✍🏻Acknowledgments**
57
- Notes and implementations inspired by the **Makemore - Part 3** video by [Andrej Karpathy](https://karpathy.ai/).
58
-
59
  For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - MuzzammilShah/people-names
5
+ language:
6
+ - en
7
+ model_name: Batch Normalization for Neural Networks
8
+ library_name: pytorch
9
+ tags:
10
+ - makemore
11
+ - batch-normalization
12
+ - neural-networks
13
+ - andrej-karpathy
14
+ ---
15
+
16
+ # Batch Normalization for Neural Networks: Makemore (Part 3)
17
+
18
+ In this repository, I implemented **Batch Normalization** within a neural network framework to enhance training stability and performance, following Andrej Karpathy's approach in the **Makemore - Part 3** video.
19
+
20
+ ## Overview
21
+ This implementation focuses on:
22
+ - **Normalizing activations and gradients**.
23
+ - Addressing initialization issues.
24
+ - Utilizing Kaiming initialization to prevent saturation of activation functions.
25
+
26
+ Additionally, **visualization graphs** were created at the end to analyze the effects of these techniques on the training process and model performance.
27
+
28
+ ## Documentation
29
+ For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part3/)**.
30
+
31
+ ## Acknowledgments
32
+ Notes and implementations inspired by the **Makemore - Part 3** video by [Andrej Karpathy](https://karpathy.ai/).
33
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  For more of my projects, visit my [Portfolio Site](https://muhammedshah.com).