README.md

SET 1 - MAKEMORE (PART 3) 🔗

 

Overview

In this repository, I implemented Batch Normalization within a neural network framework to enhance training stability and performance, following Andrej Karpathy's approach in the Makemore - Part 3 video.

This implementation focuses on normalizing activations and gradients, addressing initialization issues, and utilizing Kaiming initialization to prevent saturation of activation functions. Additionally, visualization graphs were created at the end to analyze the effects of these techniques on the training process and model performance.

 

🗂️Repository Structure

├── .gitignore
├── A-Main-Notebook.ipynb
├── StarterCode.ipynb
├── VisualizationTools.ipynb
├── README.md
├── notes/
│   ├── A-main-makemore-part3.md
│   └── README.md
└── names.txt

• Notes Directory: Contains detailed notes corresponding to each notebook section.
• Jupyter Notebooks: Step-by-step implementation and exploration of the concepts.
• README.md: Overview and guide for this repository.
• names.txt: Supplementary data file used in training the model.

 

📄Instructions

To get the best understanding:

1. Start by reading the notes in the notes/ directory. Each section corresponds to a notebook for step-by-step explanations.
2. Open the corresponding Jupyter Notebook (e.g., A-Main-Notebook.ipynb for A-main-makemore-part3.md).
3. Follow the code and comments for a deeper dive into the implementation details.

 

⭐Documentation

For a better reading experience and detailed notes, visit my Road to GPT Documentation Site.

💡Pro Tip: This site provides an interactive and visually rich explanation of the notes and code. It is highly recommended you view this project from there.

 

✍🏻Acknowledgments

Notes and implementations inspired by the Makemore - Part 3 video by Andrej Karpathy.

For more of my projects, visit my Portfolio Site.