# Introduction

<img src="https://huggingface.co/datasets/huggingface-ml-4-games-course/course-images/resolve/main/en/unit1/unity/thumbnail.png" alt="Thumbnail" />

Welcome to the first Unit of the course! 

The integration of cutting-edge AI models in video games is opening a whole range of new exciting gameplays. One of them is the ability for Non-Player Characters (NPCs) **to understand and respond to the player’s voice or text inputs**. And this is what we’re going to do today.

Indeed, in this Unit, **you're going to integrate your first AI model in your Unity game and make it run locally**.

You're going to learn:

- What is Sentence Similarity?
- The difference between **running an AI model locally or remotely** (using an API).
- What the [Hugging Face Hub 🤗](https://huggingface.co/models?library=unity-sentis) is.
- How to run an AI model locally with [Unity Sentis](https://docs.unity3d.com/Packages/com.unity.sentis@latest) and [Sharp Transformers](https://github.com/huggingface/sharp-transformers).

And you're going to make this demo, where a **smart robot that can understand your orders and perform them**.

<iframe width="560" height="315" src="https://www.youtube.com/embed/L8LlOlYc3wI?si=ZILzXsuLwl__fYbx" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

You can download the Windows demo 👉 [here](https://singularite.itch.io/jammo-the-robot-with-unity-sentis)

To make this project, we’re going to use:

- Unity Game Engine (2022.3 and +).

- The Jammo Robot asset made by [Mix and Jam](https://www.youtube.com/channel/UCLyVUwlB_Hahir_VsKkGPIA).

- [Unity Sentis library](https://docs.unity3d.com/Packages/com.unity.sentis@latest), the neural network inference library that allow us to **run our AI model directly inside our game**.

- The [Hugging Face Sharp Transformers](https://github.com/huggingface/sharp-transformers)): a Unity plugin of utilities to **run Transformer 🤗 models in Unity games**.

You can download the complete Unity Project by clicking 👉 [here](https://huggingface.co/datasets/huggingface-ml-4-games-course/unity-demos/resolve/main/unit1/Jammo%20the%20Robot%20Sentis%20(v2).zip?download=true)

At the end of the project, you’ll build your own intelligent robot game demo. 


And then, you’ll be able **to iterate with other ideas**:


For instance, after making this game, I created this Dungeon Escape demo ⚔️ with the same codebase, **where your goal is to flee from this jail by stealing the 🔑 and the gold without getting noticed by the guard**.


<iframe width="560" height="315" src="https://www.youtube.com/embed/h8Mq1ZJkFL0?si=4q21QH6PIqNeE70w" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>


But I also worked on a stealth game, where you **guide your character to sneak in a party and steal some stuff**.

<iframe width="560" height="315" src="https://www.youtube.com/embed/mU6ih2BBAus?si=Vh1XQoFVbEvuCtYb" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

You'll be also able to improve the demo, [by adding a Speech To Text model to give order to the robot with your voice!](https://thomassimonini.substack.com/p/building-ai-driven-voice-recognition)

<iframe width="560" height="315" src="https://www.youtube.com/embed/TzHTLW2zOlY?si=LekdMO6vp2_qM-aR" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>


Sounds fun? Let’s get started!


<img src="https://huggingface.co/datasets/huggingface-ml-4-games-course/course-images/resolve/main/en/unit1/unity/jammo-dance.gif" alt="Jammo dance" />

<EditOnGithub source="https://github.com/huggingface/making-games-with-ai-course/blob/main/units/en/unit1/introduction.mdx" />