Update README.md

README.md

@@ -9,194 +9,125 @@ base_model: google/gemma-7b
 
 
 
-## Model Details
+## Model Details 
+Google has released Gemma, a new family of state-of-the-art open Large Language Models (LLMs), available in 7B and 2B parameter versions. While Gemma offers versatility, fine-tuning it with ChatML has posed challenges. This blog post aims to demonstrate fine-tuning Gemma 7B using Hugging Face TRL, derived from the methodology outlined in a previous guide.
 
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
+Steps:
 
+- Setting Up the Development Environment
+- Dataset Creation and Preparation
+- Fine-Tuning Gemma with TRL and SFTTrainer
+- Testing and Evaluating the Fine-Tuned LLM
 
+### Model Description
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
+<!-- Provide a longer summary of what this model is. -->
 
-<!-- Provide the basic links for the model. -->
+- **Developed by:** AndyRasika
+- **Finetuned from model [optional]:** philschmid/gemma-tokenizer-chatml
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
 
 ## Uses
 
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+Fine-tuning Gemma, Google's advanced open Large Language Model (LLM), using ChatML and Hugging Face TRL opens up a multitude of potential applications across various domains:
 
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+1. Natural Language Understanding (NLU):
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+- Enhancing Gemma's ability to comprehend and interpret natural language queries, commands, and conversations.
+Improving sentiment analysis, named entity recognition, and text classification tasks in applications such as chatbots, virtual assistants, and customer support systems.
+Text Generation and Summarization:
 
-[More Information Needed]
+- Generating coherent and contextually relevant text for content generation tasks such as article writing, story generation, and summarization.
+Creating personalized content recommendations and summaries based on user preferences and input.
 
-### Out-of-Scope Use
+2. Language Translation and Localization:
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+- Fine-tuning Gemma for translation tasks to improve the accuracy and fluency of translated text across multiple languages.
+- Facilitating localization efforts by adapting Gemma to specific dialects, slang, or cultural nuances.
 
-[More Information Needed]
+3. Information Retrieval and Question Answering:
 
-## Bias, Risks, and Limitations
+- Improving Gemma's ability to retrieve relevant information from vast datasets or knowledge bases in response to user queries.
+- Enhancing question answering systems for providing accurate and concise responses to user questions across various domains.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+4. Content Moderation and Filtering:
 
-[More Information Needed]
+- Training Gemma to identify and moderate inappropriate or harmful content in online forums, social media platforms, and other user-generated content.
+- Implementing content filtering mechanisms to ensure user safety and compliance with community guidelines.
 
-### Recommendations
+5. Personalization and Recommendation Systems:
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+- Tailoring Gemma to individual user preferences and behavior patterns for personalized product recommendations, content suggestions, and targeted advertisements.
+- Enhancing recommendation systems in e-commerce, streaming platforms, and social media networks to improve user engagement and satisfaction.
 
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
 
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
+```py
+import torch
+from peft import AutoPeftModelForCausalLM
+from transformers import  AutoTokenizer, pipeline
+
+peft_model_id = "Andyrasika/Gemma-ChatML"
+
+# Load Model with PEFT adapter
+tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
+model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id, device_map="auto", torch_dtype=torch.float16)
+pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
+# get token id for end of conversation
+eos_token = tokenizer("<|im_end|>",add_special_tokens=False)["input_ids"][0]
+```
+
+```py
+prompts = [
+    "What is the capital of Germany? Explain why thats the case and if it was different in the past?",
+    "Write a Python function to calculate the factorial of a number.",
+    "A rectangular garden has a length of 25 feet and a width of 15 feet. If you want to build a fence around the entire garden, how many feet of fencing will you need?",
+    "What is the difference between a fruit and a vegetable? Give examples of each.",
+]
+
+def test_inference(prompt):
+    prompt = pipe.tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True)
+    outputs = pipe(prompt, max_new_tokens=1024, do_sample=True, temperature=0.7, top_k=50, top_p=0.95, eos_token_id=eos_token)
+    return outputs[0]['generated_text'][len(prompt):].strip()
+
+
+for prompt in prompts:
+    print(f"    prompt:\n{prompt}")
+    print(f"    response:\n{test_inference(prompt)}")
+    print("-"*50)
+```
+## Output
+
+```
+ prompt:
+What is the capital of Germany? Explain why thats the case and if it was different in the past?
+/opt/conda/envs/pytorch/lib/python3.10/site-packages/transformers/pipelines/base.py:1157: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
+  warnings.warn(
+    response:
+The capital of Germany is Berlin. Berlin is the capital of Germany because it is the largest city in Germany, and it is the political center of the country. Berlin is also home to the German parliament, the Bundestag, and the federal government. In the past, the capital of Germany was Berlin, but it was divided into two parts during the Cold War. The western part of Berlin was controlled by the United States, France, and the United Kingdom, while the eastern part was controlled by the Soviet Union. Berlin was also the site of the Berlin Wall, which separated the two sides of the city from 1961 to 1989. The fall of the Berlin Wall marked the end of the Cold War, and Germany was reunified in 1990. Berlin has since become the capital of a reunified Germany.
+--------------------------------------------------
+    prompt:
+Write a Python function to calculate the factorial of a number.
+    response:
+def factorial(n):
+    fact = 1
+    for i in range(1, n+1):
+        fact = fact * i
+    return fact
+--------------------------------------------------
+    prompt:
+A rectangular garden has a length of 25 feet and a width of 15 feet. If you want to build a fence around the entire garden, how many feet of fencing will you need?
+    response:
+The total amount of fencing needed is 70 feet. This is determined by multiplying the perimeter of the garden (which is 70 feet) by the cost per foot of the fencing material.
+--------------------------------------------------
+    prompt:
+What is the difference between a fruit and a vegetable? Give examples of each.
+    response:
+A fruit is the part of a plant that contains the seeds. Examples of fruits are apples, oranges, strawberries, and grapes. A vegetable is the part of a plant that is used for food but does not contain seeds. Examples of vegetables are carrots, potatoes, broccoli, and lettuce.
+--------------------------------------------------
+```
 
 - PEFT 0.8.2