MathBite commited on
Commit
5c57c70
·
verified ·
1 Parent(s): 9b40f8e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -11
README.md CHANGED
@@ -1,22 +1,85 @@
1
  ---
2
- base_model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
 
3
  tags:
4
- - text-generation-inference
5
- - transformers
6
  - unsloth
7
- - llama
8
  - trl
9
- license: apache-2.0
 
 
 
 
 
10
  language:
11
  - en
 
 
12
  ---
13
 
14
- # Uploaded model
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- - **Developed by:** MathBite
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
19
 
20
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
1
  ---
2
+ license: apache-2.0
3
+ base_model: meta-llama/Llama-3.2-1B-Instruct
4
  tags:
 
 
5
  - unsloth
 
6
  - trl
7
+ - sft
8
+ - json
9
+ - structured-output
10
+ - fine-tuned
11
+ - llama
12
+ - pydantic
13
  language:
14
  - en
15
+ pipeline_tag: text-generation
16
+ library_name: transformers
17
  ---
18
 
19
+ # Llama 3.2 1B JSON Extractor
20
+
21
+ A fine-tuned version of **Llama 3.2 1B Instruct** specialized for generating structured JSON outputs with high accuracy and schema compliance.
22
+
23
+ ## 🎯 Model Description
24
+
25
+ This model has been fine-tuned to excel at generating valid, well-structured JSON objects based on Pydantic model schemas. It transforms natural language prompts into properly formatted JSON responses with remarkable consistency.
26
+
27
+ ## 📊 Performance
28
+
29
+ **🚀 Dramatic Improvement in JSON Generation:**
30
+ - **JSON Validity Rate**: 20% → 92% (over 70% improvement)
31
+ - **Schema Compliance**: Near-perfect adherence to Pydantic model structures
32
+ - **Generalization**: Successfully handles completely new, unseen Pydantic model classes
33
+
34
+ ## 🔧 Training Details
35
+
36
+ - **Base Model**: meta-llama/Llama-3.2-1B-Instruct
37
+ - **Fine-tuning Method**: LoRA (Low-Rank Adaptation) with Unsloth
38
+ - **Training Data**: Synthetic dataset with 15+ diverse Pydantic model types
39
+ - **Training Epochs**: 15
40
+ - **Batch Size**: 16 (with gradient accumulation)
41
+ - **Learning Rate**: 1e-4
42
+
43
+ ## 🏗️ Supported Model Types
44
+
45
+ The model can generate JSON for 15+ different object types including:
46
+
47
+ - **Educational**: Course, Resume, Events
48
+ - **Entertainment**: FilmIdea, BookReview, GameIdea
49
+ - **Business**: TShirtOrder, Recipe, House
50
+ - **Characters & Gaming**: FictionalCharacter, GameArtifact
51
+ - **Travel**: Itinerary
52
+ - **Science**: SollarSystem, TextSummary
53
+ - **And many more...**
54
+
55
+ ## 🎯 Key Features
56
+
57
+ - **High JSON Validity**: 92% success rate in generating valid JSON
58
+ - **Schema Compliance**: Follows Pydantic model structures precisely
59
+ - **Strong Generalization**: Works with new, unseen model classes
60
+ - **Consistent Output**: Reliable structured data generation
61
+ - **Lightweight**: Only 1B parameters for efficient deployment
62
+
63
+ ## 📚 Training Data
64
+
65
+ The model was fine-tuned on a synthetic dataset containing thousands of examples across diverse domains:
66
+ - Character creation and game development
67
+ - Business and e-commerce objects
68
+ - Educational and professional content
69
+ - Entertainment and media descriptions
70
+ - Scientific and technical data structures
71
+
72
+ ## 🔗 Links
73
+
74
+ - **GitHub Repository**: [LLM_FineTuning_4JsonCreation](https://github.com/Dekanenko/Llama_FineTune_JSON_Creation)
75
+ - **Base Model**: [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
76
+
77
+ ## 📄 License
78
 
79
+ This model is released under the Apache 2.0 license.
 
 
80
 
81
+ ## 🙏 Acknowledgments
82
 
83
+ - **Meta** for the base Llama 3.2 model
84
+ - **Unsloth** for efficient fine-tuning framework
85
+ - **Hugging Face** for model hosting and ecosystem