Update README.md
Browse files
README.md
CHANGED
@@ -1,22 +1,85 @@
|
|
1 |
---
|
2 |
-
|
|
|
3 |
tags:
|
4 |
-
- text-generation-inference
|
5 |
-
- transformers
|
6 |
- unsloth
|
7 |
-
- llama
|
8 |
- trl
|
9 |
-
|
|
|
|
|
|
|
|
|
|
|
10 |
language:
|
11 |
- en
|
|
|
|
|
12 |
---
|
13 |
|
14 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
-
|
17 |
-
- **License:** apache-2.0
|
18 |
-
- **Finetuned from model :** unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
|
19 |
|
20 |
-
|
21 |
|
22 |
-
|
|
|
|
|
|
1 |
---
|
2 |
+
license: apache-2.0
|
3 |
+
base_model: meta-llama/Llama-3.2-1B-Instruct
|
4 |
tags:
|
|
|
|
|
5 |
- unsloth
|
|
|
6 |
- trl
|
7 |
+
- sft
|
8 |
+
- json
|
9 |
+
- structured-output
|
10 |
+
- fine-tuned
|
11 |
+
- llama
|
12 |
+
- pydantic
|
13 |
language:
|
14 |
- en
|
15 |
+
pipeline_tag: text-generation
|
16 |
+
library_name: transformers
|
17 |
---
|
18 |
|
19 |
+
# Llama 3.2 1B JSON Extractor
|
20 |
+
|
21 |
+
A fine-tuned version of **Llama 3.2 1B Instruct** specialized for generating structured JSON outputs with high accuracy and schema compliance.
|
22 |
+
|
23 |
+
## 🎯 Model Description
|
24 |
+
|
25 |
+
This model has been fine-tuned to excel at generating valid, well-structured JSON objects based on Pydantic model schemas. It transforms natural language prompts into properly formatted JSON responses with remarkable consistency.
|
26 |
+
|
27 |
+
## 📊 Performance
|
28 |
+
|
29 |
+
**🚀 Dramatic Improvement in JSON Generation:**
|
30 |
+
- **JSON Validity Rate**: 20% → 92% (over 70% improvement)
|
31 |
+
- **Schema Compliance**: Near-perfect adherence to Pydantic model structures
|
32 |
+
- **Generalization**: Successfully handles completely new, unseen Pydantic model classes
|
33 |
+
|
34 |
+
## 🔧 Training Details
|
35 |
+
|
36 |
+
- **Base Model**: meta-llama/Llama-3.2-1B-Instruct
|
37 |
+
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation) with Unsloth
|
38 |
+
- **Training Data**: Synthetic dataset with 15+ diverse Pydantic model types
|
39 |
+
- **Training Epochs**: 15
|
40 |
+
- **Batch Size**: 16 (with gradient accumulation)
|
41 |
+
- **Learning Rate**: 1e-4
|
42 |
+
|
43 |
+
## 🏗️ Supported Model Types
|
44 |
+
|
45 |
+
The model can generate JSON for 15+ different object types including:
|
46 |
+
|
47 |
+
- **Educational**: Course, Resume, Events
|
48 |
+
- **Entertainment**: FilmIdea, BookReview, GameIdea
|
49 |
+
- **Business**: TShirtOrder, Recipe, House
|
50 |
+
- **Characters & Gaming**: FictionalCharacter, GameArtifact
|
51 |
+
- **Travel**: Itinerary
|
52 |
+
- **Science**: SollarSystem, TextSummary
|
53 |
+
- **And many more...**
|
54 |
+
|
55 |
+
## 🎯 Key Features
|
56 |
+
|
57 |
+
- **High JSON Validity**: 92% success rate in generating valid JSON
|
58 |
+
- **Schema Compliance**: Follows Pydantic model structures precisely
|
59 |
+
- **Strong Generalization**: Works with new, unseen model classes
|
60 |
+
- **Consistent Output**: Reliable structured data generation
|
61 |
+
- **Lightweight**: Only 1B parameters for efficient deployment
|
62 |
+
|
63 |
+
## 📚 Training Data
|
64 |
+
|
65 |
+
The model was fine-tuned on a synthetic dataset containing thousands of examples across diverse domains:
|
66 |
+
- Character creation and game development
|
67 |
+
- Business and e-commerce objects
|
68 |
+
- Educational and professional content
|
69 |
+
- Entertainment and media descriptions
|
70 |
+
- Scientific and technical data structures
|
71 |
+
|
72 |
+
## 🔗 Links
|
73 |
+
|
74 |
+
- **GitHub Repository**: [LLM_FineTuning_4JsonCreation](https://github.com/Dekanenko/Llama_FineTune_JSON_Creation)
|
75 |
+
- **Base Model**: [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
|
76 |
+
|
77 |
+
## 📄 License
|
78 |
|
79 |
+
This model is released under the Apache 2.0 license.
|
|
|
|
|
80 |
|
81 |
+
## 🙏 Acknowledgments
|
82 |
|
83 |
+
- **Meta** for the base Llama 3.2 model
|
84 |
+
- **Unsloth** for efficient fine-tuning framework
|
85 |
+
- **Hugging Face** for model hosting and ecosystem
|