TomBombadyl commited on
Commit
d99ddab
·
verified ·
1 Parent(s): a9fdace

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +229 -0
README.md CHANGED
@@ -1,3 +1,232 @@
1
  ---
 
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
  license: mit
5
+ library_name: transformers
6
+ tags:
7
+ - robotics
8
+ - isaac-sim
9
+ - code-generation
10
+ - simulation
11
+ - qwen2
12
+ - causal-lm
13
+ - text-generation
14
+ - text2text-generation
15
+ - omni
16
+ - nvidia
17
+ - robotics-simulation
18
+ pipeline_tag: text-generation
19
+ base_model: Qwen/Qwen2.5-Coder-7B-Instruct
20
+ model-index:
21
+ - name: Qwen2.5-Coder-7B-Instruct-Omni1.1
22
+ results:
23
+ - task:
24
+ type: text-generation
25
+ name: Isaac Sim Robotics Code Generation
26
+ dataset:
27
+ type: custom
28
+ name: Isaac Sim 5.0 Synthetic Dataset
29
+ metrics:
30
+ - type: accuracy
31
+ value: 0.95
32
+ name: Domain Accuracy
33
+ - type: code_quality
34
+ value: 0.90
35
+ name: Python Code Quality
36
+ - task:
37
+ type: text-generation
38
+ name: Robotics Simulation Setup
39
+ dataset:
40
+ type: custom
41
+ name: Isaac Sim 5.0 Synthetic Dataset
42
+ metrics:
43
+ - type: accuracy
44
+ value: 0.94
45
+ name: Simulation Setup Accuracy
46
  ---
47
+
48
+ # Isaac Sim Robotics Qwen2.5-Coder-7B-Instruct-Omni1.1
49
+
50
+ [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Model-blue)](https://huggingface.co/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1)
51
+ [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
52
+ [![Isaac Sim](https://img.shields.io/badge/Isaac%20Sim-5.0-orange)](https://docs.omniverse.nvidia.com/isaacsim/)
53
+
54
+ A specialized fine-tuned Qwen2.5-Coder-7B-Instruct model optimized for Isaac Sim 5.0 robotics development, computer vision, and simulation tasks.
55
+
56
+ ## 🚀 Quick Start
57
+
58
+ ### Option 1: HuggingFace Transformers (Recommended)
59
+ ```python
60
+ from transformers import AutoModelForCausalLM, AutoTokenizer
61
+
62
+ model_name = "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1"
63
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
64
+ model = AutoModelForCausalLM.from_pretrained(model_name)
65
+
66
+ # Isaac Sim robotics query
67
+ query = """<|im_start|>user
68
+ How do I create a robot with differential drive in Isaac Sim 5.0?
69
+ <|im_end|>
70
+ <|im_start|>assistant"""
71
+
72
+ inputs = tokenizer(query, return_tensors="pt")
73
+ outputs = model.generate(**inputs, max_length=512)
74
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
75
+ print(response)
76
+ ```
77
+
78
+ ### Option 2: CTransformers (Lightweight)
79
+ ```python
80
+ from ctransformers import AutoModelForCausalLM
81
+
82
+ model = AutoModelForCausalLM.from_pretrained(
83
+ "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
84
+ model_type="qwen2",
85
+ gpu_layers=0 # CPU inference
86
+ )
87
+
88
+ # Same usage pattern as above
89
+ ```
90
+
91
+ ### Option 3: GGUF Conversion (Advanced)
92
+ ```bash
93
+ # Convert to GGUF format for llama.cpp
94
+ python scripts/convert_to_gguf.py
95
+
96
+ # Use with llama.cpp
97
+ ./llama-server --model models/gguf/isaac_sim_qwen2.5_coder_q4_0.gguf --port 8080
98
+ ```
99
+
100
+ ## 🎯 Model Capabilities
101
+
102
+ - **Isaac Sim 5.0 Expertise**: Deep knowledge of robotics simulation APIs
103
+ - **Computer Vision**: Understanding of sensor integration and perception
104
+ - **Robot Control**: Programming differential drive, manipulators, and sensors
105
+ - **Simulation Setup**: Environment configuration and physics parameters
106
+ - **Code Generation**: Python scripts for Isaac Sim workflows
107
+ - **Troubleshooting**: Common issues and solutions
108
+
109
+ ## 📊 Performance
110
+
111
+ - **Base Model**: Qwen2.5-Coder-7B-Instruct
112
+ - **Training Data**: 2,000 Isaac Sim-specific examples
113
+ - **Training Method**: LoRA fine-tuning (rank 64, alpha 128)
114
+ - **Hardware**: NVIDIA RTX 4070 Laptop GPU (8.5GB VRAM)
115
+ - **Training Steps**: 300 with curriculum learning
116
+
117
+ ## 🔧 Installation
118
+
119
+ ```bash
120
+ # Clone repository
121
+ git clone https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1.git
122
+ cd Qwen2.5-Coder-7B-Instruct-Omni1.1
123
+
124
+ # Install dependencies
125
+ pip install -r requirements.txt
126
+
127
+ # Download models (choose one)
128
+ # Option 1: HuggingFace (5.3GB)
129
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/huggingface
130
+
131
+ # Option 2: CTransformers (5.2GB)
132
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/ctransformers
133
+
134
+ # Option 3: GGUF (616MB + conversion)
135
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/gguf
136
+ ```
137
+
138
+ ## 📚 Examples
139
+
140
+ ### Isaac Sim Robot Creation
141
+ ```python
142
+ from transformers import AutoModelForCausalLM, AutoTokenizer
143
+
144
+ model = AutoModelForCausalLM.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
145
+ tokenizer = AutoTokenizer.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
146
+
147
+ query = """<|im_start|>user
148
+ Create a Python script to spawn a UR5 robot in Isaac Sim 5.0 with proper physics properties.
149
+ <|im_end|>
150
+ <|im_start|>assistant"""
151
+
152
+ # Generate response
153
+ inputs = tokenizer(query, return_tensors="pt")
154
+ outputs = model.generate(**inputs, max_length=1024, temperature=0.7)
155
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
156
+ print(response)
157
+ ```
158
+
159
+ ### Sensor Integration
160
+ ```python
161
+ query = """<|im_start|>user
162
+ How do I add a depth camera to my robot and process the depth data in Isaac Sim?
163
+ <|im_end|>
164
+ <|im_start|>assistant"""
165
+ ```
166
+
167
+ ## ⚠️ Known Limitations
168
+
169
+ ### GGUF Conversion Issues
170
+ The GGUF conversion currently has metadata compatibility issues:
171
+ - **Error**: Missing `qwen2.context_length` field
172
+ - **Workaround**: Use HuggingFace or CTransformers formats
173
+ - **Status**: Under investigation for future updates
174
+
175
+ ### Hardware Requirements
176
+ - **HuggingFace**: 8GB+ VRAM for full precision
177
+ - **CTransformers**: 4GB+ VRAM for optimized inference
178
+ - **GGUF**: 2GB+ VRAM (when conversion is fixed)
179
+
180
+ ## 🛠️ Troubleshooting
181
+
182
+ ### Common Issues
183
+
184
+ 1. **Out of Memory Errors**
185
+ ```python
186
+ # Use 8-bit quantization
187
+ model = AutoModelForCausalLM.from_pretrained(
188
+ "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
189
+ load_in_8bit=True,
190
+ device_map="auto"
191
+ )
192
+ ```
193
+
194
+ 2. **GGUF Loading Failures**
195
+ - Use HuggingFace or CTransformers formats instead
196
+ - Check [troubleshooting guide](docs/troubleshooting.md)
197
+
198
+ 3. **Isaac Sim Integration Issues**
199
+ - Ensure Isaac Sim 5.0+ is installed
200
+ - Check [integration examples](examples/isaac_sim_integration.py)
201
+
202
+ ## 📖 Documentation
203
+
204
+ - [Model Card](model_card.md) - Detailed model information
205
+ - [Training Methodology](docs/training_methodology.md) - How the model was trained
206
+ - [Performance Benchmarks](docs/performance_benchmarks.md) - Evaluation results
207
+ - [Troubleshooting Guide](docs/troubleshooting.md) - Common issues and solutions
208
+
209
+ ## 🤝 Contributing
210
+
211
+ We welcome contributions! Please see our [contributing guidelines](CONTRIBUTING.md) for details.
212
+
213
+ ## 📄 License
214
+
215
+ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
216
+
217
+ ## 🙏 Acknowledgments
218
+
219
+ - **NVIDIA Isaac Sim Team** for the simulation platform
220
+ - **Qwen Team** for the base model
221
+ - **Hugging Face** for the training infrastructure
222
+ - **Open Source Community** for tools and libraries
223
+
224
+ ## 📞 Support
225
+
226
+ - **Issues**: [GitHub Issues](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/issues)
227
+ - **Discussions**: [GitHub Discussions](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/discussions)
228
+ - **Documentation**: [Full Documentation](docs/)
229
+
230
+ ---
231
+
232
+ **Note**: This model is specifically trained for Isaac Sim 5.0 robotics development. For general coding tasks, consider using the base Qwen2.5-Coder-7B-Instruct model.