TomBombadyl commited on
Commit
db716ba
·
verified ·
1 Parent(s): 9f977ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +205 -231
README.md CHANGED
@@ -1,232 +1,206 @@
1
- ---
2
- language:
3
- - en
4
- license: mit
5
- library_name: transformers
6
- tags:
7
- - robotics
8
- - isaac-sim
9
- - code-generation
10
- - simulation
11
- - qwen2
12
- - causal-lm
13
- - text-generation
14
- - text2text-generation
15
- - omni
16
- - nvidia
17
- - robotics-simulation
18
- pipeline_tag: text-generation
19
- base_model: Qwen/Qwen2.5-Coder-7B-Instruct
20
- model-index:
21
- - name: Qwen2.5-Coder-7B-Instruct-Omni1.1
22
- results:
23
- - task:
24
- type: text-generation
25
- name: Isaac Sim Robotics Code Generation
26
- dataset:
27
- type: custom
28
- name: Isaac Sim 5.0 Synthetic Dataset
29
- metrics:
30
- - type: accuracy
31
- value: 0.95
32
- name: Domain Accuracy
33
- - type: code_quality
34
- value: 0.90
35
- name: Python Code Quality
36
- - task:
37
- type: text-generation
38
- name: Robotics Simulation Setup
39
- dataset:
40
- type: custom
41
- name: Isaac Sim 5.0 Synthetic Dataset
42
- metrics:
43
- - type: accuracy
44
- value: 0.94
45
- name: Simulation Setup Accuracy
46
- ---
47
-
48
- # Isaac Sim Robotics Qwen2.5-Coder-7B-Instruct-Omni1.1
49
-
50
- [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Model-blue)](https://huggingface.co/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1)
51
- [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
52
- [![Isaac Sim](https://img.shields.io/badge/Isaac%20Sim-5.0-orange)](https://docs.omniverse.nvidia.com/isaacsim/)
53
-
54
- A specialized fine-tuned Qwen2.5-Coder-7B-Instruct model optimized for Isaac Sim 5.0 robotics development, computer vision, and simulation tasks.
55
-
56
- ## 🚀 Quick Start
57
-
58
- ### Option 1: HuggingFace Transformers (Recommended)
59
- ```python
60
- from transformers import AutoModelForCausalLM, AutoTokenizer
61
-
62
- model_name = "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1"
63
- tokenizer = AutoTokenizer.from_pretrained(model_name)
64
- model = AutoModelForCausalLM.from_pretrained(model_name)
65
-
66
- # Isaac Sim robotics query
67
- query = """<|im_start|>user
68
- How do I create a robot with differential drive in Isaac Sim 5.0?
69
- <|im_end|>
70
- <|im_start|>assistant"""
71
-
72
- inputs = tokenizer(query, return_tensors="pt")
73
- outputs = model.generate(**inputs, max_length=512)
74
- response = tokenizer.decode(outputs[0], skip_special_tokens=True)
75
- print(response)
76
- ```
77
-
78
- ### Option 2: CTransformers (Lightweight)
79
- ```python
80
- from ctransformers import AutoModelForCausalLM
81
-
82
- model = AutoModelForCausalLM.from_pretrained(
83
- "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
84
- model_type="qwen2",
85
- gpu_layers=0 # CPU inference
86
- )
87
-
88
- # Same usage pattern as above
89
- ```
90
-
91
- ### Option 3: GGUF Conversion (Advanced)
92
- ```bash
93
- # Convert to GGUF format for llama.cpp
94
- python scripts/convert_to_gguf.py
95
-
96
- # Use with llama.cpp
97
- ./llama-server --model models/gguf/isaac_sim_qwen2.5_coder_q4_0.gguf --port 8080
98
- ```
99
-
100
- ## 🎯 Model Capabilities
101
-
102
- - **Isaac Sim 5.0 Expertise**: Deep knowledge of robotics simulation APIs
103
- - **Computer Vision**: Understanding of sensor integration and perception
104
- - **Robot Control**: Programming differential drive, manipulators, and sensors
105
- - **Simulation Setup**: Environment configuration and physics parameters
106
- - **Code Generation**: Python scripts for Isaac Sim workflows
107
- - **Troubleshooting**: Common issues and solutions
108
-
109
- ## 📊 Performance
110
-
111
- - **Base Model**: Qwen2.5-Coder-7B-Instruct
112
- - **Training Data**: 2,000 Isaac Sim-specific examples
113
- - **Training Method**: LoRA fine-tuning (rank 64, alpha 128)
114
- - **Hardware**: NVIDIA RTX 4070 Laptop GPU (8.5GB VRAM)
115
- - **Training Steps**: 300 with curriculum learning
116
-
117
- ## 🔧 Installation
118
-
119
- ```bash
120
- # Clone repository
121
- git clone https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1.git
122
- cd Qwen2.5-Coder-7B-Instruct-Omni1.1
123
-
124
- # Install dependencies
125
- pip install -r requirements.txt
126
-
127
- # Download models (choose one)
128
- # Option 1: HuggingFace (5.3GB)
129
- huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/huggingface
130
-
131
- # Option 2: CTransformers (5.2GB)
132
- huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/ctransformers
133
-
134
- # Option 3: GGUF (616MB + conversion)
135
- huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/gguf
136
- ```
137
-
138
- ## 📚 Examples
139
-
140
- ### Isaac Sim Robot Creation
141
- ```python
142
- from transformers import AutoModelForCausalLM, AutoTokenizer
143
-
144
- model = AutoModelForCausalLM.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
145
- tokenizer = AutoTokenizer.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
146
-
147
- query = """<|im_start|>user
148
- Create a Python script to spawn a UR5 robot in Isaac Sim 5.0 with proper physics properties.
149
- <|im_end|>
150
- <|im_start|>assistant"""
151
-
152
- # Generate response
153
- inputs = tokenizer(query, return_tensors="pt")
154
- outputs = model.generate(**inputs, max_length=1024, temperature=0.7)
155
- response = tokenizer.decode(outputs[0], skip_special_tokens=True)
156
- print(response)
157
- ```
158
-
159
- ### Sensor Integration
160
- ```python
161
- query = """<|im_start|>user
162
- How do I add a depth camera to my robot and process the depth data in Isaac Sim?
163
- <|im_end|>
164
- <|im_start|>assistant"""
165
- ```
166
-
167
- ## ⚠️ Known Limitations
168
-
169
- ### GGUF Conversion Issues
170
- The GGUF conversion currently has metadata compatibility issues:
171
- - **Error**: Missing `qwen2.context_length` field
172
- - **Workaround**: Use HuggingFace or CTransformers formats
173
- - **Status**: Under investigation for future updates
174
-
175
- ### Hardware Requirements
176
- - **HuggingFace**: 8GB+ VRAM for full precision
177
- - **CTransformers**: 4GB+ VRAM for optimized inference
178
- - **GGUF**: 2GB+ VRAM (when conversion is fixed)
179
-
180
- ## 🛠️ Troubleshooting
181
-
182
- ### Common Issues
183
-
184
- 1. **Out of Memory Errors**
185
- ```python
186
- # Use 8-bit quantization
187
- model = AutoModelForCausalLM.from_pretrained(
188
- "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
189
- load_in_8bit=True,
190
- device_map="auto"
191
- )
192
- ```
193
-
194
- 2. **GGUF Loading Failures**
195
- - Use HuggingFace or CTransformers formats instead
196
- - Check [troubleshooting guide](docs/troubleshooting.md)
197
-
198
- 3. **Isaac Sim Integration Issues**
199
- - Ensure Isaac Sim 5.0+ is installed
200
- - Check [integration examples](examples/isaac_sim_integration.py)
201
-
202
- ## 📖 Documentation
203
-
204
- - [Model Card](model_card.md) - Detailed model information
205
- - [Training Methodology](docs/training_methodology.md) - How the model was trained
206
- - [Performance Benchmarks](docs/performance_benchmarks.md) - Evaluation results
207
- - [Troubleshooting Guide](docs/troubleshooting.md) - Common issues and solutions
208
-
209
- ## 🤝 Contributing
210
-
211
- We welcome contributions! Please see our [contributing guidelines](CONTRIBUTING.md) for details.
212
-
213
- ## 📄 License
214
-
215
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
216
-
217
- ## 🙏 Acknowledgments
218
-
219
- - **NVIDIA Isaac Sim Team** for the simulation platform
220
- - **Qwen Team** for the base model
221
- - **Hugging Face** for the training infrastructure
222
- - **Open Source Community** for tools and libraries
223
-
224
- ## 📞 Support
225
-
226
- - **Issues**: [GitHub Issues](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/issues)
227
- - **Discussions**: [GitHub Discussions](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/discussions)
228
- - **Documentation**: [Full Documentation](docs/)
229
-
230
- ---
231
-
232
  **Note**: This model is specifically trained for Isaac Sim 5.0 robotics development. For general coding tasks, consider using the base Qwen2.5-Coder-7B-Instruct model.
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ library_name: transformers
6
+ tags:
7
+ - robotics
8
+ - isaac-sim
9
+ - code-generation
10
+ - simulation
11
+ - qwen2
12
+ - causal-lm
13
+ - text-generation
14
+ - text2text-generation
15
+ - omni
16
+ - nvidia
17
+ - robotics-simulation
18
+ pipeline_tag: text-generation
19
+ base_model: Qwen/Qwen2.5-Coder-7B-Instruct
20
+ ---
21
+
22
+ # Isaac Sim Robotics Qwen2.5-Coder-7B-Instruct-Omni1.1
23
+
24
+ [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Model-blue)](https://huggingface.co/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1)
25
+ [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
26
+ [![Isaac Sim](https://img.shields.io/badge/Isaac%20Sim-5.0-orange)](https://docs.omniverse.nvidia.com/isaacsim/)
27
+
28
+ A specialized fine-tuned Qwen2.5-Coder-7B-Instruct model optimized for Isaac Sim 5.0 robotics development, computer vision, and simulation tasks.
29
+
30
+ ## 🚀 Quick Start
31
+
32
+ ### Option 1: HuggingFace Transformers (Recommended)
33
+ ```python
34
+ from transformers import AutoModelForCausalLM, AutoTokenizer
35
+
36
+ model_name = "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1"
37
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
38
+ model = AutoModelForCausalLM.from_pretrained(model_name)
39
+
40
+ # Isaac Sim robotics query
41
+ query = """<|im_start|>user
42
+ How do I create a robot with differential drive in Isaac Sim 5.0?
43
+ <|im_end|>
44
+ <|im_start|>assistant"""
45
+
46
+ inputs = tokenizer(query, return_tensors="pt")
47
+ outputs = model.generate(**inputs, max_length=512)
48
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
49
+ print(response)
50
+ ```
51
+
52
+ ### Option 2: CTransformers (Lightweight)
53
+ ```python
54
+ from ctransformers import AutoModelForCausalLM
55
+
56
+ model = AutoModelForCausalLM.from_pretrained(
57
+ "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
58
+ model_type="qwen2",
59
+ gpu_layers=0 # CPU inference
60
+ )
61
+
62
+ # Same usage pattern as above
63
+ ```
64
+
65
+ ### Option 3: GGUF Conversion (Advanced)
66
+ ```bash
67
+ # Convert to GGUF format for llama.cpp
68
+ python scripts/convert_to_gguf.py
69
+
70
+ # Use with llama.cpp
71
+ ./llama-server --model models/gguf/isaac_sim_qwen2.5_coder_q4_0.gguf --port 8080
72
+ ```
73
+
74
+ ## 🎯 Model Capabilities
75
+
76
+ - **Isaac Sim 5.0 Expertise**: Deep knowledge of robotics simulation APIs
77
+ - **Computer Vision**: Understanding of sensor integration and perception
78
+ - **Robot Control**: Programming differential drive, manipulators, and sensors
79
+ - **Simulation Setup**: Environment configuration and physics parameters
80
+ - **Code Generation**: Python scripts for Isaac Sim workflows
81
+ - **Troubleshooting**: Common issues and solutions
82
+
83
+ ## 📊 Performance
84
+
85
+ - **Base Model**: Qwen2.5-Coder-7B-Instruct
86
+ - **Training Data**: 2,000 Isaac Sim-specific examples
87
+ - **Training Method**: LoRA fine-tuning (rank 64, alpha 128)
88
+ - **Hardware**: NVIDIA RTX 4070 Laptop GPU (8.5GB VRAM)
89
+ - **Training Steps**: 300 with curriculum learning
90
+
91
+ ## 🔧 Installation
92
+
93
+ ```bash
94
+ # Clone repository
95
+ git clone https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1.git
96
+ cd Qwen2.5-Coder-7B-Instruct-Omni1.1
97
+
98
+ # Install dependencies
99
+ pip install -r requirements.txt
100
+
101
+ # Download models (choose one)
102
+ # Option 1: HuggingFace (5.3GB)
103
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/huggingface
104
+
105
+ # Option 2: CTransformers (5.2GB)
106
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/ctransformers
107
+
108
+ # Option 3: GGUF (616MB + conversion)
109
+ huggingface-cli download TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1 --local-dir models/gguf
110
+ ```
111
+
112
+ ## 📚 Examples
113
+
114
+ ### Isaac Sim Robot Creation
115
+ ```python
116
+ from transformers import AutoModelForCausalLM, AutoTokenizer
117
+
118
+ model = AutoModelForCausalLM.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
119
+ tokenizer = AutoTokenizer.from_pretrained("TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1")
120
+
121
+ query = """<|im_start|>user
122
+ Create a Python script to spawn a UR5 robot in Isaac Sim 5.0 with proper physics properties.
123
+ <|im_end|>
124
+ <|im_start|>assistant"""
125
+
126
+ # Generate response
127
+ inputs = tokenizer(query, return_tensors="pt")
128
+ outputs = model.generate(**inputs, max_length=1024, temperature=0.7)
129
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
130
+ print(response)
131
+ ```
132
+
133
+ ### Sensor Integration
134
+ ```python
135
+ query = """<|im_start|>user
136
+ How do I add a depth camera to my robot and process the depth data in Isaac Sim?
137
+ <|im_end|>
138
+ <|im_start|>assistant"""
139
+ ```
140
+
141
+ ## ⚠️ Known Limitations
142
+
143
+ ### GGUF Conversion Issues
144
+ The GGUF conversion currently has metadata compatibility issues:
145
+ - **Error**: Missing `qwen2.context_length` field
146
+ - **Workaround**: Use HuggingFace or CTransformers formats
147
+ - **Status**: Under investigation for future updates
148
+
149
+ ### Hardware Requirements
150
+ - **HuggingFace**: 8GB+ VRAM for full precision
151
+ - **CTransformers**: 4GB+ VRAM for optimized inference
152
+ - **GGUF**: 2GB+ VRAM (when conversion is fixed)
153
+
154
+ ## 🛠️ Troubleshooting
155
+
156
+ ### Common Issues
157
+
158
+ 1. **Out of Memory Errors**
159
+ ```python
160
+ # Use 8-bit quantization
161
+ model = AutoModelForCausalLM.from_pretrained(
162
+ "TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1",
163
+ load_in_8bit=True,
164
+ device_map="auto"
165
+ )
166
+ ```
167
+
168
+ 2. **GGUF Loading Failures**
169
+ - Use HuggingFace or CTransformers formats instead
170
+ - Check [troubleshooting guide](docs/troubleshooting.md)
171
+
172
+ 3. **Isaac Sim Integration Issues**
173
+ - Ensure Isaac Sim 5.0+ is installed
174
+ - Check [integration examples](examples/isaac_sim_integration.py)
175
+
176
+ ## 📖 Documentation
177
+
178
+ - [Model Card](model_card.md) - Detailed model information
179
+ - [Training Methodology](docs/training_methodology.md) - How the model was trained
180
+ - [Performance Benchmarks](docs/performance_benchmarks.md) - Evaluation results
181
+ - [Troubleshooting Guide](docs/troubleshooting.md) - Common issues and solutions
182
+
183
+ ## 🤝 Contributing
184
+
185
+ We welcome contributions! Please see our [contributing guidelines](CONTRIBUTING.md) for details.
186
+
187
+ ## 📄 License
188
+
189
+ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
190
+
191
+ ## 🙏 Acknowledgments
192
+
193
+ - **NVIDIA Isaac Sim Team** for the simulation platform
194
+ - **Qwen Team** for the base model
195
+ - **Hugging Face** for the training infrastructure
196
+ - **Open Source Community** for tools and libraries
197
+
198
+ ## 📞 Support
199
+
200
+ - **Issues**: [GitHub Issues](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/issues)
201
+ - **Discussions**: [GitHub Discussions](https://github.com/TomBombadyl/Qwen2.5-Coder-7B-Instruct-Omni1.1/discussions)
202
+ - **Documentation**: [Full Documentation](docs/)
203
+
204
+ ---
205
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
  **Note**: This model is specifically trained for Isaac Sim 5.0 robotics development. For general coding tasks, consider using the base Qwen2.5-Coder-7B-Instruct model.