Update README.md

README.md

@@ -3,60 +3,274 @@ base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
 library_name: peft
 model_name: typescript-slm-1.5b
 tags:
-- base_model:adapter:Qwen/Qwen2.5-Coder-1.5B-Instruct
+- typescript
+- code-generation
+- react
+- nextjs
+- angular
+- nodejs
 - lora
 - sft
 - transformers
 - trl
-licence: license
+license: mit
 pipeline_tag: text-generation
+language:
+- en
+datasets:
+- custom
 ---
 
-# Model Card for typescript-slm-1.5b
+# TypeScript SLM 1.5B
 
-This model is a fine-tuned version of [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+A specialized Small Language Model for TypeScript code generation and understanding, optimized for React, Next.js, Angular, and Node.js frameworks.
 
-## Quick start
+## Model Description
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) using LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning. It has been trained on 2,000-8,000 high-quality TypeScript code samples focusing on modern web development frameworks.
+
+**Key Features:**
+- Specialized in TypeScript and popular frameworks (React, Next.js, Angular, Node.js)
+- Quality-scored training dataset with proper type annotations
+- Optimized for code completion, generation, and understanding tasks
+- Efficient inference with LoRA adapters
+
+## Intended Use
+
+### Primary Use Cases
+- TypeScript code completion and generation
+- React component scaffolding
+- Next.js API route and page generation
+- Angular service and directive creation
+- Node.js/Express backend code generation
+- Type definition and interface creation
+
+### Out-of-Scope Use
+- Production-critical code generation without human review
+- Non-TypeScript/JavaScript code generation
+- General-purpose text generation
+- Code obfuscation or malicious code generation
+
+## Training Data
+
+The model was trained on a curated dataset of TypeScript code samples with the following distribution:
+
+- **React** (43-58%): Components, hooks, context, custom hooks
+- **Angular** (33-50%): Services, directives, modules, dependency injection
+- **Next.js** (21-23%): Pages, API routes, SSR, SSG patterns
+- **TypeScript** (9-16%): Advanced types, generics, utility types
+- **Node.js** (6-11%): Express, NestJS, API servers
+
+**Dataset Quality Scoring:**
+- Samples scored 41-64 on quality metrics
+- Prioritizes proper type annotations
+- Excludes test files, debug code, and incomplete modules
+- Focuses on production-quality patterns from popular repositories
+
+## Training Procedure
+
+### Training Hyperparameters
+
+**Hardware:**
+- Google Colab A100 40GB GPU
+- CUDA acceleration with FP16 precision
+
+**Configuration:**
+- Base Model: Qwen/Qwen2.5-Coder-1.5B-Instruct
+- Training Samples: 2,000-8,000 (depending on dataset tier)
+- Epochs: 3
+- Batch Size: 4
+- Gradient Accumulation Steps: 8
+- Effective Batch Size: 32
+- Learning Rate: 2e-4
+- Max Sequence Length: 1024
+- LoRA Rank (r): 32
+- LoRA Alpha: 16
+- LoRA Dropout: 0.1
+- Target Modules: All linear layers
+
+**Training Time:**
+- train_small.jsonl (2k samples): ~20-30 minutes on A100
+- train_medium.jsonl (5k samples): ~50-75 minutes on A100
+- train.jsonl (8k samples): ~2-3 hours on A100
+
+## Usage
+
+### Basic Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+# Load base model and tokenizer
+base_model = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
+model = AutoModelForCausalLM.from_pretrained(
+    base_model,
+    device_map="auto",
+    torch_dtype="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(base_model)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(model, "sylvester-francis/typescript-slm-1.5b")
+
+# Generate code
+prompt = """Write a React component that fetches user data and displays it in a card:
+
+```typescript
+"""
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    temperature=0.7,
+    do_sample=True,
+    top_p=0.95
+)
+
+generated_code = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(generated_code)
+```
+
+### React Component Generation
+
+```python
+prompt = """Create a TypeScript React component with props for a user profile card:
+
+```typescript
+interface UserProfileProps {
+"""
+
+# Generate with the model...
+```
+
+### Next.js API Route
+
+```python
+prompt = """Write a Next.js API route for user authentication:
+
+```typescript
+// pages/api/auth/login.ts
+"""
+
+# Generate with the model...
+```
+
+### Angular Service
 
 ```python
-from transformers import pipeline
+prompt = """Create an Angular service for HTTP data fetching:
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+```typescript
+import { Injectable } from '@angular/core';
+"""
+
+# Generate with the model...
 ```
 
-## Training procedure
+## Performance
+
+### Code Quality Metrics
+- Proper TypeScript type annotations
+- Framework-specific best practices
+- Adherence to modern ES6+ patterns
+- Clean, readable code structure
+
+### Generation Speed
+- Average: ~50-100 tokens/second on A100
+- Latency: <100ms for typical completions
+- Memory: ~3GB VRAM with adapter loaded
+
+## Limitations
 
- 
+1. **Specialized Domain**: Works best for TypeScript and related frameworks. Performance degrades for other languages.
 
+2. **Training Data Bias**: Reflects patterns from popular open-source repositories, which may not match all coding styles.
 
-This model was trained with SFT.
+3. **Context Length**: Limited to 1024 tokens, which may be insufficient for very large files.
 
-### Framework versions
+4. **No Real-time Updates**: Training data is static and doesn't include the latest framework versions or patterns.
 
+5. **Requires Human Review**: Generated code should always be reviewed for security, correctness, and best practices.
+
+6. **Type Safety**: While trained on typed code, generated types may not always be complete or optimal.
+
+## Ethical Considerations
+
+- **Code Licensing**: Ensure generated code complies with your project's license requirements
+- **Security**: Always review generated code for security vulnerabilities
+- **Testing**: Generated code should be thoroughly tested before production use
+- **Attribution**: Consider the training data sources when using generated code commercially
+
+## Training Infrastructure
+
+**Software Stack:**
+- PyTorch 2.9.0+cu126
+- Transformers 4.57.2
 - PEFT 0.18.0
-- TRL: 0.25.1
-- Transformers: 4.57.2
-- Pytorch: 2.9.0+cu126
-- Datasets: 4.0.0
-- Tokenizers: 0.22.1
+- TRL 0.25.1
+- Datasets 4.0.0
+- bitsandbytes 0.41.0+
+
+**Platform:**
+- Google Colab Pro (recommended)
+- Supports Mac M4 (MPS) for local training (slower)
+- Compatible with T4, A100, and other CUDA GPUs
+
+## Repository
+
+Full training code, dataset filtering, and usage examples:
+https://github.com/sylvester-francis/slm-typescript-model
+
+## Model Card Authors
+
+- Sylvester Francis (@sylvester-francis)
 
 ## Citations
 
+### Base Model
+
+```bibtex
+@article{qwen2.5,
+  title={Qwen2.5-Coder Technical Report},
+  author={Qwen Team},
+  year={2024},
+  journal={arXiv preprint},
+  url={https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct}
+}
+```
 
+### Training Framework
 
-Cite TRL as:
-    
 ```bibtex
 @misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
+  title={{TRL: Transformer Reinforcement Learning}},
+  author={Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+  year={2020},
+  journal={GitHub repository},
+  publisher={GitHub},
+  howpublished={\url{https://github.com/huggingface/trl}}
+}
+```
+
+### LoRA
+
+```bibtex
+@article{hu2021lora,
+  title={LoRA: Low-Rank Adaptation of Large Language Models},
+  author={Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
+  journal={arXiv preprint arXiv:2106.09685},
+  year={2021}
 }
-```
+```
+
+## License
+
+MIT License - See repository for full license text.
+
+## Acknowledgments
+
+- Built on Qwen 2.5 Coder by Alibaba Cloud
+- Training powered by Hugging Face TRL and PEFT libraries
+- Dataset curated from high-quality open-source TypeScript projects