File size: 4,944 Bytes
7d00b39 981ac6c 7d00b39 f1c4854 ca006df f1c4854 0bb2e97 f1c4854 0bb2e97 f1c4854 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
---
license: apache-2.0
datasets:
- Magpie-Align/Magpie-Pro-300K-Filtered
- mlabonne/FineTome-100k
- unsloth/OpenMathReasoning-mini
- prithivMLmods/Grade-Math-18K
language:
- en
base_model:
- Qwen/Qwen3-0.6B
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- math
- code
- moe
---
# Magpie-Qwen-CortexDual-0.6B
> **Magpie-Qwen-CortexDual-0.6B** is a specialized, general-purpose model designed for **math**, **code**, and **structured reasoning**. Built with **CortexDual thinking mode**, it dynamically adapts to the complexity of a problem, automatically shifting into a stepwise reasoning mode for intricate logic or math tasks. This 0.6B parameter model leverages **80% of the Magpie Pro 330k dataset** and a modular blend of datasets for general-purpose proficiency and domain versatility.
> \[!note]
> GGUF : [https://huggingface.co/prithivMLmods/Magpie-Qwen-CortexDual-0.6B-GGUF](https://huggingface.co/prithivMLmods/Magpie-Qwen-CortexDual-0.6B-GGUF)
---
## Key Features
1. **Adaptive Reasoning via CortexDual**
Automatically switches into a deeper thinking mode for complex problems, simulating trace-style deduction for higher-order tasks in math and code.
2. **Efficient and Compact**
At 0.6B parameters, it is optimized for deployment in constrained environments while retaining high fidelity in logic, computation, and structural formatting.
3. **Magpie-Driven Data Synthesis**
Trained using 80% of **Magpie Pro 330k**—a high-quality alignment and reasoning dataset—complemented with curated modular datasets for enhanced general-purpose capabilities.
4. **Mathematical Precision**
Fine-tuned for arithmetic, algebra, calculus, and symbolic logic; ideal for STEM learning platforms, math solvers, and step-by-step tutoring.
5. **Lightweight Code Assistance**
Understands and generates code in Python, JavaScript, and other common languages with contextual accuracy and explanation support.
6. **Structured Output Generation**
Specializes in Markdown, JSON, and table outputs, suitable for technical documentation, instruction generation, and structured reasoning.
7. **Multilingual Competence**
Supports over 20 languages with reasoning and translation support, expanding its reach for global educational and development use.
---
## Quickstart with Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "prithivMLmods/Magpie-Qwen-CortexDual-0.6B"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "Write a Python function to check if a number is prime. Explain each step."
messages = [
{"role": "system", "content": "You are an AI tutor skilled in both math and code."},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
```
---
## Demo Inference
> [!warning]
non-thinking (direct, reactive, retrieval-based responses)
> [!warning]
thinking (reasoning, planning, deeper analysis)
---
## Intended Use
* General-purpose problem solving in math, logic, and code
* Interactive STEM tutoring and reasoning explanation
* Compact assistant for technical documentation and structured data tasks
* Multilingual applications with a focus on accurate technical reasoning
* Efficient offline deployment on low-resource devices
---
## Limitations
* Lower creativity and open-domain generation due to reasoning-focused tuning
* Limited context window size due to compact model size
* May produce simplified logic paths in highly abstract domains
* Trade-offs in diversity and expressiveness compared to larger instruction-tuned models
---
## References
1. [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://arxiv.org/pdf/2406.08464)
2. [Qwen2.5 Technical Report](https://arxiv.org/pdf/2412.15115)
3. [YaRN: Efficient Context Window Extension of Large Language Models](https://arxiv.org/pdf/2309.00071) |