Kevinmastascusa/qwen2p5-math-1p5b-merged

qwen2p5-math-1p5b-merged

Model summary

• Base: Qwen/Qwen2.5-Math-1.5B
• Type: Causal LM (decoder-only)
• Weights: LoRA adapters merged into full model weights
• Focus: math word problems (final numeric/text answers)
• Prompt formats:
  • GSM8K/SVAMP: “Question: …\nAnswer:”
  • MATH-500: “Problem: …\nSolution:”
  • MathQA: prepend “Options: …” when available

Intended use

• Math word problem solving and reasoning research
• Not for high-stakes decisions, formal proofs, or competition-level math

Training and data

• Objective: next-token causal LM
• Datasets (preprocessed): GSM8K, MathQA, SVAMP, MATH-500
• Fine-tuning via LoRA; merged to full weights for deployment

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "YOUR_ORG/qwen2p5-math-1p5b-merged"
tok = AutoTokenizer.from_pretrained(model_id)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else None,
)

def solve(q, max_new_tokens=64):
    prompt = f"Question: {q}\nAnswer:"
    inputs = tok(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs, max_new_tokens=max_new_tokens,
            do_sample=False, num_beams=1, pad_token_id=tok.eos_token_id
        )
    return tok.decode(out[0], skip_special_tokens=True)

Prompting tips

• GSM8K/SVAMP: “Question: …\nAnswer:”
• MATH-500: “Problem: …\nSolution:”
• MathQA: add “Options: …\nSolution:”

Limitations

• Final-answer formatting can affect correctness
• MathQA may require task-specific decoding to improve accuracy
• May hallucinate outside math tasks; use with caution

Hardware/latency

• ~1.5B parameters; GPU recommended for interactive use
• Typical decoding: 32–64 tokens

License

• Inherits the license terms of the base model (Qwen/Qwen2.5-Math-1.5B)

Citation

If you use this model, please cite the upstream Qwen2.5‑Math model and your own work as appropriate.

Acknowledgements

• Alibaba Qwen team for the Qwen2.5‑Math base
• GSM8K, MathQA, SVAMP, and MATH‑500 dataset authors