File size: 1,578 Bytes
0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 0a2552d 1951d49 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
license: apache-2.0
base_model: openai/gpt-oss-20b
library_name: transformers
pipeline_tag: text-generation
tags:
- peft
- lora
- bigcodebench
- gpt-oss
- code
- causal-lm
inference: false
---
# GPT-OSS-20B BigCodeBench LoRA Adapter
LoRA adapter weights fine-tuned from `openai/gpt-oss-20b` on BigCodeBench split `v0.1.4` (~1.1K samples).
## Training Summary
- Steps: 100
- Final train_loss: 0.7833267974853516
- Runtime (s): 3717.3139
- Samples/sec: 0.43
- Total FLOPs: 6.825417425085542e+16
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = 'openai/gpt-oss-20b'
adapter = 'unlimitedbytes/gptoss-bigcodebench-20b-lora'
model = AutoModelForCausalLM.from_pretrained(base, device_map='auto', torch_dtype='auto')
model = PeftModel.from_pretrained(model, adapter)
tokenizer = AutoTokenizer.from_pretrained(base)
messages = [
{'role': 'system', 'content': 'You are a helpful coding assistant.'},
{'role': 'user', 'content': 'Write a Python function to add two numbers.'}
]
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors='pt').to(model.device)
out = model.generate(input_ids, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=False))
```
Merge adapter:
```python
model = model.merge_and_unload()
model.save_pretrained('merged-model')
```
## Limitations
- 100 training steps only; not fully converged.
- Adapter only, no merged full weights.
- Outputs may include control tokens.
## License
Apache-2.0 (base) + dataset licenses.
|