Wisent-Qwen2.5-Coder-7B-Instruct with CAA Steering

Model Description

This is an enhanced version of Qwen2.5-Coder-7B-Instruct that integrates Contrastive Activation Addition (CAA) steering directly into the model architecture. The steering parameters have been optimized using Optuna to improve code generation quality on the MBPP Plus benchmark.

Key Features

🚀 Automatic CAA Steering: No manual hook management required
🎯 Optimized Parameters: Layer 24, α=0.9
🗂️ Trait-Based Organization: Steering vectors organized by traits
🔧 Runtime Configurable: Adjust or disable steering on the fly
🤗 HuggingFace Compatible: Works with standard transformers API

Installation

pip install transformers torch

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model - CAA steering is automatically applied!
model = AutoModelForCausalLM.from_pretrained("./huggingface_qwen_generated", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("./huggingface_qwen_generated")

# Generate code
prompt = "Write a Python function to calculate the factorial of a number"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Advanced Usage

Adjusting Steering Strength

# Increase steering strength for stronger safety alignment
model.set_caa_alpha(1.2)

# Decrease for more creative outputs
model.set_caa_alpha(0.5)

Disabling CAA Steering

# Disable CAA to get baseline model behavior
model.set_caa_enabled(False)

# Re-enable CAA
model.set_caa_enabled(True)

Accessing Steering Configuration

print(f"CAA Layer: {model.caa_layer_id}")
print(f"CAA Alpha: {model.caa_alpha}")
print(f"Steering Method: {model.steering_method}")

Trait-Based Vector Organization

The model uses a trait-based organization for steering vectors:

vectors/
├── coding/         # Current: Optimized for code generation
├── safety/         # Future: Safety-aligned behavior
├── creativity/     # Future: Enhanced creative outputs  
├── helpfulness/    # Future: Improved helpfulness
└── reasoning/      # Future: Enhanced logical reasoning

To switch traits, simply update the configuration:

{
  "steering_vector_path": "./vectors/safety/steering_vector.safetensors"
}

Technical Details

CAA Steering Parameters

Steering Method: Contrastive Activation Addition (CAA)
Optimal Layer: 24 (out of 28 transformer layers)
Steering Strength (α): 0.9
Vector Format: Safetensors format for efficient loading and HuggingFace compatibility
Vector Dimension: 3584 (pre-normalized during training)
Storage Path: ./vectors/coding/steering_vector.safetensors

How It Works

Trait-based Organization: Steering vectors are organized by behavioral traits (vectors/{trait}/)
Dynamic Loading: The model loads the specified steering vector from the configured path
Layer Application: Steering is applied to hidden states at layer 24 during forward pass
Generation Integration: Steering affects the last token position during generation
Configurable Strength: The α parameter (default: 0.9) controls steering intensity
Pre-optimized Vectors: Steering vectors are pre-normalized and ready for immediate use

Optimization Process

The CAA parameters were optimized using:

Framework: Optuna with TPE sampler
Search Space: Layers 15-28, α ∈ [0.1, 5.0]
Objective: Maximize accuracy on MBPP Plus validation set
Best Validation Score: 64% accuracy

Model Architecture

WisentQwen2ForCausalLM
├── Base: Qwen2.5-Coder-7B-Instruct
├── CAA Integration: Layer 24
├── Steering Vector: ./vectors/coding/steering_vector.safetensors
└── Auto-applied during generation

File Structure

huggingface_qwen_generated/
├── config.json                    # Model configuration with CAA params
├── modeling_wisent_qwen.py        # Custom model class
├── tokenizer files               # Standard Qwen tokenizer
├── wisent_config.json            # Optimization results
└── vectors/                       # Trait-based steering vectors
    └── coding/
        └── steering_vector.safetensors  # Optimized coding steering vector

Evaluation

MBPP Plus Benchmark

The model should be evaluated on the complete MBPP Plus dataset (378 problems) to measure improvement over the baseline. Expected improvements based on validation results.

Running Evaluation

# Use with bigcode-evaluation-harness
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "./huggingface_qwen_generated",
    trust_remote_code=True
)

# CAA steering is automatically applied during evaluation!
# No manual hooks or modifications needed

Citation

If you use this model, please cite:

@software{wisent_qwen_caa_2025,
  title={Wisent-Qwen2.5-Coder with CAA Steering},
  author={Wisent AI},
  year={2025},
  url={https://github.com/wisent-ai/wisent-guard}
}

License

This model inherits the license from the base Qwen2.5-Coder-7B-Instruct model. Please refer to the original model's license for usage terms.

Acknowledgments

Base model: Qwen2.5-Coder-7B-Instruct by Alibaba
CAA method: Contrastive Activation Addition
Optimization: Optuna framework
Implementation: Wisent Guard framework