metadata

library_name: transformers
license: mit
base_model: roberta-base
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: vulnerability-severity-classification-roberta-base
    results: []
datasets:
  - CIRCL/vulnerability-scores

VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification

Severity classification

This model is a fine-tuned version of roberta-base on the dataset CIRCL/vulnerability-scores.

The model was presented in the paper VLAI: A RoBERTa-Based Model for Automated Vulnerability Severity Classification [arXiv].

Abstract: VLAI is a transformer-based model that predicts software vulnerability severity levels directly from text descriptions. Built on RoBERTa, VLAI is fine-tuned on over 600,000 real-world vulnerabilities and achieves over 82% accuracy in predicting severity categories, enabling faster and more consistent triage ahead of manual CVSS scoring. The model and dataset are open-source and integrated into the Vulnerability-Lookup service.

You can read this page for more information.

Model description

It is a classification model and is aimed to assist in classifying vulnerabilities by severity based on their descriptions.

It achieves the following results on the evaluation set:

Loss: 0.5093
Accuracy: 0.8249

How to get started with the model

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

labels = ["low", "medium", "high", "critical"]

model_name = "CIRCL/vulnerability-severity-classification-roberta-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

test_description = "SAP NetWeaver Visual Composer Metadata Uploader is not protected with a proper authorization, allowing unauthenticated agent to upload potentially malicious executable binaries \
that could severely harm the host system. This could significantly affect the confidentiality, integrity, and availability of the targeted system."
inputs = tokenizer(test_description, return_tensors="pt", truncation=True, padding=True)

# Run inference
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)

# Print results
print("Predictions:", predictions)
predicted_class = torch.argmax(predictions, dim=-1).item()
print("Predicted severity:", labels[predicted_class])

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6783	1.0	28699	0.6468	0.7397
0.6438	2.0	57398	0.5705	0.7716
0.5289	3.0	86097	0.5305	0.7943
0.4965	4.0	114796	0.5140	0.8150
0.4104	5.0	143495	0.5093	0.8249

Framework versions

Transformers 4.55.2
Pytorch 2.8.0+cu128
Datasets 4.0.0
Tokenizers 0.21.4