eromang/cyberscale-scorer-v1

CyberScale Scorer v1

Vulnerability severity scorer (0-10) based on ModernBERT-base. Predicts CVSS-compatible severity scores from vulnerability descriptions, with Monte Carlo dropout confidence estimation.

Model Description

• Architecture: ModernBERT-base with regression head (sigmoid x 10)
• Training: Huber loss with boundary-aware weighting, early stopping
• Confidence: Monte Carlo dropout (20 forward passes) maps variance to high/medium/low
• Post-hoc calibration: Predictions near band boundaries (4.0, 7.0, 9.0) are nudged away to reduce band-flip errors

Intended Use

Score vulnerability severity from text descriptions when authoritative CVSS scores are unavailable. Part of the CyberScale multi-phase cyber severity assessment system.

Input format: <description> [SEP] cwe: <CWE-ID> (CWE optional)

Training Data

• Source: cvelistV5 (CVE.org), quality-filtered and deduplicated
• Size: 12,000 CVEs (3,000 per CVSS band)
• CVSS version: 88% v3.1, 12% v3.0
• Selection: Boundary-enriched sampling (33% from +/-1.0 of band edges)
• Quality filters: RESERVED/REJECTED rejection, min 10 tokens, SHA-256 description dedup

Metrics

Metric	Value	Target
MAE	N/A	< 1.0
RMSE	N/A	-
Pearson r	N/A	-
Band Accuracy	N/A	> 0.75

CVSS Bands

Band	Range
Critical	9.0 - 10.0
High	7.0 - 8.9
Medium	4.0 - 6.9
Low	0.0 - 3.9

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained("eromang/cyberscale-scorer-v1", num_labels=1)
tokenizer = AutoTokenizer.from_pretrained("eromang/cyberscale-scorer-v1")

text = "Buffer overflow in libpng allows remote code execution via crafted PNG file [SEP] cwe: CWE-119"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=192)

with torch.no_grad():
    score = torch.sigmoid(model(**inputs).logits).item() * 10.0
    print(f"Severity: {score:.1f}/10")

Limitations

• Trained on English descriptions only
• Band accuracy (~70%) limited by regression-to-classification boundary effects
• Confidence estimation requires dropout layers (set classifier_dropout > 0)

Citation

Part of the CyberScale project — multi-phase cyber severity assessment MCP server.