Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling
Large Language Models (LLMs) are typically evaluated for safety under single-shot or low-budget adversarial prompting, which underestimates real-world risk. In practice, attackers can exploit large-scale parallel sampling to repeatedly probe a model until a harmful response is produced. While recent work shows that attack success increases with repeated sampling, principled methods for predicting large-scale adversarial risk remain limited. We propose a scaling-aware Best-of-N estimation of risk, SABER, for modeling jailbreak vulnerability under Best-of-N sampling. We model sample-level success probabilities using a Beta distribution, the conjugate prior of the Bernoulli distribution, and derive an analytic scaling law that enables reliable extrapolation of large-N attack success rates from small-budget measurements. Using only n=100 samples, our anchored estimator predicts ASR@1000 with a mean absolute error of 1.66, compared to 12.04 for the baseline, which is an 86.2% reduction in estimation error. Our results reveal heterogeneous risk scaling profiles and show that models appearing robust under standard evaluation can experience rapid nonlinear risk amplification under parallel adversarial pressure. This work provides a low-cost, scalable methodology for realistic LLM safety assessment. We will release our code and evaluation scripts upon publication to future research.
6980426e6676f93322706680
Mingqian Feng
6980426e6676f93322706681
Xiaodong Liu
claimed_verified
2026-02-20T08:38:07.488Z
674f57bec0127abc80173bcb
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/00U0ZND5vlBqks6MYN1Tu.png
Liu
AllenMS
user
AllenMS
6980426e6676f93322706682
Weiwei Yang
6980426e6676f93322706683
Chenliang Xu
6980426e6676f93322706684
Christopher White
6980426e6676f93322706685
Jianfeng Gao
Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling
Large Language Models (LLMs) are typically evaluated for safety under single-shot or low-budget adversarial prompting, which underestimates real-world risk. In practice, attackers can exploit large-scale parallel sampling to repeatedly probe a model until a harmful response is produced. While recent work shows that attack success increases with repeated sampling, principled methods for predicting large-scale adversarial risk remain limited. We propose a scaling-aware Best-of-N estimation of risk, SABER, for modeling jailbreak vulnerability under Best-of-N sampling. We model sample-level success probabilities using a Beta distribution, the conjugate prior of the Bernoulli distribution, and derive an analytic scaling law that enables reliable extrapolation of large-N attack success rates from small-budget measurements. Using only n=100 samples, our anchored estimator predicts ASR@1000 with a mean absolute error of 1.66, compared to 12.04 for the baseline, which is an 86.2% reduction in estimation error. Our results reveal heterogeneous risk scaling profiles and show that models appearing robust under standard evaluation can experience rapid nonlinear risk amplification under parallel adversarial pressure. This work provides a low-cost, scalable methodology for realistic LLM safety assessment. We will release our code and evaluation scripts upon publication to future research.
A scaling-aware risk estimation method called SABER is introduced for predicting large-scale adversarial vulnerability in language models through Best-of-N sampling, enabling accurate assessment with reduced computational costs.
large language models adversarial prompting Best-of-N sampling jailbreak vulnerability Beta distribution scaling law attack success rate parallel sampling risk estimation