Building Production-Ready Probes For Gemini
Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architecture that handle this long-context distribution shift.
  We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant shifts, including multi-turn conversations, static jailbreaks, and adaptive red teaming. Our results demonstrate that while multimax addresses context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes.
  These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.
696d99c23f1837bfb897097f
János Kramár
admin_assigned
2026-01-19T14:45:38.688Z
6647488b45ee22196e798ba6
/avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
Janos Kramar
jkramar
user
jkramar
696d99c23f1837bfb8970980
Joshua Engels
696d99c23f1837bfb8970981
Zheng Wang
696d99c23f1837bfb8970982
Bilal Chughtai
admin_assigned
2026-01-19T14:45:47.795Z
64ad563f4beffa272de6efac
/avatars/f1a4902a95830cc3936058449626f8e4.svg
Bilal Chughtai
bilalchughtai
user
bilalchughtai
696d99c23f1837bfb8970983
Rohin Shah
696d99c23f1837bfb8970984
Neel Nanda
696d99c23f1837bfb8970985
Arthur Conmy
Building Production-Ready Probes For Gemini
Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architecture that handle this long-context distribution shift.
  We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant shifts, including multi-turn conversations, static jailbreaks, and adaptive red teaming. Our results demonstrate that while multimax addresses context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes.
  These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.
Activation probes for language model misuse mitigation face challenges with long-context generalization, requiring new architectures and diverse training for robust performance across production shifts.
activation probes language model misuse mitigation context length probe architecture cyber-offensive domain multi-turn conversations jailbreaks red teaming AlphaEvolve automated AI safety research