DeepSight: An All-in-One LM Safety Toolkit
As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) safety workflow, evaluation, diagnosis, and alignment are often handled by separate tools. Specifically, safety evaluation can only locate external behavioral risks but cannot figure out internal root causes. Meanwhile, safety diagnosis often drifts from concrete risk scenarios and remains at the explainable level. In this way, safety alignment lack dedicated explanations of changes in internal mechanisms, potentially degrading general capabilities. To systematically address these issues, we propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm. DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan. By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight. Besides, DeepSight is the first open source toolkit that support the frontier AI risk evaluation and joint safety evaluation and diagnosis.
698e9081cace060ff123ac81
Bo Zhang
698e9081cace060ff123ac82
Jiaxuan Guo
698e9081cace060ff123ac83
Lijun Li
698e9081cace060ff123ac84
Dongrui Liu
claimed_verified
2026-02-20T08:37:51.892Z
6621f4eb64e84619e578aad6
/avatars/b1ad96ee354b999fcafb2998a636609c.svg
Dongrui Liu
shenqiorient
user
shenqiorient
698e9081cace060ff123ac85
Sujin Chen
698e9081cace060ff123ac86
Guanxu Chen
698e9081cace060ff123ac87
Zhijie Zheng
698e9081cace060ff123ac88
Qihao Lin
698e9081cace060ff123ac89
Lewen Yan
698e9081cace060ff123ac8a
Chen Qian
698e9081cace060ff123ac8b
Yijin Zhou
698e9081cace060ff123ac8c
Yuyao Wu
698e9081cace060ff123ac8d
Shaoxiong Guo
698e9081cace060ff123ac8e
Tianyi Du
claimed_verified
2026-02-13T09:36:32.367Z
6552ed1886270cc7f8ca4885
/avatars/8f9f8a5dfbcab2d38490045925af9152.svg
dutianyi
coincidence2000
user
coincidence2000
698e9081cace060ff123ac8f
Jingyi Yang
698e9081cace060ff123ac90
Xuhao Hu
698e9081cace060ff123ac91
Ziqi Miao
698e9081cace060ff123ac92
Xiaoya Lu
698e9081cace060ff123ac93
Jing Shao
698e9081cace060ff123ac94
Xia Hu
DeepSight: An All-in-One LM Safety Toolkit
As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) safety workflow, evaluation, diagnosis, and alignment are often handled by separate tools. Specifically, safety evaluation can only locate external behavioral risks but cannot figure out internal root causes. Meanwhile, safety diagnosis often drifts from concrete risk scenarios and remains at the explainable level. In this way, safety alignment lack dedicated explanations of changes in internal mechanisms, potentially degrading general capabilities. To systematically address these issues, we propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm. DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan. By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight. Besides, DeepSight is the first open source toolkit that support the frontier AI risk evaluation and joint safety evaluation and diagnosis.
DeepSight is an open-source project that integrates safety evaluation and diagnosis for large language and multimodal models, enabling white-box insights through unified protocols and specialized toolkits.
Large Language Models Multimodal Large Language Models safety evaluation safety diagnosis safety alignment DeepSafe DeepScan