🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by 
text
🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by 
@stecas
mention
stecas
text
@carsonezell
mention
carsonezell
 et al.
text
 et al.
new_line
new_line
Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.
text
Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.
new_line
new_line
📄 Paper: 
text
📄 Paper: 
Black-Box Access is Insufficient for Rigorous AI Audits (2401.14446)
https://huggingface.co/papers/2401.14446
2401.14446
paper
resource
https://huggingface.co/papers/2401.14446
new_line
new_line
🔍 Further readings:
text
🔍 Further readings:
new_line
new_line
📄Taxonomy of AI system access: 
text
📄Taxonomy of AI system access: 
https://bit.ly/struct-access
https://bit.ly/struct-access
link
new_line
💻An API for transparent science on Black-box AI (NNsight): 
text
💻An API for transparent science on Black-box AI (NNsight): 
https://nnsight.net/about
https://nnsight.net/about
link
🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by @stecas @carsonezell et al.

Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.

📄 Paper: https://huggingface.co/papers/2401.14446

🔍 Further readings:

📄Taxonomy of AI system access: https://bit.ly/struct-access
💻An API for transparent science on Black-box AI (NNsight): https://nnsight.net/about
❤️
clem skrishna osanseviero merve stecas
👍
dhuynh95 skrishna osanseviero stecas