Anthropic introduces "Many-shot Jailbreaking" (MSJ), a new attack on large language models! MSJ exploits long context windows to override safety constraints.
text
Anthropic introduces "Many-shot Jailbreaking" (MSJ), a new attack on large language models! MSJ exploits long context windows to override safety constraints.
new_line
new_line
Key Points:
text
Key Points:
new_line
* Prompts LLMs with hundreds of examples of harmful behavior formatted as a dialogue
text
* Prompts LLMs with hundreds of examples of harmful behavior formatted as a dialogue
new_line
* Generates malicious examples using an uninhibited "helpful-only" model
text
* Generates malicious examples using an uninhibited "helpful-only" model
new_line
* Effective at jailbreaking models like Claude 2.0, GPT-3.5, GPT-4
text
* Effective at jailbreaking models like Claude 2.0, GPT-3.5, GPT-4
new_line
* Standard alignment techniques provide limited protection against long context attacks
text
* Standard alignment techniques provide limited protection against long context attacks
new_line
new_line
Paper: 
text
Paper: 
https://www.anthropic.com/research/many-shot-jailbreaking
https://www.anthropic.com/research/many-shot-jailbreaking
link
new_line
More details in my blog: 
text
More details in my blog: 
https://huggingface.co/blog/vladbogo/many-shot-jailbreaking
https://huggingface.co/blog/vladbogo/many-shot-jailbreaking
link
new_line
new_line
Congrats to the authors for their work!
text
Congrats to the authors for their work!
Anthropic introduces "Many-shot Jailbreaking" (MSJ), a new attack on large language models! MSJ exploits long context windows to override safety constraints.

Key Points:
* Prompts LLMs with hundreds of examples of harmful behavior formatted as a dialogue
* Generates malicious examples using an uninhibited "helpful-only" model
* Effective at jailbreaking models like Claude 2.0, GPT-3.5, GPT-4
* Standard alignment techniques provide limited protection against long context attacks

Paper: https://www.anthropic.com/research/many-shot-jailbreaking
More details in my blog: https://huggingface.co/blog/vladbogo/many-shot-jailbreaking

Congrats to the authors for their work!
❤️
samusenps nikgr gruber sauravssss
👀
monsoon-nlp gruber
634262af8d8089ebaefd410e
https://cdn-avatars.huggingface.co/v1/production/uploads/634262af8d8089ebaefd410e/pcnqe74uMV90K3HVuM76F.png
Fizz 🏳️‍⚧️
Fizzarolli
user
64fffae82ad36636be7f3c94
/avatars/eac1b6903a19bf24057e914b03796f46.svg
Saurav singh
sauravssss
user