Mixture of experts: beware 🛡️⚔️
text
Mixture of experts: beware 🛡️⚔️
new_line
new_line
New paper by DeepMind: Buffer Overflow in MoE 
text
New paper by DeepMind: Buffer Overflow in MoE 
Buffer Overflow in Mixture of Experts (2402.05526)
https://huggingface.co/papers/2402.05526
2402.05526
paper
resource
https://huggingface.co/papers/2402.05526
new_line
new_line
The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
text
The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
new_line
new_line
So if in the same batch we have 
text
So if in the same batch we have 
new_line
- User A benign query
text
- User A benign query
new_line
- User B malicious query
text
- User B malicious query
new_line
The response for A might be altered!😱
text
The response for A might be altered!😱
new_line
new_line
How is this possible?
text
How is this possible?
new_line
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
text
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
new_line
new_line
This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
text
This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
new_line
new_line
How to mitigate this?
text
How to mitigate this?
new_line
- Randomize batch order (and even run twice if some queries are very sensitive)
text
- Randomize batch order (and even run twice if some queries are very sensitive)
new_line
- Use a large capacity slack
text
- Use a large capacity slack
new_line
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
text
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
new_line
new_line
Very cool paper!!
text
Very cool paper!!
Mixture of experts: beware 🛡️⚔️

New paper by DeepMind: Buffer Overflow in MoE https://huggingface.co/papers/2402.05526

The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 

So if in the same batch we have 
- User A benign query
- User B malicious query
The response for A might be altered!😱

How is this possible?
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 

This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 

How to mitigate this?
- Randomize batch order (and even run twice if some queries are very sensitive)
- Use a large capacity slack
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)

Very cool paper!!
👍
NickyNicky julien-c macadeliccc ivanfioravanti clem samusenps victor JosephGesnouin pratjz blancsw marinaretik juancopi81 Dlbk djmcflush EmelMk surfiend Yasbok 0xe69756
❤️
macadeliccc clem samusenps JosephGesnouin blancsw santiviquez Dlbk abunchofrandomwords 0xe69756
🤯
meefs blancsw davanstrien Dlbk Yhyu13 Epiculous
🤝
Dlbk Epiculous
64438bcb1bc692d87b237c04
/avatars/19f9eeb9281b47b34f68c312092ca468.svg
Yu
Yhyu13
user
64adfd277b5ff762771e4571
https://cdn-avatars.huggingface.co/v1/production/uploads/64adfd277b5ff762771e4571/HVmAmGOtBPYd72492XGar.png
Epiculous
Epiculous
user