🔍 Today's pick in Interpretability & Analysis of LMs: x2 edition!
text
🔍 Today's pick in Interpretability & Analysis of LMs: x2 edition!
new_line
new_line
Today's highlighted works aim reproduce findings from Transformer-centric interpretability literature on new RNN-based architectures such as Mamba and RWKV:
text
Today's highlighted works aim reproduce findings from Transformer-centric interpretability literature on new RNN-based architectures such as Mamba and RWKV:
new_line
new_line
Does Transformer Interpretability Transfer to RNNs? (2404.05971)
https://huggingface.co/papers/2404.05971
2404.05971
paper
resource
https://huggingface.co/papers/2404.05971
 by 
text
 by 
@MrGonao
mention
MrGonao
 T. Marshall 
text
 T. Marshall 
@norabelrose
mention
norabelrose
new_line
new_line
Locating and Editing Factual Associations in Mamba (2404.03646)
https://huggingface.co/papers/2404.03646
2404.03646
paper
resource
https://huggingface.co/papers/2404.03646
  by 
text
  by 
@sensharma
mention
sensharma
text
@datkinson
mention
datkinson
text
@davidbau
mention
davidbau
new_line
new_line
The first paper applies contrastive activation addition, the tuned lens and probing for eliciting latent knowledge in quirky models to Mamba and RWKV LMs, finding these Transformer-specific methods can be applied with slight adaptation to these architectures, obtaining similar results.
text
The first paper applies contrastive activation addition, the tuned lens and probing for eliciting latent knowledge in quirky models to Mamba and RWKV LMs, finding these Transformer-specific methods can be applied with slight adaptation to these architectures, obtaining similar results.
new_line
new_line
The second work applies the ROME method to Mamba, finding weights playing the role of MLPs in encoding factual relations across several Mamba layers, and can be patched to perform model editing. A new SSM-specific technique is also introduced to emulate attention knockout (value zeroing) revealing information flows similar to the ones in Transformers when processing factual statements.
text
The second work applies the ROME method to Mamba, finding weights playing the role of MLPs in encoding factual relations across several Mamba layers, and can be patched to perform model editing. A new SSM-specific technique is also introduced to emulate attention knockout (value zeroing) revealing information flows similar to the ones in Transformers when processing factual statements.
new_line
new_line
💻 Code: 
text
💻 Code: 
https://github.com/arnab-api/romba
https://github.com/arnab-api/romba
link
new_line
new_line
🔍 All daily picks: 
text
🔍 All daily picks: 
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
link
🔍 Today's pick in Interpretability & Analysis of LMs: x2 edition!

Today's highlighted works aim reproduce findings from Transformer-centric interpretability literature on new RNN-based architectures such as Mamba and RWKV:

https://huggingface.co/papers/2404.05971 by @MrGonao T. Marshall @norabelrose

https://huggingface.co/papers/2404.03646  by @sensharma @datkinson @davidbau

The first paper applies contrastive activation addition, the tuned lens and probing for eliciting latent knowledge in quirky models to Mamba and RWKV LMs, finding these Transformer-specific methods can be applied with slight adaptation to these architectures, obtaining similar results.

The second work applies the ROME method to Mamba, finding weights playing the role of MLPs in encoding factual relations across several Mamba layers, and can be patched to perform model editing. A new SSM-specific technique is also introduced to emulate attention knockout (value zeroing) revealing information flows similar to the ones in Transformers when processing factual statements.

💻 Code: https://github.com/arnab-api/romba

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
❤️
samusenps osanseviero clem datkinson
🔥
samusenps osanseviero clem datkinson