EmbTracker: Traceable Black-box Watermarking for Federated Language Models
Federated Language Model (FedLM) allows a collaborative learning without sharing raw data, yet it introduces a critical vulnerability, as every untrustworthy client may leak the received functional model instance. Current watermarking schemes for FedLM often require white-box access and client-side cooperation, providing only group-level proof of ownership rather than individual traceability. We propose EmbTracker, a server-side, traceable black-box watermarking framework specifically designed for FedLMs. EmbTracker achieves black-box verifiability by embedding a backdoor-based watermark detectable through simple API queries. Client-level traceability is realized by injecting unique identity-specific watermarks into the model distributed to each client. In this way, a leaked model can be attributed to a specific culprit, ensuring robustness even against non-cooperative participants. Extensive experiments on various language and vision-language models demonstrate that EmbTracker achieves robust traceability with verification rates near 100\%, high resilience against removal attacks (fine-tuning, pruning, quantization), and negligible impact on primary task performance (typically within 1-2\%).
69b3837a2ad6c5d4cf486aef
Haodong Zhao
69b3837a2ad6c5d4cf486af0
Jinming Hu
69b3837a2ad6c5d4cf486af1
Yijie Bai
69b3837a2ad6c5d4cf486af2
Tian Dong
69b3837a2ad6c5d4cf486af3
Wei Du
69b3837a2ad6c5d4cf486af4
Zhuosheng Zhang
69b3837a2ad6c5d4cf486af5
Yanjiao Chen
69b3837a2ad6c5d4cf486af6
Haojin Zhu
69b3837a2ad6c5d4cf486af7
Gongshen Liu
EmbTracker: Traceable Black-box Watermarking for Federated Language Models
Federated Language Model (FedLM) allows a collaborative learning without sharing raw data, yet it introduces a critical vulnerability, as every untrustworthy client may leak the received functional model instance. Current watermarking schemes for FedLM often require white-box access and client-side cooperation, providing only group-level proof of ownership rather than individual traceability. We propose EmbTracker, a server-side, traceable black-box watermarking framework specifically designed for FedLMs. EmbTracker achieves black-box verifiability by embedding a backdoor-based watermark detectable through simple API queries. Client-level traceability is realized by injecting unique identity-specific watermarks into the model distributed to each client. In this way, a leaked model can be attributed to a specific culprit, ensuring robustness even against non-cooperative participants. Extensive experiments on various language and vision-language models demonstrate that EmbTracker achieves robust traceability with verification rates near 100\%, high resilience against removal attacks (fine-tuning, pruning, quantization), and negligible impact on primary task performance (typically within 1-2\%).
EmbTracker is a server-side black-box watermarking framework for federated language models that provides client-level traceability through unique identity-specific watermarks embedded via backdoor detection mechanisms.
Federated Language Model watermarking black-box verification backdoor-based watermark traceability client-level attribution model watermark embedding robustness removal attacks fine-tuning pruning quantization