Taxonomy-Adaptive Moderation Model with Robust Guardrails for Large Language Models
Large Language Models (LLMs) are typically aligned for safety during the post-training phase; however, they may still generate inappropriate outputs that could potentially pose risks to users. This challenge underscores the need for robust safeguards that operate across both model inputs and outputs. In this work, we introduce Roblox Guard 1.0, a state-of-the-art instruction fine-tuned LLM designed to enhance the safety of LLM systems through comprehensive input-output moderation, using a pipeline of LLMs to enhance moderation capability. Built on the Llama-3.1-8B-Instruct backbone, our model is instruction fine-tuned to generalize across previously unseen safety taxonomies and demonstrates strong performance on out-of-domain safety benchmarks. The instruction fine-tuning process uses a mix of synthetic and open-source safety datasets, augmented with chain-of-thought (CoT) rationales and input inversion to enhance contextual understanding and decision making. To support systematic evaluation, we also release RobloxGuard-Eval, a new benchmark featuring an extensible safety taxonomy to assess the effectiveness of LLM guardrails and moderation frameworks.
69370e3a19d912300c34a0dc
Mahesh Kumar Nandwana
69370e3a19d912300c34a0dd
Youngwan Lim
69370e3a19d912300c34a0de
Joseph Liu
69370e3a19d912300c34a0df
Alex Yang
69370e3a19d912300c34a0e0
Varun Notibala
69370e3a19d912300c34a0e1
Nishchaie Khanna
Taxonomy-Adaptive Moderation Model with Robust Guardrails for Large Language Models
Large Language Models (LLMs) are typically aligned for safety during the post-training phase; however, they may still generate inappropriate outputs that could potentially pose risks to users. This challenge underscores the need for robust safeguards that operate across both model inputs and outputs. In this work, we introduce Roblox Guard 1.0, a state-of-the-art instruction fine-tuned LLM designed to enhance the safety of LLM systems through comprehensive input-output moderation, using a pipeline of LLMs to enhance moderation capability. Built on the Llama-3.1-8B-Instruct backbone, our model is instruction fine-tuned to generalize across previously unseen safety taxonomies and demonstrates strong performance on out-of-domain safety benchmarks. The instruction fine-tuning process uses a mix of synthetic and open-source safety datasets, augmented with chain-of-thought (CoT) rationales and input inversion to enhance contextual understanding and decision making. To support systematic evaluation, we also release RobloxGuard-Eval, a new benchmark featuring an extensible safety taxonomy to assess the effectiveness of LLM guardrails and moderation frameworks.
Roblox Guard 1.0 is an instruction fine-tuned LLM that enhances safety through comprehensive input-output moderation using a pipeline of LLMs and demonstrates strong performance on out-of-domain safety benchmarks.
instruction fine-tuned LLM safety taxonomies chain-of-thought input inversion contextual understanding RobloxGuard-Eval safety benchmarks