LSRIF: Logic-Structured Reinforcement Learning for Instruction Following
Instruction-following is critical for large language models, but real-world instructions often contain logical structures such as sequential dependencies and conditional branching. Existing methods typically construct datasets with parallel constraints and optimize average rewards, ignoring logical dependencies and yielding noisy signals. We propose a logic-structured training framework LSRIF that explicitly models instruction logic. We first construct a dataset LSRInstruct with constraint structures such as parallel, sequential, and conditional types, and then design structure-aware rewarding method LSRIF including average aggregation for parallel structures, failure-penalty propagation for sequential structures, and selective rewards for conditional branches. Experiments show LSRIF brings significant improvements in instruction-following (in-domain and out-of-domain) and general reasoning. Analysis reveals that learning with explicit logic structures brings parameter updates in attention layers and sharpens token-level attention to constraints and logical operators.
6969d8c032f0333869ff950e
Qingyu Ren
6969d8c032f0333869ff950f
Qianyu He
6969d8c032f0333869ff9510
Jingwen Chang
admin_assigned
2026-01-16T15:50:07.361Z
67ff4ecb58ed263257befc9e
/avatars/3b903411404e7c75d228afaaa88285ff.svg
JingwenChang
Chang992
user
Chang992
6969d8c032f0333869ff9511
Jie Zeng
6969d8c032f0333869ff9512
Jiaqing Liang
admin_assigned
2026-01-16T15:50:27.203Z
634014022c72fd4ca251fed7
/avatars/32a85d41f7435f7facfd580d9d124c33.svg
liangjiaqing
liangjiaqing
user
liangjiaqing
6969d8c032f0333869ff9513
Yanghua Xiao
6969d8c032f0333869ff9514
Han Xia
6969d8c032f0333869ff9515
Zeye Sun
admin_assigned
2026-01-16T15:50:42.120Z
635f79e9514c057dd4db5b27
/avatars/ba758201e9ff70a020899598c3b9d6f4.svg
zeye sun
sunzeyeah
user
sunzeyeah
6969d8c032f0333869ff9516
Fei Yu
LSRIF: Logic-Structured Reinforcement Learning for Instruction Following
Instruction-following is critical for large language models, but real-world instructions often contain logical structures such as sequential dependencies and conditional branching. Existing methods typically construct datasets with parallel constraints and optimize average rewards, ignoring logical dependencies and yielding noisy signals. We propose a logic-structured training framework LSRIF that explicitly models instruction logic. We first construct a dataset LSRInstruct with constraint structures such as parallel, sequential, and conditional types, and then design structure-aware rewarding method LSRIF including average aggregation for parallel structures, failure-penalty propagation for sequential structures, and selective rewards for conditional branches. Experiments show LSRIF brings significant improvements in instruction-following (in-domain and out-of-domain) and general reasoning. Analysis reveals that learning with explicit logic structures brings parameter updates in attention layers and sharpens token-level attention to constraints and logical operators.
A logic-structured training framework explicitly models instruction logic through constraint-aware reward mechanisms, improving instruction-following and reasoning capabilities in large language models.
instruction-following logical structures sequential dependencies conditional branching LSRInstruct LSRIF structure-aware rewarding parallel structures sequential structures conditional branches attention layers token-level attention