POS-ISP: Pipeline Optimization at the Sequence Level for Task-aware ISP
Recent work has explored optimizing image signal processing (ISP) pipelines for various tasks by composing predefined modules and adapting them to task-specific objectives. However, jointly optimizing module sequences and parameters remains challenging. Existing approaches rely on neural architecture search (NAS) or step-wise reinforcement learning (RL), but NAS suffers from a training-inference mismatch, while step-wise RL leads to unstable training and high computational overhead due to stage-wise decision-making. We propose POS-ISP, a sequence-level RL framework that formulates modular ISP optimization as a global sequence prediction problem. Our method predicts the entire module sequence and its parameters in a single forward pass and optimizes the pipeline using a terminal task reward, eliminating the need for intermediate supervision and redundant executions. Experiments across multiple downstream tasks show that POS-ISP improves task performance while reducing computational cost, highlighting sequence-level optimization as a stable and efficient paradigm for task-aware ISP. The project page is available at https://w1jyun.github.io/POS-ISP
69d79c90e33e601f8dfcd639
Jiyun Won
claimed_verified
2026-04-09T18:50:38.124Z
69d79bb236aa554bc01990b8
/avatars/5754841e0fc0d4bdf3cf1fec4f290c87.svg
Jiyun Won
jiyunwon
user
jiyunwon
69d79c90e33e601f8dfcd63a
Heemin Yang
69d79c90e33e601f8dfcd63b
Woohyeok Kim
69d79c90e33e601f8dfcd63c
Jungseul Ok
69d79c90e33e601f8dfcd63d
Sunghyun Cho
POS-ISP: Pipeline Optimization at the Sequence Level for Task-aware ISP
Recent work has explored optimizing image signal processing (ISP) pipelines for various tasks by composing predefined modules and adapting them to task-specific objectives. However, jointly optimizing module sequences and parameters remains challenging. Existing approaches rely on neural architecture search (NAS) or step-wise reinforcement learning (RL), but NAS suffers from a training-inference mismatch, while step-wise RL leads to unstable training and high computational overhead due to stage-wise decision-making. We propose POS-ISP, a sequence-level RL framework that formulates modular ISP optimization as a global sequence prediction problem. Our method predicts the entire module sequence and its parameters in a single forward pass and optimizes the pipeline using a terminal task reward, eliminating the need for intermediate supervision and redundant executions. Experiments across multiple downstream tasks show that POS-ISP improves task performance while reducing computational cost, highlighting sequence-level optimization as a stable and efficient paradigm for task-aware ISP. The project page is available at https://w1jyun.github.io/POS-ISP
POS-ISP presents a sequence-level reinforcement learning framework for optimizing image signal processing pipelines by predicting complete module sequences and parameters in a single forward pass, improving task performance while reducing computational costs.
neural architecture search reinforcement learning image signal processing module sequences task-specific optimization terminal task reward forward pass computational overhead