POLCA: Stochastic Generative Optimization with LLM
Optimizing complex systems, ranging from LLM prompts to multi-turn agents, traditionally requires labor-intensive manual iteration. We formalize this challenge as a stochastic generative optimization problem where a generative language model acts as the optimizer, guided by numerical rewards and text feedback to discover the best system. We introduce Prioritized Optimization with Local Contextual Aggregation (POLCA), a scalable framework designed to handle stochasticity in optimization -- such as noisy feedback, sampling minibatches, and stochastic system behaviors -- while effectively managing the unconstrained expansion of solution space. POLCA maintains a priority queue to manage the exploration-exploitation tradeoff, systematically tracking candidate solutions and their evaluation histories. To enhance efficiency, we integrate an varepsilon-Net mechanism to maintain parameter diversity and an LLM Summarizer to perform meta-learning across historical trials. We theoretically prove that POLCA converges to near-optimal candidate solutions under stochasticity. We evaluate our framework on diverse benchmarks, including τ-bench, HotpotQA (agent optimization), VeriBench (code translation) and KernelBench (CUDA kernel generation). Experimental results demonstrate that POLCA achieves robust, sample and time-efficient performance, consistently outperforming state-of-the-art algorithms in both deterministic and stochastic problems. The codebase for this work is publicly available at https://github.com/rlx-lab/POLCA.
69b9c6515d2b9c28ea09ef15
Xuanfei Ren
claimed_verified
2026-03-18T08:45:48.884Z
661f79477273a6cb227f3b30
/avatars/40aabf64de9755da04ef38a53b6a1894.svg
Xuanfei Ren
xuanfeiren
user
xuanfeiren
69b9c6515d2b9c28ea09ef16
Allen Nie
claimed_verified
2026-03-19T13:03:35.482Z
649466d90560480110a72247
/avatars/79fc060693328f4345c01adef4aba17c.svg
Allen Nie
allenanie
user
allenanie
69b9c6515d2b9c28ea09ef17
Tengyang Xie
claimed_verified
2026-03-24T09:23:11.614Z
63c343af70a076754b007cf5
/avatars/b4341786aeefdc4065af610a56c83266.svg
Tengyang Xie
tengyangx
user
tengyangx
69b9c6515d2b9c28ea09ef18
Ching-An Cheng
POLCA: Stochastic Generative Optimization with LLM
Optimizing complex systems, ranging from LLM prompts to multi-turn agents, traditionally requires labor-intensive manual iteration. We formalize this challenge as a stochastic generative optimization problem where a generative language model acts as the optimizer, guided by numerical rewards and text feedback to discover the best system. We introduce Prioritized Optimization with Local Contextual Aggregation (POLCA), a scalable framework designed to handle stochasticity in optimization -- such as noisy feedback, sampling minibatches, and stochastic system behaviors -- while effectively managing the unconstrained expansion of solution space. POLCA maintains a priority queue to manage the exploration-exploitation tradeoff, systematically tracking candidate solutions and their evaluation histories. To enhance efficiency, we integrate an varepsilon-Net mechanism to maintain parameter diversity and an LLM Summarizer to perform meta-learning across historical trials. We theoretically prove that POLCA converges to near-optimal candidate solutions under stochasticity. We evaluate our framework on diverse benchmarks, including τ-bench, HotpotQA (agent optimization), VeriBench (code translation) and KernelBench (CUDA kernel generation). Experimental results demonstrate that POLCA achieves robust, sample and time-efficient performance, consistently outperforming state-of-the-art algorithms in both deterministic and stochastic problems. The codebase for this work is publicly available at https://github.com/rlx-lab/POLCA.
A generative language model-based framework optimizes complex systems through stochastic generative optimization with prioritized exploration and contextual aggregation, achieving efficient convergence in both deterministic and stochastic environments.
generative language model stochastic generative optimization prioritized optimization local contextual aggregation priority queue ε-Net mechanism meta-learning convergence stochasticity exploration-exploitation tradeoff