SEAR: Schema-Based Evaluation and Routing for LLM Gateways
Evaluating production LLM responses and routing requests across providers in LLM gateways requires fine-grained quality signals and operationally grounded decisions. To address this gap, we present SEAR, a schema-based evaluation and routing system for multi-model, multi-provider LLM gateways. SEAR defines an extensible relational schema covering both LLM evaluation signals (context, intent, response characteristics, issue attribution, and quality scores) and gateway operational metrics (latency, cost, throughput), with cross-table consistency links across around one hundred typed, SQL-queryable columns. To populate the evaluation signals reliably, SEAR proposes self-contained signal instructions, in-schema reasoning, and multi-stage generation that produces database-ready structured outputs. Because signals are derived through LLM reasoning rather than shallow classifiers, SEAR captures complex request semantics, enables human-interpretable routing explanations, and unifies evaluation and routing in a single query layer. Across thousands of production sessions, SEAR achieves strong signal accuracy on human-labeled data and supports practical routing decisions, including large cost reductions with comparable quality.
69cb8c8922f9e9dce3fe22d5
Zecheng Zhang
claimed_verified
2026-03-31T09:23:37.791Z
609a5543456ee4a3392e8427
https://cdn-avatars.huggingface.co/v1/production/uploads/1620727835835-609a5543456ee4a3392e8427.jpeg
Zecheng Zhang
zechengz
user
zechengz
69cb8c8922f9e9dce3fe22d6
Han Zheng
69cb8c8922f9e9dce3fe22d7
Yue Xu
SEAR: Schema-Based Evaluation and Routing for LLM Gateways
Evaluating production LLM responses and routing requests across providers in LLM gateways requires fine-grained quality signals and operationally grounded decisions. To address this gap, we present SEAR, a schema-based evaluation and routing system for multi-model, multi-provider LLM gateways. SEAR defines an extensible relational schema covering both LLM evaluation signals (context, intent, response characteristics, issue attribution, and quality scores) and gateway operational metrics (latency, cost, throughput), with cross-table consistency links across around one hundred typed, SQL-queryable columns. To populate the evaluation signals reliably, SEAR proposes self-contained signal instructions, in-schema reasoning, and multi-stage generation that produces database-ready structured outputs. Because signals are derived through LLM reasoning rather than shallow classifiers, SEAR captures complex request semantics, enables human-interpretable routing explanations, and unifies evaluation and routing in a single query layer. Across thousands of production sessions, SEAR achieves strong signal accuracy on human-labeled data and supports practical routing decisions, including large cost reductions with comparable quality.
SEAR is a schema-based system for evaluating and routing LLM responses that uses structured signals derived from LLM reasoning to enable accurate, interpretable routing decisions across multiple providers.
schema-based evaluation multi-model multi-provider LLM gateways LLM reasoning structured outputs routing decisions operational metrics latency cost throughput cross-table consistency SQL-queryable columns signal accuracy