QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation
Large Language Models (LLMs) are increasingly used for code generation, yet quantum code generation is still evaluated mostly within single frameworks, making it difficult to separate quantum reasoning from framework familiarity. We introduce QuanBench+, a unified benchmark spanning Qiskit, PennyLane, and Cirq, with 42 aligned tasks covering quantum algorithms, gate decomposition, and state preparation.
  We evaluate models with executable functional tests, report Pass@1 and Pass@5, and use KL-divergence-based acceptance for probabilistic outputs. We additionally study Pass@1 after feedback-based repair, where a model may revise code after a runtime error or wrong answer. Across frameworks, the strongest one-shot scores reach 59.5% in Qiskit, 54.8% in Cirq, and 42.9% in PennyLane; with feedback-based repair, the best scores rise to 83.3%, 76.2%, and 66.7%, respectively. These results show clear progress, but also that reliable multi-framework quantum code generation remains unsolved and still depends strongly on framework-specific knowledge.
69dcf80865e6d8cd9cd0ffe7
Ali Slim
claimed_verified
2026-04-13T19:23:46.792Z
69dcfd8679e0e55a4ced6949
/avatars/fc08c5a64c91468c83d50dcc2e1e63b0.svg
Ali Slim
ali21slim
user
ali21slim
69dcf80865e6d8cd9cd0ffe8
Haydar Hamieh
claimed_verified
2026-04-13T19:23:50.103Z
69dcf9dc8c724cfaffa1059b
/avatars/5f39d10a155514a07ead7cd1aabe61da.svg
Haydar Hamieh
haydarhamieh
user
haydarhamieh
69dcf80865e6d8cd9cd0ffe9
Jawad Kotaich
claimed_verified
2026-04-13T19:23:53.186Z
69dcf55dcbf3a92504ee4e4a
/avatars/4f21c12d00f32f4b0bd0f82ddfe0fabf.svg
Jawad Kotaich
Jawadkotaich
user
Jawadkotaich
69dcf80865e6d8cd9cd0ffea
Yehya Ghosn
claimed_verified
2026-04-13T19:23:44.102Z
691e1bc7e4dedcb160e51a2e
/avatars/ac32bbeabb30095edef8295f8aef6ecf.svg
Yehya Ghosn
YehyaGhosn
user
YehyaGhosn
69dcf80865e6d8cd9cd0ffeb
Mahdi Chehimi
69dcf80865e6d8cd9cd0ffec
Ammar Mohanna
claimed_verified
2026-04-15T12:08:29.287Z
61f01168943fd4875cd86724
/avatars/415cc5388c16e2527d286bc56faeb810.svg
Ammar Mohanna
AmmarMohanna
user
AmmarMohanna
69dcf80865e6d8cd9cd0ffed
Hasan Abed Al Kader Hammoud
69dcf80865e6d8cd9cd0ffee
Bernard Ghanem
QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation
Large Language Models (LLMs) are increasingly used for code generation, yet quantum code generation is still evaluated mostly within single frameworks, making it difficult to separate quantum reasoning from framework familiarity. We introduce QuanBench+, a unified benchmark spanning Qiskit, PennyLane, and Cirq, with 42 aligned tasks covering quantum algorithms, gate decomposition, and state preparation.
  We evaluate models with executable functional tests, report Pass@1 and Pass@5, and use KL-divergence-based acceptance for probabilistic outputs. We additionally study Pass@1 after feedback-based repair, where a model may revise code after a runtime error or wrong answer. Across frameworks, the strongest one-shot scores reach 59.5% in Qiskit, 54.8% in Cirq, and 42.9% in PennyLane; with feedback-based repair, the best scores rise to 83.3%, 76.2%, and 66.7%, respectively. These results show clear progress, but also that reliable multi-framework quantum code generation remains unsolved and still depends strongly on framework-specific knowledge.
QuanBench+ evaluates large language models on quantum code generation across multiple frameworks using functional testing and repair-based feedback, revealing significant progress but persistent dependence on framework-specific knowledge.
large language models quantum code generation Qiskit PennyLane Cirq Pass@1 Pass@5 KL-divergence feedback-based repair functional tests