Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization
Large Language Models (LLMs) are nowadays extensively used for various types of software engineering tasks, primarily code generation. Previous research has shown how suitable prompt engineering could help developers in improving their code generation prompts. However, so far, there do not exist specific guidelines driving developers towards writing suitable prompts for code generation. In this work, we derive and evaluate development-specific prompt optimization guidelines. First, we use an iterative, test-driven approach to automatically refine code generation prompts, and we analyze the outcome of this process to identify prompt improvement items that lead to test passes. We use such elements to elicit 10 guidelines for prompt improvement, related to better specifying I/O, pre-post conditions, providing examples, various types of details, or clarifying ambiguities. We conduct an assessment with 50 practitioners, who report their usage of the elicited prompt improvement patterns, as well as their perceived usefulness, which does not always correspond to the actual usage before knowing our guidelines. Our results lead to implications not only for practitioners and educators, but also for those aimed at creating better LLM-aided software development tools.
697710675d41524304c136de
Alessandro Midolo
697710675d41524304c136df
Alessandro Giagnorio
claimed_verified
2026-01-26T08:28:28.789Z
663486a1f64712540644cb68
https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg
Alessandro
Devy1
user
Devy1
697710675d41524304c136e0
Fiorella Zampetti
697710675d41524304c136e1
Rosalia Tufano
697710675d41524304c136e2
Gabriele Bavota
697710675d41524304c136e3
Massimiliano Di Penta
Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization
Large Language Models (LLMs) are nowadays extensively used for various types of software engineering tasks, primarily code generation. Previous research has shown how suitable prompt engineering could help developers in improving their code generation prompts. However, so far, there do not exist specific guidelines driving developers towards writing suitable prompts for code generation. In this work, we derive and evaluate development-specific prompt optimization guidelines. First, we use an iterative, test-driven approach to automatically refine code generation prompts, and we analyze the outcome of this process to identify prompt improvement items that lead to test passes. We use such elements to elicit 10 guidelines for prompt improvement, related to better specifying I/O, pre-post conditions, providing examples, various types of details, or clarifying ambiguities. We conduct an assessment with 50 practitioners, who report their usage of the elicited prompt improvement patterns, as well as their perceived usefulness, which does not always correspond to the actual usage before knowing our guidelines. Our results lead to implications not only for practitioners and educators, but also for those aimed at creating better LLM-aided software development tools.
Research derives and evaluates prompt optimization guidelines for code generation tasks in software engineering, identifying 10 specific improvement patterns related to input/output specification, conditions, examples, and clarity.
large language models code generation prompt engineering test-driven approach prompt optimization software engineering