UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models
Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, their effectiveness heavily relies on supervised training with extensive labeled (e.g., question-answering pairs) or unlabeled datasets (e.g., code snippets), which are often expensive and difficult to obtain at scale. To address this limitation, this paper introduces a method IPC, an unsupervised framework that leverages Internal Probing of LLMs for Code generation without any external corpus, even unlabeled code snippets. We introduce the problem space probing, test understanding probing, solution space probing, and knowledge consolidation and reinforcement to probe the internal knowledge and confidence patterns existing in LLMs. Further, IPC identifies reliable code candidates through self-consistency mechanisms and representation-based quality estimation to train UCoder (coder with unsupervised learning). We validate the proposed approach across multiple code benchmarks, demonstrating that unsupervised methods can achieve competitive performance compared to supervised approaches while significantly reducing the dependency on labeled data and computational resources. Analytic experiments reveal that internal model states contain rich signals about code quality and correctness, and that properly harnessing these signals enables effective unsupervised learning for code generation tasks, opening new directions for training code LLMs in resource-constrained scenarios.
6948fd1234f46eaf46cbb3be
Jiajun Wu
claimed_verified
2025-12-25T20:51:28.918Z
66a8e2538407031e388c501f
/avatars/d16d51f7b1e111efd6d0985995b614be.svg
wjj
wuyuverse
user
wuyuverse
6948fd1234f46eaf46cbb3bf
Jian Yang
6948fd1234f46eaf46cbb3c0
Wei Zhang
claimed_verified
2026-03-27T12:58:07.381Z
69c67b2fa994b07915a6e083
/avatars/3d0fd966df540d34095d2c84ce449180.svg
wei zhang
zwpride
user
zwpride
6948fd1234f46eaf46cbb3c1
Lin Jing
6948fd1234f46eaf46cbb3c2
Yuqing Ma
6948fd1234f46eaf46cbb3c3
Ensheng Shi
6948fd1234f46eaf46cbb3c4
Yuchi Ma
6948fd1234f46eaf46cbb3c5
Zhoujun Li
6948fd1234f46eaf46cbb3c6
Xianglong Liu
UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models
Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, their effectiveness heavily relies on supervised training with extensive labeled (e.g., question-answering pairs) or unlabeled datasets (e.g., code snippets), which are often expensive and difficult to obtain at scale. To address this limitation, this paper introduces a method IPC, an unsupervised framework that leverages Internal Probing of LLMs for Code generation without any external corpus, even unlabeled code snippets. We introduce the problem space probing, test understanding probing, solution space probing, and knowledge consolidation and reinforcement to probe the internal knowledge and confidence patterns existing in LLMs. Further, IPC identifies reliable code candidates through self-consistency mechanisms and representation-based quality estimation to train UCoder (coder with unsupervised learning). We validate the proposed approach across multiple code benchmarks, demonstrating that unsupervised methods can achieve competitive performance compared to supervised approaches while significantly reducing the dependency on labeled data and computational resources. Analytic experiments reveal that internal model states contain rich signals about code quality and correctness, and that properly harnessing these signals enables effective unsupervised learning for code generation tasks, opening new directions for training code LLMs in resource-constrained scenarios.
IPC is an unsupervised framework that uses internal probing of large language models to generate code without labeled datasets, achieving competitive performance with reduced resource dependency.
large language models unsupervised framework internal probing problem space probing test understanding probing solution space probing knowledge consolidation reinforcement self-consistency mechanisms representation-based quality estimation UCoder code benchmarks internal model states code quality correctness resource-constrained scenarios