C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling
We present C2LLM - Contrastive Code Large Language Models, a family of code embedding models in both 0.5B and 7B sizes. Building upon Qwen-2.5-Coder backbones, C2LLM adopts a Pooling by Multihead Attention (PMA) module for generating sequence embedding from token embeddings, effectively 1) utilizing the LLM's causal representations acquired during pretraining, while also 2) being able to aggregate information from all tokens in the sequence, breaking the information bottleneck in EOS-based sequence embeddings, and 3) supporting flexible adaptation of embedding dimension, serving as an alternative to MRL. Trained on three million publicly available data, C2LLM models set new records on MTEB-Code among models of similar sizes, with C2LLM-7B ranking 1st on the overall leaderboard.
694c9fd0746a34b55dd54020
Jin Qin
694c9fd0746a34b55dd54021
Zihan Liao
claimed_verified
2026-01-07T09:28:16.413Z
64c0e9969f7d8f3e79bc3469
/avatars/49d0fe2a696b05bc9fa80422ffb016e9.svg
Liaozihan
Elvis-t9
user
Elvis-t9
694c9fd0746a34b55dd54022
Ziyin Zhang
claimed_verified
2025-12-25T20:43:45.411Z
6430bdd8cd31d174a9f900fb
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Y9SPnRfpKSbYc7MhNdP-H.jpeg
Ziyin Zhang
Geralt-Targaryen
user
Geralt-Targaryen
694c9fd0746a34b55dd54023
Hang Yu
694c9fd0746a34b55dd54024
Peng Di
694c9fd0746a34b55dd54025
Rui Wang
C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling
We present C2LLM - Contrastive Code Large Language Models, a family of code embedding models in both 0.5B and 7B sizes. Building upon Qwen-2.5-Coder backbones, C2LLM adopts a Pooling by Multihead Attention (PMA) module for generating sequence embedding from token embeddings, effectively 1) utilizing the LLM's causal representations acquired during pretraining, while also 2) being able to aggregate information from all tokens in the sequence, breaking the information bottleneck in EOS-based sequence embeddings, and 3) supporting flexible adaptation of embedding dimension, serving as an alternative to MRL. Trained on three million publicly available data, C2LLM models set new records on MTEB-Code among models of similar sizes, with C2LLM-7B ranking 1st on the overall leaderboard.