PEFT
Safetensors
English
reranker
generative-engine-optimization
citation
lora

GEO Citation Ranker 4B

GEO Citation Ranker 4B is a LoRA adapter on Qwen3-Reranker-4B. Given a query and a candidate passage, it predicts how much of an answer engine's citations the passage will receive when the engine answers the query from it and its competitors. Use it to rank the candidate pages of one query.

Paper: Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval, Elisha Bajemon and André-Louis Rochet, TW3 Partners (Citead), 2026. Benchmark: citead/geo-citation-benchmark.

Results

Benchmark: 229 GEO-Bench queries with five candidate sources each, answered by ten engines that receive the five sources in their prompt (1220 engine-query groups). Metric: Spearman correlation between predicted and observed citation share within each group, averaged over groups, with 95% bootstrap intervals clustered by query. Differences are paired on the same groups.

Scorer Size rho [95% CI] GEO Citation Ranker 4B minus scorer
Other engines on the same query (reference) n/a 0.704 [0.669, 0.737] -0.159 [-0.201, -0.116]
autotrust/JEV-27B, zero-shot 27B 0.578 [0.529, 0.621] -0.037 [-0.080, +0.013]
Jev 1.13, hosted n/a 0.575 [0.523, 0.623] -0.033 [-0.072, +0.006]
perplexity-ai/pplx-decider-v1.1-27b, zero-shot 27B 0.575 [0.524, 0.623] -0.033 [-0.070, +0.005]
GEO Citation Ranker 4B 4B 0.542 [0.491, 0.591]
autotrust/JEV-9B, zero-shot 9B 0.516 [0.463, 0.565] +0.026 [-0.015, +0.072]
TokenRhythm/NeoHorse-Jev-4B, zero-shot 4B 0.501 [0.445, 0.556] +0.048 [+0.002, +0.101]
Qwen3-Reranker-4B, zero-shot 4B 0.452 [0.394, 0.509] +0.090 [+0.059, +0.121]
bge-reranker-v2-m3, zero-shot 0.6B 0.412 [0.351, 0.472] +0.130 [+0.076, +0.179]
BM25 n/a 0.131 [0.063, 0.203]

The ranker's differences with the two 27B decision models and with Jev cover zero on 229 queries. It scores one passage at a time; asked one passage at a time, Jev scores 0.529 and JEV-27B 0.540. Top-1 accuracy (the predicted top source is a most-cited source) is 0.567 for the ranker, 0.563 for Jev and 0.566 for JEV-27B, against 0.23 for a random choice.

Usage

python predict.py --adapter . --input passages.jsonl --output scored.jsonl

passages.jsonl holds one {"query": ..., "text": ...} object per line. The output adds a score field: the log-odds of "yes" against "no". Scores rank the passages of one query; they are not probabilities and are not comparable across queries.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-Reranker-4B"
tok = AutoTokenizer.from_pretrained(base, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16).to("cuda")
model = PeftModel.from_pretrained(model, "citead/geo-citation-ranker-4b").eval()

prefix = ("<|im_start|>system\nJudge whether the Document meets the requirements based on the Query "
          "and the Instruct provided. Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n"
          "<|im_start|>user\n")
suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
instruct = ("Given a user query, judge whether this passage would be cited by a generative answer "
            "engine (Perplexity, ChatGPT Search, Gemini) writing an answer to the query")

def score(query, passages):
    texts = []
    for p in passages:
        ids = tok(p, add_special_tokens=False).input_ids
        p = tok.decode(ids[:864]) if len(ids) > 864 else p
        texts.append(prefix + f"<Instruct>: {instruct}\n<Query>: {query}\n<Document>: {p}" + suffix)
    enc = tok(texts, padding=True, return_tensors="pt").to("cuda")
    with torch.no_grad():
        last = model(**enc, logits_to_keep=1).logits[:, -1, :].float()
    yes, no = tok.convert_tokens_to_ids("yes"), tok.convert_tokens_to_ids("no")
    return (last[:, yes] - last[:, no]).tolist()

Requirements: torch 2.5 or later, transformers 5 or later, peft 0.15 or later. The 4B needs about 10 GB of GPU memory in bf16 and scores a passage in about 133 ms on an RTX 3090.

Files

Path Base rho Note
. (root) Qwen3-Reranker-4B 0.542 GEO Citation Ranker 4B, seed 42
seeds/s1337 Qwen3-Reranker-4B 0.546 same recipe, seed 1337
seeds/s7 Qwen3-Reranker-4B 0.549 same recipe, seed 7
variants/0.6b Qwen3-Reranker-0.6B 0.457 smaller variant, about 51 ms per passage

Averaging the three 4B seeds gives 0.549 [0.497, 0.600].

Training

Data: 757 GEO-Bench test queries outside the evaluation set, five sources each (first 320 words). Labels: synthetic citation-share labels from two teachers, the hosted typed decision model Jev 1.13 and an LLM judge with a fixed rubric; the target is the mean of the two, each rescaled to [0, 1]. No engine answers were used for training.

Recipe: LoRA rank 16, alpha 32, dropout 0.05, on the q, k, v, o, gate, up and down projections; pairwise logistic loss within each group over pairs whose targets differ by more than 0.05, weighted by that difference; AdamW, learning rate 1e-4 with linear decay, 4 epochs, one group per step, bf16, passages truncated to 864 tokens. About two hours on one RTX 3090.

Limitations

  • Evaluated on one English benchmark under controlled retrieval: engines receive five fixed sources in their prompt. Transfer to engines that search the web is not measured.
  • The score needs a query. It is not a query-agnostic quality score for a page.
  • Equivalence with Jev within plus or minus 0.05 is not established.

License

Apache 2.0, like the base model.

Citation

@misc{bajemon2026predicting,
  author = {Bajemon, Elisha and Rochet, Andr{\'e}-Louis},
  title  = {Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval},
  year   = {2026},
  note   = {arXiv preprint}
}
Downloads last month
23
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for citead/geo-citation-ranker-4b

Adapter
(6)
this model

Datasets used to train citead/geo-citation-ranker-4b