Instructions to use citead/geo-citation-ranker-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use citead/geo-citation-ranker-4b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-Reranker-4B") model = PeftModel.from_pretrained(base_model, "citead/geo-citation-ranker-4b") - Notebooks
- Google Colab
- Kaggle
GEO Citation Ranker 4B
GEO Citation Ranker 4B is a LoRA adapter on Qwen3-Reranker-4B. Given a query and a candidate passage, it predicts how much of an answer engine's citations the passage will receive when the engine answers the query from it and its competitors. Use it to rank the candidate pages of one query.
Paper: Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval, Elisha Bajemon and André-Louis Rochet, TW3 Partners (Citead), 2026. Benchmark: citead/geo-citation-benchmark.
Results
Benchmark: 229 GEO-Bench queries with five candidate sources each, answered by ten engines that receive the five sources in their prompt (1220 engine-query groups). Metric: Spearman correlation between predicted and observed citation share within each group, averaged over groups, with 95% bootstrap intervals clustered by query. Differences are paired on the same groups.
| Scorer | Size | rho [95% CI] | GEO Citation Ranker 4B minus scorer |
|---|---|---|---|
| Other engines on the same query (reference) | n/a | 0.704 [0.669, 0.737] | -0.159 [-0.201, -0.116] |
| autotrust/JEV-27B, zero-shot | 27B | 0.578 [0.529, 0.621] | -0.037 [-0.080, +0.013] |
| Jev 1.13, hosted | n/a | 0.575 [0.523, 0.623] | -0.033 [-0.072, +0.006] |
| perplexity-ai/pplx-decider-v1.1-27b, zero-shot | 27B | 0.575 [0.524, 0.623] | -0.033 [-0.070, +0.005] |
| GEO Citation Ranker 4B | 4B | 0.542 [0.491, 0.591] | |
| autotrust/JEV-9B, zero-shot | 9B | 0.516 [0.463, 0.565] | +0.026 [-0.015, +0.072] |
| TokenRhythm/NeoHorse-Jev-4B, zero-shot | 4B | 0.501 [0.445, 0.556] | +0.048 [+0.002, +0.101] |
| Qwen3-Reranker-4B, zero-shot | 4B | 0.452 [0.394, 0.509] | +0.090 [+0.059, +0.121] |
| bge-reranker-v2-m3, zero-shot | 0.6B | 0.412 [0.351, 0.472] | +0.130 [+0.076, +0.179] |
| BM25 | n/a | 0.131 [0.063, 0.203] |
The ranker's differences with the two 27B decision models and with Jev cover zero on 229 queries. It scores one passage at a time; asked one passage at a time, Jev scores 0.529 and JEV-27B 0.540. Top-1 accuracy (the predicted top source is a most-cited source) is 0.567 for the ranker, 0.563 for Jev and 0.566 for JEV-27B, against 0.23 for a random choice.
Usage
python predict.py --adapter . --input passages.jsonl --output scored.jsonl
passages.jsonl holds one {"query": ..., "text": ...} object per line. The output adds a
score field: the log-odds of "yes" against "no". Scores rank the passages of one query; they
are not probabilities and are not comparable across queries.
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen3-Reranker-4B"
tok = AutoTokenizer.from_pretrained(base, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16).to("cuda")
model = PeftModel.from_pretrained(model, "citead/geo-citation-ranker-4b").eval()
prefix = ("<|im_start|>system\nJudge whether the Document meets the requirements based on the Query "
"and the Instruct provided. Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n"
"<|im_start|>user\n")
suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
instruct = ("Given a user query, judge whether this passage would be cited by a generative answer "
"engine (Perplexity, ChatGPT Search, Gemini) writing an answer to the query")
def score(query, passages):
texts = []
for p in passages:
ids = tok(p, add_special_tokens=False).input_ids
p = tok.decode(ids[:864]) if len(ids) > 864 else p
texts.append(prefix + f"<Instruct>: {instruct}\n<Query>: {query}\n<Document>: {p}" + suffix)
enc = tok(texts, padding=True, return_tensors="pt").to("cuda")
with torch.no_grad():
last = model(**enc, logits_to_keep=1).logits[:, -1, :].float()
yes, no = tok.convert_tokens_to_ids("yes"), tok.convert_tokens_to_ids("no")
return (last[:, yes] - last[:, no]).tolist()
Requirements: torch 2.5 or later, transformers 5 or later, peft 0.15 or later. The 4B needs about 10 GB of GPU memory in bf16 and scores a passage in about 133 ms on an RTX 3090.
Files
| Path | Base | rho | Note |
|---|---|---|---|
. (root) |
Qwen3-Reranker-4B | 0.542 | GEO Citation Ranker 4B, seed 42 |
seeds/s1337 |
Qwen3-Reranker-4B | 0.546 | same recipe, seed 1337 |
seeds/s7 |
Qwen3-Reranker-4B | 0.549 | same recipe, seed 7 |
variants/0.6b |
Qwen3-Reranker-0.6B | 0.457 | smaller variant, about 51 ms per passage |
Averaging the three 4B seeds gives 0.549 [0.497, 0.600].
Training
Data: 757 GEO-Bench test queries outside the evaluation set, five sources each (first 320 words). Labels: synthetic citation-share labels from two teachers, the hosted typed decision model Jev 1.13 and an LLM judge with a fixed rubric; the target is the mean of the two, each rescaled to [0, 1]. No engine answers were used for training.
Recipe: LoRA rank 16, alpha 32, dropout 0.05, on the q, k, v, o, gate, up and down projections; pairwise logistic loss within each group over pairs whose targets differ by more than 0.05, weighted by that difference; AdamW, learning rate 1e-4 with linear decay, 4 epochs, one group per step, bf16, passages truncated to 864 tokens. About two hours on one RTX 3090.
Limitations
- Evaluated on one English benchmark under controlled retrieval: engines receive five fixed sources in their prompt. Transfer to engines that search the web is not measured.
- The score needs a query. It is not a query-agnostic quality score for a page.
- Equivalence with Jev within plus or minus 0.05 is not established.
License
Apache 2.0, like the base model.
Citation
@misc{bajemon2026predicting,
author = {Bajemon, Elisha and Rochet, Andr{\'e}-Louis},
title = {Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval},
year = {2026},
note = {arXiv preprint}
}
- Downloads last month
- 23