# Using GNN4Colliders GNN4Colliders trains event classifiers on reconstructed collider objects: ```text ROOT files --prepare--> event cache (one row per object) --> graph or token batches --> model ``` ## Install Python 3.14 and PyTorch 2.14 (CUDA 13), managed by [uv](https://docs.astral.sh/uv/): ```bash uv sync --dev # includes pytest, ruff and the ONNX packages uv sync --dev --extra tabpfn # also TabPFN (configs/gamgam_tabpfn.yaml) uv sync --dev --extra tabicl # also TabICL (configs/gamgam_tabicl.yaml) uv sync --dev --extra tabdpt # also TabDPT (configs/gamgam_tabdpt.yaml) uv sync --dev --extra plots # matplotlib, for scripts/plot_label_efficiency.py uv run pytest # about 2 minutes; GPU tests run when CUDA is available uv run pytest --device mps # the fixture runs on an Apple GPU ``` ## Commands Every command takes an experiment file and optional `dotted.key=value` overrides (values are YAML): ```bash uv run gnn4colliders prepare configs/gamgam.yaml # ROOT -> data.cache uv run gnn4colliders train configs/gamgam.yaml # -> run_dir/{best,last}.pt uv run gnn4colliders evaluate configs/gamgam.yaml # -> run_dir/test_metrics.json uv run gnn4colliders predict configs/gamgam.yaml # -> run_dir/test_predictions.npz uv run gnn4colliders export configs/gamgam.yaml # -> run_dir/model.onnx uv run gnn4colliders probe configs/gamgam_jepa_finetune.yaml \ model.pretrained=outputs/delphes_jepa/last.pt data.train_events=1000 # -> run_dir/test_probe.json ``` `train` writes, in `run_dir` (default `outputs/`): * `config.yaml`: the resolved configuration; * `history.jsonl`: one line per epoch with the loss and metrics of the training and validation splits, the learning rate and the epoch time; * `best.pt`: the epoch with the best validation `train.best_metric` (AUC by default), `last.pt`: the latest epoch (resume with `train.resume=/last.pt`), and `epoch_NNNN.pt` every `train.save_every` epochs. `probe` fits k-NN and linear classifiers to the frozen event embeddings of the encoder of `model.pretrained` (JEPA, contrastive or supervised; random without) on `data.train_events` training events and reports their metrics on `split`, to compare pretraining without fine-tuning. `data.eval_events=20000` evaluates (and validates during training, predicts and probes) only that many random validation or test events, seeded by `data.seed`: AUCs to about 0.003, in minutes, for quick runs. `evaluate`, `predict` and `export` use `checkpoint` (default `best.pt`, else `last.pt`) on `split` (default `test`); `output` overrides the output file. Predictions hold the logits, scores, predicted class, label, weight, fold, source file and entry of every event; `gnn4colliders.inference.write_root_scores` adds scores to a copy of a ROOT tree. ## Experiments | File | Task | | --- | --- | | `configs/gamgam.yaml` | ATLAS Open Data GamGam, 5 Higgs production modes, from scratch (the paper baseline and main benchmark) | | `configs/gamgam_finetune.yaml` | the same task fine-tuned from the pretrained multiclass model | | `configs/gamgam_transformer.yaml` | the same task with a transformer with pair attention bias (`model.pair_bias=false` for a plain transformer; same event cache) | | `configs/gamgam_deep_sets.yaml` | the same task with Deep Sets, the baseline without object interactions (same event cache) | | `configs/tth_cp_even_odd.yaml` | CP-even versus CP-odd ttH (Delphes, binary; local files) | | `configs/delphes_pretraining.yaml` | 12-class pretraining on the Delphes samples | | `configs/delphes_jepa.yaml` | JEPA pretraining of the transformer encoder on the same samples, without labels (same event cache) | | `configs/delphes_jepa_edge.yaml` | the same for the edge network (fine-tune with `configs/gamgam_finetune.yaml model.pretrained=outputs/delphes_jepa_edge/last.pt`) | | `configs/delphes_contrastive.yaml` | contrastive (SimCLR) pretraining of the transformer encoder on the same samples (`model.backbone=edge_network` for the edge network) | | `configs/delphes_masked.yaml` | masked object modelling: JEPA predicting hidden objects' kinematics | | `configs/gamgam_jepa_finetune.yaml` | GamGam with the JEPA-pretrained encoder (`model.pretrained=null`: from scratch); also takes contrastive and masked-object encoders, and `probe` | | `configs/gamgam_tabpfn.yaml` | GamGam classified in context by TabPFN-3.5, no training (`evaluate` only) | | `configs/gamgam_tabicl.yaml` | the same by TabICLv2 (BSD-3 weights, downloaded from Hugging Face on first use) | | `configs/gamgam_tabdpt.yaml` | the same by TabDPT (Apache-2.0 weights; `model.tabdpt_context_size` retrieves each event's context) | | `configs/delphes_pfn.yaml` | meta-training of our own PFN on tasks drawn from the Delphes samples (same event cache) | | `configs/gamgam_pfn.yaml` | GamGam classified in context by that PFN, no training (`evaluate` only) | | `configs/delphes_tabpfn.yaml` | meta-training of TabPFN-3.5 (LoRA by default) on the same tasks; evaluate with `configs/gamgam_tabpfn.yaml checkpoint=outputs/delphes_tabpfn/best.pt` | | `configs/delphes_tabicl.yaml` | the same for TabICL; evaluate with `configs/gamgam_tabicl.yaml checkpoint=outputs/delphes_tabicl/best.pt` | Each file documents its keys; `src/gnn4colliders/config.py` lists every key with its default. Common overrides: ```bash train.fast=true # torch.compile + CUDA graphs (edge network) + bfloat16 + TF32 train.seed=3 data.seed=3 # initialization and shuffling seeds train.max_epochs=100 model.pretrained=Pretrained_GNN/multiclass_pretrained_model_12/model_epoch_71.pt model.freeze_backbone=true # train only the new classifier data.files=[{path: my.root, label: 0}] data.hf_repo=null # local inputs ``` ### Data Large inputs belong on scratch storage, not in the repository: set `HF_HOME` (where Hugging Face downloads go) and pass `data.cache=...`, e.g. on Perlmutter `HF_HOME=$SCRATCH/.cache/huggingface` and `data.cache=$SCRATCH/gnn4colliders/delphes/events.pt` (the 105M Delphes events take 52 GB of ROOT files and a 21 GB cache). `data.files` lists the ROOT files and their class labels (or a label branch name). With `data.hf_repo` the paths are inside that Hugging Face dataset at `data.hf_revision` and are downloaded once and checked against `sha256` and `size_bytes`. Each object becomes a row of `pt, eta, phi, energy, btag, charge, node_type` built from `feature_branches` (one branch or constant per object type, `CALC_E` = pt cosh(eta), `NODE_TYPE` = object-type index) and multiplied by `scales`. `derived_branches` select objects or threshold tagger scores before that (the GamGam file splits `lep_*` into electrons and muons). The fold of an event is `fold_var`, modulo `fold_modulus` when set, and `train_folds`/`validation_folds`/`test_folds` select the splits. Events are unweighted unless `weight_var` names a weight branch. ### In-context models and label efficiency `data.train_events=N` keeps N training events, an equal number per class (smaller N give subsets of larger ones). Supervised models train on them; the in-context models (`model.name: tabpfn` or `pfn`) use them as the labelled context and are only evaluated: ```bash uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation data.train_events=1000 uv run gnn4colliders train configs/delphes_pfn.yaml # meta-training uv run gnn4colliders evaluate configs/gamgam_pfn.yaml split=validation data.train_events=1000 uv run gnn4colliders train configs/gamgam.yaml data.train_events=1000 # supervised reference ``` `scripts/label_efficiency.py` runs such a sweep over several `N` and configs (restartable) and prints validation AUC and seconds per run; the metrics JSON records `seconds` (setup, inference) and, on CUDA, `peak_memory_gib`: ```bash uv run python scripts/label_efficiency.py configs/gamgam_tabpfn.yaml configs/gamgam.yaml \ --events 100 1000 10000 50000 ``` TabPFN-2.5 and later weights are non-commercial (research and internal benchmarking are allowed). tabpfn downloads them after a one-time login at https://ux.priorlabs.ai (token in `~/.cache/tabpfn/auth_token` or `TABPFN_TOKEN`), but a checkpoint already in its cache directory loads without the login, so on compute nodes download it from Hugging Face: ```bash export TABPFN_MODEL_CACHE_DIR=$SCRATCH/.cache/tabpfn # default ~/Library/Caches/tabpfn on macOS uvx --from huggingface_hub hf download Prior-Labs/tabpfn_3_5 config.json \ tabpfn-v3.5-20260909.safetensors tabpfn-v3.5-20260909_multiclass.safetensors \ --local-dir $TABPFN_MODEL_CACHE_DIR ``` `model.tabpfn_checkpoint=tabpfn-v3.5-20260909_multiclass.safetensors` selects the multiclass checkpoint. `model.tabpfn_version=v2` needs no login. TabDPT retrieval (`model.tabdpt_context_size`) on macOS needs `OMP_NUM_THREADS=1`: faiss and torch load separate OpenMP runtimes. Tabular models can use a pretrained encoder's event embeddings instead of or beside the rows: `model.tabular_inputs=embeddings` (or `both`) with `model.pretrained=outputs/delphes_jepa/last.pt` and the encoder's sizes (those of `configs/delphes_jepa.yaml`: `model.hid_size=128 model.n_layers=4 model.n_heads=8 model.ffn_size=512 model.pair_bias=true`). TabPFN and TabICL classify the same table rows (`model.slots` objects of each type). Two options add physics they cannot learn from a few events: `model.relative_phi=true` measures every azimuth from the leading object's, and `model.pair_objects=4` appends the invariant mass and Delta R of every pair of the four leading objects (12 columns). Compare with and without, one run name per variant: ```bash uv run python scripts/label_efficiency.py configs/gamgam_tabicl.yaml --events 100 1000 \ --override name=gamgam_tabicl_physics model.relative_phi=true model.pair_objects=4 ``` TabPFN-3.5 can also be meta-trained on the Delphes tasks, like our PFN. By default only LoRA updates of its in-context blocks train (`model.lora_rank`, 8; `model.lora_rank=0` trains all 219M weights), and checkpoints hold only those (33 MB with the optimizer state, against 2.6 GB for all weights). With a `checkpoint`, TabPFN is evaluated with its own forward pass, not the classifier: ```bash uv run gnn4colliders train configs/delphes_tabpfn.yaml uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation \ data.train_events=1000 checkpoint=outputs/delphes_tabpfn/best.pt ``` Activations are recomputed in the backward pass. On a 32 GB Apple M2 Max one task of 2048 context events and 256 queries takes about 4.5 s and 16 GiB per step with all weights trained, so the Mac is for development; meta-train on a GPU node. ### Fine-tuning `Pretrained_GNN/` holds the pretrained models of the paper, the final epoch of each: `multiclass_pretrained_model_12/model_epoch_71.pt` and `multilabel_pretrained_model_41/model_epoch_44.pt` (with their training configuration and log). With `model.pretrained` the edge network is loaded from that checkpoint and gets a new `model.out_size` classifier; `model.hid_size`, `n_layers` and `n_proc_steps` must match it (64, 4, 4). A JEPA checkpoint (`configs/delphes_jepa*.yaml`) loads the same way: its context encoder initializes the edge network or the transformer encoder. ## GPUs, Perlmutter and DDP `train.device: auto` uses CUDA, else an Apple GPU (MPS), else the CPU. MPS is for development on fixtures and small runs (no float64: metrics are computed on the CPU; no DDP, CUDA graphs or TF32). On a single GPU, `train.fast=true` is about 3x faster than float32 for the GamGam model (see [design.md](design.md)). Set `OMP_NUM_THREADS` to the CPU allocation and, on Perlmutter compute nodes, `UV_CACHE_DIR=$SCRATCH/.cache/uv` (uv cannot lock its home-directory cache there). ```bash # One training per seed, each followed by test-split evaluation: for seed in 1 2 3 4 5; do sbatch -A --export=ALL,SEED=$seed scripts/train.sh train.fast=true done uv run python scripts/summarize_ensemble.py outputs/gamgam/seed* # mean +- std ``` The study of in-context models and pretraining has one wrapper per kind of job, each on one GPU (the header of each script lists its variables): ```bash # Label-efficiency sweep of released in-context models and supervised references: sbatch -A --export=ALL,CONFIGS="configs/gamgam_tabpfn.yaml configs/gamgam_tabicl.yaml configs/gamgam.yaml" \ scripts/label_efficiency.sh # Meta-training, then GamGam in context with the meta-trained model: sbatch -A --export=ALL,CONFIG=configs/delphes_tabicl.yaml,EVAL_CONFIG=configs/gamgam_tabicl.yaml \ scripts/meta_train.sh task.prior_classes=[1,6,7,8,9,10,11] # Label-free pretraining, then k-NN and linear probes on GamGam: sbatch -A --export=ALL,CONFIG=configs/delphes_contrastive.yaml scripts/pretrain.sh # Linear CKA between the pretrained encoders, layer by layer (a login node is enough): uv run python scripts/compare_encoders.py configs/gamgam_jepa_finetune.yaml --events 5000 \ --encoders random=random jepa=outputs/delphes_jepa/last.pt \ contrastive=outputs/delphes_contrastive/last.pt masked=outputs/delphes_masked/last.pt # Curves and tables of every finished run: uv run --extra plots python scripts/plot_label_efficiency.py --sweeps outputs/label_efficiency \ --probes outputs/delphes_jepa outputs/delphes_contrastive outputs/delphes_masked ``` Under `torchrun` with more than one process, training is data parallel: `data.batch_size` is per process, training events are sharded over ranks (padded to equal length) and metrics are computed over all ranks' events. CUDA graphs need a single process (so `train.fast=true` does too; DDP is tested in float32 eager). `scripts/train_ddp.sh` launches all GPUs of one or more nodes.