---
license: mit
tags:
- biology
- protein
- protein-embeddings
- benchmark
---

# ProtCompass embeddings

Frozen protein embeddings, input-audit records and label files for the paper
**ProtCompass: Biological Information and Predictive Utility in Protein Representations**
(anonymous submission to ICLR 2027).

Protein encoders are usually evaluated by structural probing, downstream prediction and concept erasure. These
measurements can mix signal learned by the encoder with information that is already present in the input or that the
evaluation adds. ProtCompass separates these sources. It audits which inputs each encoder actually reads, compares
structural probe scores with non-learned descriptions of the same inputs, compares downstream scores with simple
sequence baselines, and controls concept-erasure experiments for the representation capacity that is removed together
with the target property. The paper evaluates 33 encoders on a probing suite of 21 probes built from CATH 4.3 domains
and on ten downstream tasks.

- Code, result tables and analysis scripts: <https://anonymous.4open.science/r/ProEmbedEva-C4F6>
- Predicted structures for four downstream tasks: <https://huggingface.co/datasets/Anonymoususer2223/ProtEnv>

This repository holds the release that backs the paper: embeddings, labels, the input audit and the pretrained
weights of the structure encoders. Its paths mirror the working directory of the code (for example
`extract/embeddings/<task>/<encoder>/` and `checkpoints/`). An earlier release was removed; see
[Earlier release (removed)](#earlier-release-removed).

## Contents

Sizes are in GB (10^9 bytes).

| Path | Content | Files | Size | Paper appendix |
|---|---|---:|---:|---|
| `extract/embeddings/<task>/<encoder>/` | pooled embeddings of 33 encoders and one non-learned reference on the ten downstream tasks | 2,237 | 6.5 | Tasks and protocols (The downstream tasks); Controls for the downstream comparison; Controls for the erasure analysis; Seed variability |
| `extract/manifest/<task>.csv` | one row per protein: id, split, sequence, structure file, label | 11 | 0.12 | Tasks and protocols (The downstream tasks) |
| `extract/pairs/ppi_affinity_pairs.csv` | chain pairs and labels of the PPI affinity task | 1 | <0.01 | Tasks and protocols (The downstream tasks) |
| `extract/encoder_input_audit.csv`, `encoder_atom_audit.csv`, `encoder_sidechain_audit.csv` | measured input audit | 3 | <0.01 | Study Design; The Encoder Panel |
| `probing_embeddings/<encoder>/` | per-residue and pooled embeddings of the 33 encoders on 19,461 CATH 4.3 domains | 101 | 96.9 | Tasks and protocols (Probe protocol); Controls for the probing comparison; Two residue-level tasks, and what the pooled readout hides |
| `probing_embeddings_vframe/<encoder>_vframe/` | GVP, GCPNet (random and pretrained) and MACE with the direction of their vector channels kept | 19 | 5.2 | Controls for the probing comparison (Equivariant encoders with the direction of their vector channels kept) |
| `probing_data/labels.h5`, `probing_data/splits/` | probe labels, coordinates and splits of the CATH domains | 8 | 0.32 | Tasks and protocols (The probing suite; Probe protocol) |
| `extract/ab_modality/embeddings/`, `extract/ab_modality/probing_embeddings/` | ESM3 and SaProt with the structure input removed, and SaProt with shuffled 3Di tokens | 210 | 24.9 | Controls for the downstream comparison (Removing the structure input from the same checkpoint) |
| `extract/ab_modality/encoder_input_audit_ab*.csv` | input audit of those three sources | 2 | <0.01 | same |
| `confirmation/`, `confirmation2/` | probing cohorts A and B: CATH 4.4 domains absent from CATH 4.3 | 52 | 6.3 | Confirmation on data and splits the development did not use (The probing headline on CATH 4.4 domains absent from CATH 4.3) |
| `confirmation_downstream/` | held-out downstream proteins never used in development | 280 | 0.54 | Confirmation on data and splits the development did not use (The demand results on downstream proteins never scored) |
| `masif_site/` | residue-level interface task rebuilt from MaSIF-site | 90 | 20.2 | Two residue-level tasks, and what the pooled readout hides (Interface residues) |
| `atlas/data_v4/` | latent-space atlas: 2D maps, k-NN purity and CKA for every task layer | 12 | 0.06 | Latent-Space Atlas |
| `checkpoints/<encoder>/` | pretrained weights of six structure encoders, with licences and `SHA256SUMS` | 14 | 0.16 | The Encoder Panel |

The current release has 3,040 files and 161.2 GB in total. The appendix names are section and subsection titles of
the paper.

## Encoders

The paper evaluates 33 encoders. They are declared as 9 sequence-based, 19 structure-based and 5 multimodal models.
The analyses group them by a measured input audit, which finds that 10 encoders do not read coordinates
(sequence-only) and 23 do (coordinate-reading). The only encoder whose measured group differs from its declared class
is ProstT5, which is declared multimodal and read on its amino-acid track only.

The directory name is the key used in every path. *Width* is the width of the stored array. *Atoms read* comes from
the atom audit (`extract/encoder_atom_audit.csv`); "side chain" marks the three encoders that the side-chain audit
(`extract/encoder_sidechain_audit.csv`) finds to read atoms beyond CB.

| Directory | Paper name | Declared class | Weights | Width | Audit group | Reads residue identity | Atoms read |
|---|---|---|---|---:|---|---|---|
| `ankh_base` | Ankh | sequence | pretrained | 768 | sequence-only | yes | - |
| `esm1b` | ESM-1b | sequence | pretrained | 1280 | sequence-only | yes | - |
| `esm2_8m` | ESM2-8M | sequence | pretrained | 320 | sequence-only | yes | - |
| `esm2_35m` | ESM2-35M | sequence | pretrained | 480 | sequence-only | yes | - |
| `esm2_150m` | ESM2-150M | sequence | pretrained | 640 | sequence-only | yes | - |
| `esm2_650m` | ESM2-650M | sequence | pretrained | 1280 | sequence-only | yes | - |
| `esm2_3b` | ESM2-3B | sequence | pretrained | 2560 | sequence-only | yes | - |
| `protbert_bfd` | ProtBert | sequence | pretrained | 1024 | sequence-only | yes | - |
| `prottrans_t5_xl_u50` | ProtT5-XL | sequence | pretrained | 1024 | sequence-only | yes | - |
| `prostt5` | ProstT5 | multimodal | pretrained | 1024 | sequence-only | yes | - |
| `dimenet` | DimeNet | structure | random | 128 | coordinate-reading | yes | CA |
| `esm_if1` | ESM-IF1 | structure | pretrained | 512 | coordinate-reading | no | N, CA, C |
| `foldseek_3di` | Foldseek | structure | pretrained (tokeniser, no learned parameters) | 20 | coordinate-reading | no | N, CA, CB |
| `gcpnet_random` | GCPNet | structure | random | 116 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
| `gcpnet_pretrained` | GCPNet-pre | structure | pretrained | 116 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
| `gvp` | GVP | structure | random | 116 | coordinate-reading | yes | N, CA, C |
| `gearnet` | GearNet | structure | random | 512 | coordinate-reading | yes | CA |
| `gearnet_pretrained` | GearNet-pre | structure | pretrained | 512 | coordinate-reading | yes | CA |
| `ipa` | IPA | structure | random | 128 | coordinate-reading | yes | CA |
| `ipa_pretrained` | IPA-pre | structure | pretrained | 128 | coordinate-reading | yes | CA |
| `mace` | MACE | structure | random | 128 | coordinate-reading | yes | CA |
| `pottsmpnn` | PottsMPNN | structure | random | 128 | coordinate-reading | no | N, CA, C, O |
| `pottsmpnn_ft` | PottsMPNN-ft | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
| `pottsmpnn_pretrained` | PottsMPNN-pre | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
| `pottsmpnn_soluble` | PottsMPNN-sol | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
| `proteinmpnn` | ProteinMPNN | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
| `schnet` | SchNet | structure | random | 128 | coordinate-reading | yes | CA |
| `tfn` | TFN | structure | random | 128 | coordinate-reading | yes | CA |
| `tfn_pretrained` | TFN-pre | structure | pretrained | 128 | coordinate-reading | yes | CA |
| `cdconv` | CDConv | multimodal | random | 256 | coordinate-reading | yes | CA |
| `esm3` | ESM3 | multimodal | pretrained | 1536 | coordinate-reading | yes | N, CA, C |
| `foldvision` | FoldVision | multimodal | pretrained | 1024 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
| `saprot` | SaProt | multimodal | pretrained | 1280 | coordinate-reading | yes | N, CA, CB |

Randomly initialised encoders use one fixed seed for all proteins; the hash of their weights is recorded in
`meta.json`. Three further encoders were audited and are not released: EGNN (its embeddings do not change when the
coordinates change), GeoBPE (its output changes under a rigid motion) and dMaSIF (all-zero embeddings). FoldVision has
no extraction on remote homology, so it is scored on nine tasks.

### Which layer is read

Every embedding is taken from the last representation the encoder computes before any task head or decoder. Where an
encoder has residue states, a protein is the masked mean of its residue states. The rule of each encoder is recorded in
the `rule` field of its `meta.json`.

| Encoders | Representation read |
|---|---|
| Ankh, ESM-1b, ESM2 (all sizes), ProtBert, ProtT5-XL | final hidden state, special tokens removed |
| ProstT5 | final hidden state of the amino-acid track with the `<AA2fold>` prefix, special tokens removed; no 3Di input |
| ESM3 | final trunk residue embeddings, sequence and structure tracks |
| SaProt | final hidden states at the residue positions of the combined amino-acid and 3Di tokens |
| SchNet, TFN, DimeNet, MACE, GearNet, CDConv, IPA (random and pretrained versions) | final invariant node states; IPA reads the structure in a canonical pose |
| GVP, GCPNet (random and pretrained) | final scalar channels plus the norm of each vector channel (100 + 16 = 116) |
| ProteinMPNN, PottsMPNN (all four) | encoder output h_V, before the decoder |
| ESM-IF1 | output of the GVP-Transformer encoder, before decoding |
| Foldseek | one-hot 3Di state per residue; the protein vector is the 3Di state frequency |
| FoldVision | the model's global embedding; no residue states |

## Pretrained weights

`checkpoints/` holds the local checkpoint files that six of the released structure encoders read. Every file is
byte-identical to its upstream release, and loading it with the project's wrapper on CPU reproduces the weight hash
(`weights_sha256`) recorded in the `meta.json` of the released embeddings. The upstream licence is stored next to each
file, and `checkpoints/SHA256SUMS` lists the full hashes. `checkpoints/README.md` has a script that links the files
to the paths the code reads.

| Encoder | File in this repository | Size (MB) | SHA-256 (first 16) | Source | Licence | Path the code reads |
|---|---|---:|---|---|---|---|
| ProteinMPNN | `checkpoints/proteinmpnn/proteinmpnn_v48_020.pt` | 6.7 | `c9cb4a671d796041` | ProteinMPNN repository, `vanilla_model_weights/v_48_020.pt` | MIT | `protcompass/checkpoints/proteinmpnn_v48_020.pt` |
| GearNet-pre | `checkpoints/gearnet_pretrained/mc_gearnet_edge.pth` | 80.7 | `6dc11e2db6eff3c9` | Zenodo, DOI 10.5281/zenodo.7593637 | CC BY 4.0 | `protcompass/checkpoints/gearnet/mc_gearnet_edge.pth` |
| GCPNet-pre (GCPNet reads its hyper-parameters) | `checkpoints/gcpnet_pretrained/model_epoch_115_..._globalkendall_0_676.ckpt` | 14.3 | `ff7b73be01fb57d5` | GCPNet repository, `checkpoints/PSR/` (PSR task model) | MIT | `protcompass/external_repos/GCPNet/checkpoints/PSR/` |
| PottsMPNN-pre | `checkpoints/pottsmpnn_pretrained/pottsmpnn_20.pt` | 20.5 | `28b1620a71883142` | PottsMPNN repository, `proteinmpnn_compatible_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/proteinmpnn_compatible_model_weights/pottsmpnn_20.pt` |
| PottsMPNN-sol | `checkpoints/pottsmpnn_soluble/sol_pottsmpnn_20.pt` | 20.7 | `f203a69773df41a1` | PottsMPNN repository, `soluble_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/soluble_model_weights/sol_pottsmpnn_20.pt` |
| PottsMPNN-ft | `checkpoints/pottsmpnn_ft/potts_ft.pt` | 20.7 | `b672ff4ed1492560` | PottsMPNN repository, `ft_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/ft_model_weights/potts_ft.pt` |

The code paths are relative to the root of the code repository, whose `protcompass/` folder mirrors this repository.
The upstream repositories are github.com/dauparas/ProteinMPNN (commit 8907e66), github.com/BioinfoMachineLearning/GCPNet
(commit 172733b) and github.com/KeatingLab/PottsMPNN (commit a6ccac3); a clone of GCPNet or PottsMPNN at that commit
already holds its files at the paths the code reads.

Weights loaded from a public source, which this repository does not copy:

| Encoder | Source | Revision |
|---|---|---|
| ESM-1b | `facebook/esm1b_t33_650M_UR50S` | `7b37824baec4d3658e1df7479222a7c79b465b76` |
| ESM2-8M | `facebook/esm2_t6_8M_UR50D` | `c731040fcd8d73dceaa04b0a8e6329b345b0f5df` |
| ESM2-35M | `facebook/esm2_t12_35M_UR50D` | `6fbf070e65b0b7291e7bbcd451118c216cff79d8` |
| ESM2-150M | `facebook/esm2_t30_150M_UR50D` | `a695f6045e2e32885fa60af20c13cb35398ce30c` |
| ESM2-650M | `facebook/esm2_t33_650M_UR50D` | `08e4846e537177426273712802403f7ba8261b6c` |
| ESM2-3B | `facebook/esm2_t36_3B_UR50D` | `476b639933c8baad5ad09a60ac1a87f987b656fc` |
| ProtBert | `Rostlab/prot_bert_bfd` | `6c5c8a55a52ff08a664dfd584aa1773f125a0487` |
| ProtT5-XL | `Rostlab/prot_t5_xl_half_uniref50-enc` | `94a6abc029ae13029317b140b7424e012bf8dfbf` |
| Ankh | `ElnaggarLab/ankh-base` | `d99cb6b966530dfc2ae96bc69d9255c2a07308b0` |
| ProstT5 | `Rostlab/ProstT5` | `d7d097d5bf9a993ab8f68488b4681d6ca70db9e5` |
| SaProt | `westlake-repl/SaProt_650M_AF2` | `d9b9ad00ef61c0990e611b2b43f2231c7de24b38` |
| FoldVision | `AlexanderKroll/foldvision-encoder` | not recorded |
| ESM-IF1 | `torch.hub.load("facebookresearch/esm:main", "esm_if1_gvp4_t16_142M_UR50")` (ESM release, MIT) | not recorded |
| Foldseek 3Di | the `mini3di` package, version 0.2.1 (BSD-3-Clause); no weights file | 0.2.1 |

The first twelve are Hugging Face hub ids; the revisions are those recorded in the `checkpoint_revision` field of each
`meta.json`.

- **ESM3.** The `esm3-sm-open-v1` weights are distributed by EvolutionaryScale under the Cambrian Non-Commercial
  License Agreement, which does not allow redistribution, so they are not in this repository. Accept the licence and
  download them from the gated Hugging Face repository `EvolutionaryScale/esm3-sm-open-v1`. The code expects them under
  `protcompass/esm3-sm-open-v1/` (weights file `data/weights/esm3_sm_open_v1.pth`).
- **TFN-pre and IPA-pre.** Their weights were taken from the models' released repositories and are not redistributed
  here. The `meta.json` of their embeddings records the SHA-256 prefix of the file that was used (`17af58de61a4674c`
  for TFN-pre, `cd4ca8e7f61daba2` for IPA-pre). The code expects them at `protcompass/checkpoints/tfn_pretrained/best.pt`
  and `protcompass/checkpoints/ipa_pretrained/best.pt`.
- **Randomly initialised encoders** (SchNet, TFN, DimeNet, MACE, GearNet, IPA, GVP, GCPNet, CDConv, PottsMPNN) have no
  weights file. They are built with the fixed seed 20260919, and the hash of the resulting weights is in each
  `meta.json`.

## Downstream tasks

Fourteen tasks were assembled. Four (contact prediction, mutation effect, PPI sites and metal binding) were excluded
because their labels or metric were unsuitable (appendix "Tasks and analyses the paper evaluated and does not use"),
and ten are scored. Every encoder reads the same rows of a task. For
each task the embeddings cover the rows of the manifest with `stage1 == 1`.

| Directory | Task | Metric | Source dataset | Structures | train / valid / test |
|---|---|---|---|---|---|
| `ec_classification` | EC classification | accuracy | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 3,000 |
| `go_mf` | GO molecular function | F_max | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 2,904 |
| `go_bp` | GO biological process | F_max | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 2,903 |
| `remote_homology` | remote homology (fold) | accuracy | TAPE | experimental (PDB) | 6,000 / 506 / 480 |
| `subcellular_localization` | localisation, 10 classes | accuracy | DeepLoc 2 | predicted (AlphaFold DB) | 6,000 / 1,500 / 2,473 |
| `membrane_soluble` | membrane versus soluble | AUROC | DeepLoc 2 | predicted (AlphaFold DB) | 6,000 / 1,500 / 3,000 |
| `stability` | stability | Spearman | TAPE | predicted (ESMFold) | 6,000 / 1,500 / 3,000 |
| `fluorescence` | fluorescence | Spearman | TAPE | predicted (ESMFold) | 6,000 / 1,500 / 3,000 |
| `solubility` | solubility | AUROC | FLIP | predicted (ESMFold) | 5,417 / 1,363 / 1,329 |
| `ppi_affinity` | PPI affinity | Spearman | SKEMPI2 | predicted (ESMFold) | 2,589 chains; 2,352 / 194 / 326 pairs |

The structure sources and split sizes are those of the paper's table "Structures and splits of the ten scored tasks".
Every extracted row has a structure file whose sequence equals the task sequence (`structure_coverage` in `meta.json`).

## File formats

### Downstream embeddings: `extract/embeddings/<task>/<encoder>/`

| File | Content |
|---|---|
| `train.npy`, `valid.npy`, `test.npy` | float32 array of shape (number of proteins in the split, width) |
| `ids_train.txt`, `ids_valid.txt`, `ids_test.txt` | one protein id per line; line i is row i of the array of the same split |
| `meta.json` | extraction record (fields below) |
| `chains.npy`, `ids_chains.txt` | PPI affinity only: one row per chain, in place of the three splits |

- Row order equals the order of the manifest rows with `stage1 == 1` and the same `split`. The id files are identical
  for every encoder of a task.
- Labels are in the `label` column of `extract/manifest/<task>.csv`: an integer class index (EC, localisation), a
  fold index (remote homology), a list of term indices written as a string (GO-MF, GO-BP), 0 or 1 (solubility,
  membrane versus soluble) or a real value (stability, fluorescence).
- PPI affinity: `extract/pairs/ppi_affinity_pairs.csv` has columns `id, split, chain_a, chain_b, label`. The paper
  represents a pair by concatenating the embeddings of `chain_a` and `chain_b` in that order.
- `ref_composition/` in every task directory is not an encoder. It is the non-learned composition reference of the
  paper: 20 amino-acid frequencies, log length, length/600 and the fraction of residues outside the 20-letter
  alphabet (23 values).
- `extract/manifest/<task>.csv` has columns `id, split, sequence, structure_path, length, label, stage1`, plus
  `align_path, struct_coverage` (experimental and AlphaFold DB structures) or `mean_plddt, chain_break` (ESMFold
  structures).

Fields of `meta.json` (not every field is present for every encoder):

| Field | Meaning |
|---|---|
| `encoder`, `task` | directory keys |
| `family`, `target`, `patch`, `env` | how the encoder was loaded: a Hugging Face language model (`hf_plm`), a model wrapper (`registry`) or a patched official implementation (`patched`) |
| `weights` | `pretrained`, `random` or `none` |
| `checkpoint`, `checkpoint_revision`, `checkpoint_sha256`, `checkpoint_loaded_by`, `weights_sha256`, `seed` | the weights used. Hub checkpoints give the hub id and revision; local checkpoint paths are given relative to the project directory |
| `rule` | which representation is read (see [Which layer is read](#which-layer-is-read)) |
| `pooling` | how residue states become one protein vector (masked mean over all residues of the chain) |
| `inputs_passed`, `pose` | what the encoder receives |
| `dtype`, `hidden_size`, `n_layers`, `n_parameters` | model details |
| `manifest`, `rows`, `structure_coverage`, `n_per_split` | which manifest rows were extracted |
| `health` | `n_rows`, `dim`, `nan_values`, `all_zero_rows`, `distinct_rows`, `numerical_rank`, `max_abs` of the stored array |
| `input_audit` | relative change of the embedding under a 1 Å coordinate jitter, a coordinate swap, a sequence shuffle and a rigid motion, and the audit verdict |
| `date`, `seconds` | when and how long |

### Probing embeddings: `probing_embeddings/`

The probing set is 19,461 CATH 4.3 domains with 2,823,525 residues. The same row convention holds for every probing
directory in this repository.

| File | Content |
|---|---|
| `ids.txt` | CATH domain id per line; line k is domain k |
| `offsets.npy`, `lengths.npy` | int64, shape (19461,); the residues of domain k are rows `offsets[k]` to `offsets[k] + lengths[k] - 1` |
| `<encoder>/residue_fp16.npy` | float16, shape (2823525, width); row r is row r of `residue/*` in `probing_data/labels.h5` |
| `<encoder>/pooled_fp32.npy` | float32, shape (19461, width); row k is domain k (mean over its residues) |
| `<encoder>/meta.json` | extraction record; its `row_convention` or `rows` field states the row order |

FoldVision has no residue states, so `probing_embeddings/foldvision/` holds `pooled_fp32.npy` only, and only the
protein-level probes apply to it. Language models were run on the sequence of the residues kept in `labels.h5`; a
discontinuous CATH domain is passed as one concatenated sequence.

`probing_data/labels.h5` holds three groups. `protein/` has one row per domain (id, length, CATH labels, split codes,
protein-level targets). `residue/` has one row per residue (amino acid, N, CA, C, O and CB coordinates with CB built
from ideal geometry where it is absent, DSSP 3- and 8-state, relative solvent accessibility, B-factor, backbone angles, contact counts and other residue labels). `pair/`
has the sampled residue pairs with their distances. The split codes are `split_superfamily` (15,565 train, 1,948
validation, 1,948 test; no superfamily is shared between splits) and `split_topology` (15,378 / 2,159 / 1,924; the
topology-disjoint split of the confirmation experiments), with 0 = train, 1 = validation and 2 = test.
`probing_data/splits/` lists the same splits as domain ids. The CATH domain structure files are public and are not
redistributed here.

`probing_embeddings_vframe/<encoder>_vframe/` follows the same layout. It stores the scalar channels together with the
vector channels projected onto a per-residue frame built from N, CA and C (width 148 for GVP and the two GCPNets, 176
for MACE). The two GCPNet directories also hold `residue_atommean_fp16.npy` and `pooled_atommean_fp32.npy`, the same
representation averaged over the heavy atoms of each residue. The frame definition is in each `meta.json`.

### Modality ablation: `extract/ab_modality/`

Three extra sources hold the weights fixed and change only the input.

| Source | Input |
|---|---|
| `esm3_seqonly` | ESM3 with the true sequence and the mask token at every structure position (no coordinates) |
| `saprot_seqonly` | SaProt with every 3Di token replaced by `#` (the token for a residue without structure) |
| `saprot_shuf3di` | SaProt with each protein's own 3Di tokens permuted within the protein (seed from the protein id) |

`extract/ab_modality/embeddings/<task>/<source>/` follows the downstream format for all ten tasks, and
`extract/ab_modality/probing_embeddings/<source>/` follows the probing format. The audit rows of these sources are in
`extract/ab_modality/encoder_input_audit_ab.csv` and `encoder_input_audit_ab_shuf.csv`. The full-input ESM3 and
SaProt embeddings they are compared with are the `esm3` and `saprot` entries of the main release.

### Confirmation sets

| Path | Content |
|---|---|
| `confirmation/` | cohort A: 1,990 CATH v4.4 S40 domains absent from CATH v4.3. `embeddings/<encoder>/` in the probing format for Ankh, ESM2-650M, ESM3, ESM-IF1, ProteinMPNN and ProtT5-XL; `labels_v44.h5` in the schema of `labels.h5`; `cohort_manifest.csv` with the CATH class, architecture, topology and superfamily of each domain |
| `confirmation2/` | cohort B: 1,316 further CATH v4.4 domains, same layout, the six encoders of cohort A plus both GCPNets |
| `confirmation_downstream/` | eight held-out downstream cohorts, 13,632 proteins taken from manifest rows that were never extracted or scored in development. `embeddings/<task>/<encoder>/` holds `test.npy`, `ids_test.txt` and `meta.json`; `ids_<task>.txt` and `cohort_<task>.csv` list the proteins and labels. Solubility and PPI affinity have no unused rows and have no cohort |

### Residue-level interface task: `masif_site/`

Interface residues rebuilt from MaSIF-site with a label defined from geometry: a residue of the target chain is an
interface residue if any of its heavy atoms lies within 5 Å of a heavy atom of another chain. There are 3,327 chains
(2,677 train, 297 validation, 353 test) and 647,332 residues. `embeddings/<encoder>/` follows the probing format for 28
of the 33 encoders (SaProt, Foldseek, both GCPNets and FoldVision are not scored on this task). `labels.h5` holds the
residue labels (`residue/interface`) and the split (`protein/split_masif`); `exclusions.csv` lists the 35 entries that
were removed.

### Latent-space atlas: `atlas/data_v4/`

`atlas.json` holds every layer; the other files hold one task layer each (`ec`, `go_mf`, `go_bp`, `localisation`,
`homology`, `solubility`, `membrane`, `stability`, `fluorescence`, `affinity`). Each layer is a stratified sample of
about 2,000 proteins of the task's own rows. For every protein it gives the id, split and label; for every encoder the
UMAP coordinates (k = 10, seed 42, Procrustes-aligned to ESM2-150M), the k-NN purity with its chance level and a
five-fold linear probe accuracy; and between encoders the
debiased linear CKA and k-NN overlap, both computed on the full embeddings. `summary.json` holds the per-layer
summary. The atlas is built from the embeddings in `extract/embeddings/`.

### Input audit

`extract/encoder_input_audit.csv` has one row per encoder and dataset (`probing` for the CATH set, `stage1` for the
downstream set). Each row gives the relative change of the pooled embedding over the same 32 proteins under a 1 Å
coordinate jitter, a coordinate swap with another protein, a sequence shuffle and a rigid motion, the measured
`uses_coordinates`, `uses_residue_identity` and `invariant`, the declared inputs, and a `verdict`. The file also holds
the failing rows of EGNN and GeoBPE. `extract/encoder_atom_audit.csv` gives the change when each of the N, CA, C, O
and CB coordinates is perturbed and the resulting `atom_set`. `extract/encoder_sidechain_audit.csv` gives the same for
side-chain atoms.

## Loading example

```python
import numpy as np
import pandas as pd
from huggingface_hub import snapshot_download

repo = "Anonymoususer2223/ProtCompass_Embeddings"
task, encoder = "subcellular_localization", "esm2_650m"

# Download one encoder on one task and the task manifest (about 65 MB).
local = snapshot_download(repo, allow_patterns=[f"extract/embeddings/{task}/{encoder}/*",
                                                f"extract/manifest/{task}.csv"])
d = f"{local}/extract/embeddings/{task}/{encoder}"
splits = ("train", "valid", "test")
X = {s: np.load(f"{d}/{s}.npy") for s in splits}
ids = {s: open(f"{d}/ids_{s}.txt").read().split() for s in splits}

manifest = pd.read_csv(f"{local}/extract/manifest/{task}.csv")
manifest = manifest[manifest.stage1 == 1].set_index("id")
y = {s: manifest.loc[ids[s], "label"].to_numpy() for s in splits}

# A quick check with a logistic regression. The paper's heads (linear and MLP-2, selected on validation,
# three seeds) are in extract/score2.py of the code repository.
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().fit(X["train"])
clf = LogisticRegression(max_iter=1000).fit(scaler.transform(X["train"]), y["train"])
print("test accuracy", clf.score(scaler.transform(X["test"]), y["test"]))
```

Per-residue probing embeddings are large (up to 14.5 GB for ESM2-3B), so open them with `mmap_mode="r"`:

```python
import h5py
# In allow_patterns a * also matches '/', so the top-level files are named one by one.
local = snapshot_download(repo, allow_patterns=["probing_embeddings/esm2_650m/*", "probing_embeddings/offsets.npy",
                                                "probing_embeddings/lengths.npy", "probing_embeddings/ids.txt",
                                                "probing_data/labels.h5"])
R = np.load(f"{local}/probing_embeddings/esm2_650m/residue_fp16.npy", mmap_mode="r")
offsets = np.load(f"{local}/probing_embeddings/offsets.npy")
lengths = np.load(f"{local}/probing_embeddings/lengths.npy")
with h5py.File(f"{local}/probing_data/labels.h5", "r") as f:
    k = 10
    rows = slice(offsets[k], offsets[k] + lengths[k])
    residue_embeddings = R[rows]          # (lengths[k], 1280)
    dssp3 = f["residue/dssp3"][rows]      # 0 = H, 1 = E, 2 = C
    split = f["protein/split_superfamily"][k]
```

## Structures and datasets

The manifests give, for every row, the structure file the extraction read (`structure_path`, relative to the
directory that holds the code repository's `protcompass/` folder). The ESMFold structures of stability, fluorescence, solubility and PPI affinity are in the
[ProtEnv](https://huggingface.co/datasets/Anonymoususer2223/ProtEnv) dataset under `predicted_structures/`; the last
two components of `structure_path` (for example `predictions_fluorescence/fluor_20897.pdb`) are the member name in the
corresponding archive. The experimental (PDB), AlphaFold DB and CATH structures are public and are not redistributed.
The scripts that fetch them and build the manifests are in the code repository.

## Earlier release (removed)

An earlier release, produced by an earlier extraction pipeline for a previous version of this work, was stored
under `embeddings/` and `probing_IF/`. The current paper does not use it. An audit in September 2026 found defects
in it: structure encoders received placeholder or all-zero coordinates on several tasks, some entries were
degenerate or mislabelled, and four of its tasks had defects in their labels or metric. It was removed from this
repository. Every file the paper uses is in the current release.

## License

The embeddings, labels, manifests, audit files and atlas data produced for this work are released under the MIT
licence. The files under `checkpoints/` keep their upstream licences: MIT for the ProteinMPNN, GCPNet and PottsMPNN
files (the licence text is next to each file) and CC BY 4.0 for the GearNet weights (`checkpoints/gearnet_pretrained/NOTICE.md`).