Model Card for ESFM/ESFM_s_wm_ci_prepre

CMIP6-pretraining checkpoint used to initialize the ESFM_s,ci ablation lineage. It was trained across eight climate-model simulations at different resolutions with the missing-data masking protocol.

Checkpoint selection: Use as an initialization for climate-model transfer or to reproduce the CMIP6-initialized ERA5 ablation. It is not the recommended direct ERA5 endpoint.

Model Details

• Developed by: The ESFM research team, with the full contributor and author lists linked below.
• Shared by: ESFM on Hugging Face
• Model type: CMIP6 initialization checkpoint (ESFM_s,ci lineage); modified 3D Swin-UNet encoder-decoder
• Model size: Approximately 115 million parameters
• Masking protocol: Variable, pressure-level, and spatial masking
• Forecast lead time: 6-hour forecasting objective
• License: MIT
• Repository: https://huggingface.co/ESFM/ESFM_s_wm_ci_prepre

Model Sources

• Code: https://github.com/swiss-ai/ESFM
• Paper: https://arxiv.org/abs/2605.00850
• Project page: https://swiss-ai.github.io/ESFM/

The paper is currently available as an arXiv preprint.

Uses

Direct Use

Use as an initialization for climate-model transfer or to reproduce the CMIP6-initialized ERA5 ablation. It is not the recommended direct ERA5 endpoint.

Downstream Use

Base checkpoint for ESFM_s_wm_ci_pre and the CMIP6-initialization transfer studies.

Out-of-Scope Use

Do not assume ERA5-calibrated forecast accuracy or the final variable set before downstream finetuning.

Bias, Risks, and Limitations

CMIP6 simulations have model-specific biases, resolutions, variables, and pressure-level availability. This checkpoint reflects those simulation distributions and is not independently validated as an operational forecast model.

All ESFM checkpoints are research artifacts. Users should validate forecasts for their variables, regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.

How to Get Started

The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct the ESFM architecture with the matching repository config, then load the state dictionary. The released notebook contains the complete download, model-construction, normalization, and inference workflow.

git clone https://github.com/swiss-ai/ESFM.git
cd ESFM
# Open notebooks/inference_ESFMs_on_ERA5.ipynb

In the notebook, set:

EXPERIMENT_NAME = "ESFM_s_wm_ci_prepre"

To download the weights directly:

from huggingface_hub import hf_hub_download

model_name = "ESFM_s_wm_ci_prepre"
weights_path = hf_hub_download(
    repo_id=f"ESFM/{model_name}",
    filename=f"{model_name}.safetensors",
)
print(weights_path)

Use configs/config_ESFM_s_wm_ci_prepre.yaml with the released training/inference code. The config lists the exact eight dataset identifiers: cmcc, miroc, taiesm1, nesm, awi, mpi, ec, and mri.

Training Details

Training Data

Eight CMIP6 simulations: CMCC-CM2-HR4, MIROC6, TaiESM1, NESM3, AWI-ESM, MPI-M, EC-Earth3, and MRI-ESM2. The manuscript reports resolutions from approximately 0.7 to 1.9 degrees and all available pressure levels.

Dataset preprocessing and the exact variable registry are documented in the ESFM repository and preprint.

Training Procedure

Trained for 91,500 steps on eight CMIP6 datasets with the missing-data masking protocol. This experiment was run on 16 GPUs; the manuscript rounds the training duration to approximately 92,000 steps.

• Training objective: Six-hour forecast learning, as specified above
• Nominal architecture: ESFM small, approximately 115M parameters
• Software environment: PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
• Training regime: Lightning precision="32-true" with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through torch.autocast(dtype=torch.bfloat16).

Evaluation

The manuscript evaluates downstream CMIP6-initialized models on ERA5 and transfer tasks. It reports improved initialization over random starts but lower performance than knowledge distillation for the main ERA5 comparison.

The manuscript uses held-out temporal data and reports task-appropriate metrics: latitude-weighted MAE and Pearson correlation for gridded deterministic forecasts, relative MAE for MODIS comparisons, station metrics for station models, and CRPS for ensembles. Detailed values are intentionally not copied into this card.

Technical Specifications

ESFM retains Aurora's 3D Swin-UNet backbone and adds variable-specific tokenization, axial attention across variables, perceiver aggregation across variables and pressure levels, learnable NaN tokens for missing patches, resolution-specific tokenizers where configured, and a decoder queried at target pressure levels. The small configuration uses a 256-dimensional embedding and approximately 115M parameters.

Environmental Impact

• Hardware type: NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
• Total training time: 53 hours
• Compute location: Training used CSCS Alps infrastructure.

Citation

@misc{ozdemir2026esfm,
  title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
  author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
  year={2026},
  eprint={2605.00850},
  archivePrefix={arXiv},
  primaryClass={physics.ao-ph},
  url={https://arxiv.org/abs/2605.00850}
}

More Information

• ESFM code and configurations
• Current ESFM preprint
• Data preprocessing scripts
• Project page

Model Card Contact

Firat Ozdemir: firat.ozdemir@sdsc.ethz.ch