add model card
Browse files
README.md
CHANGED
|
@@ -1,3 +1,139 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: ESFM/ESFM_s_nm
|
| 4 |
+
tags:
|
| 5 |
+
- weather-forecasting
|
| 6 |
+
- climate
|
| 7 |
+
- earth-system
|
| 8 |
+
- ensemble
|
| 9 |
+
- probabilistic-forecasting
|
| 10 |
+
- era5
|
| 11 |
+
- pytorch
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Model Card for ESFM/ESFM_s_nm_ens
|
| 15 |
+
|
| 16 |
+
Eight-member probabilistic ESFM small checkpoint obtained by finetuning the complete-input deterministic ERA5 model. Ensemble members share one backbone pass and are conditioned through adaptive layer normalization.
|
| 17 |
+
|
| 18 |
+
> **Checkpoint selection:** Probabilistic six-hour global forecasting from complete ERA5-style inputs when an eight-member predictive ensemble is required.
|
| 19 |
+
|
| 20 |
+
## Model Details
|
| 21 |
+
|
| 22 |
+
- **Developed by:** The ESFM research team, with the full contributor and author lists linked below.
|
| 23 |
+
- **Shared by:** [ESFM on Hugging Face](https://huggingface.co/ESFM)
|
| 24 |
+
- **Model type:** Probabilistic eight-member ERA5 ensemble (ESFM_s+); modified 3D Swin-UNet encoder-decoder
|
| 25 |
+
- **Model size:** Approximately 115 million parameters plus ensemble-conditioning parameters
|
| 26 |
+
- **Masking protocol:** No observation masking in the base/final ensemble lineage
|
| 27 |
+
- **Ensemble members:** 8
|
| 28 |
+
- **Forecast lead time:** 6 hours per forecast step
|
| 29 |
+
- **License:** MIT
|
| 30 |
+
- **Repository:** https://huggingface.co/ESFM/ESFM_s_nm_ens
|
| 31 |
+
|
| 32 |
+
### Model Sources
|
| 33 |
+
|
| 34 |
+
- **Code:** https://github.com/swiss-ai/ESFM
|
| 35 |
+
- **Paper:** https://arxiv.org/abs/2605.00850
|
| 36 |
+
- **Project page:** https://swiss-ai.github.io/ESFM/
|
| 37 |
+
|
| 38 |
+
The paper is currently available as an arXiv preprint.
|
| 39 |
+
|
| 40 |
+
## Uses
|
| 41 |
+
|
| 42 |
+
### Direct Use
|
| 43 |
+
|
| 44 |
+
Probabilistic six-hour global forecasting from complete ERA5-style inputs when an eight-member predictive ensemble is required.
|
| 45 |
+
|
| 46 |
+
### Downstream Use
|
| 47 |
+
|
| 48 |
+
Uncertainty research, ensemble calibration studies, and autoregressive ensemble experiments with the released ESFM rollout code.
|
| 49 |
+
|
| 50 |
+
### Out-of-Scope Use
|
| 51 |
+
|
| 52 |
+
Not intended for sparse or missing inputs, calibrated decision support without local verification, or the assumption that eight members fully characterize tail risk.
|
| 53 |
+
|
| 54 |
+
## Bias, Risks, and Limitations
|
| 55 |
+
|
| 56 |
+
The model inherits ERA5 biases and the complete-input assumptions of `ESFM_s_nm`. The manuscript notes that operational ensemble systems pull ahead at longer lead times; ESFM reinitializes each autoregressive step from the previous ensemble mean, trading cost for long-range accuracy.
|
| 57 |
+
|
| 58 |
+
All ESFM checkpoints are research artifacts. Validate forecasts for the target variables, stations or regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.
|
| 59 |
+
|
| 60 |
+
## How to Get Started
|
| 61 |
+
|
| 62 |
+
The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct ESFM with the matching config and load the state dictionary. The released notebook demonstrates checkpoint download and architecture construction.
|
| 63 |
+
|
| 64 |
+
```python
|
| 65 |
+
from huggingface_hub import hf_hub_download
|
| 66 |
+
|
| 67 |
+
model_name = "ESFM_s_nm_ens"
|
| 68 |
+
weights_path = hf_hub_download(
|
| 69 |
+
repo_id=f"ESFM/{model_name}",
|
| 70 |
+
filename=f"{model_name}.safetensors",
|
| 71 |
+
)
|
| 72 |
+
print(weights_path)
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
Set `EXPERIMENT_NAME = "ESFM_s_nm_ens"` in the released inference notebook. The config is `configs/config_ESFM_s_nm_ens.yaml`; preserve `num_ensemble: 8`, `num_max_ensembles: 8`, and the matching ensemble conditioning settings.
|
| 76 |
+
|
| 77 |
+
Clone the implementation first:
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
git clone https://github.com/swiss-ai/ESFM.git
|
| 81 |
+
cd ESFM
|
| 82 |
+
```
|
| 83 |
+
|
| 84 |
+
## Training Details
|
| 85 |
+
|
| 86 |
+
### Training Data
|
| 87 |
+
|
| 88 |
+
WeatherBench2 ERA5 at 0.25-degree resolution. The ensemble is initialized from the no-masking deterministic `ESFM_s_nm` checkpoint, whose ERA5 training data span 1979 through 2020.
|
| 89 |
+
|
| 90 |
+
Preprocessing and the exact variable registry are documented in the [ESFM repository](https://github.com/swiss-ai/ESFM), [preprocessing repository](https://github.com/swiss-ai/SwissClim_data_processing_scripts), and [preprint](https://arxiv.org/abs/2605.00850).
|
| 91 |
+
|
| 92 |
+
### Training Procedure
|
| 93 |
+
|
| 94 |
+
Finetuned for 10,000 steps with eight members on 16 GPUs. The manuscript specifies an equally weighted sum of almost-fair CRPS and ensemble-mean MAE; all members share one backbone evaluation and use member-specific AdaLN-Zero conditioning.
|
| 95 |
+
|
| 96 |
+
- **Nominal architecture:** ESFM small, approximately 115M parameters
|
| 97 |
+
- **Software environment:** PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
|
| 98 |
+
- **Training regime:** Lightning `precision="32-true"` with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through `torch.autocast(dtype=torch.bfloat16)`.
|
| 99 |
+
|
| 100 |
+
## Evaluation
|
| 101 |
+
|
| 102 |
+
The manuscript evaluates ensemble-mean MAE and continuous ranked probability score. It reports competitive behavior at short lead times, while detailed and evolving comparisons remain in the linked preprint.
|
| 103 |
+
|
| 104 |
+
Detailed numerical results are intentionally not copied into this card.
|
| 105 |
+
|
| 106 |
+
## Technical Specifications
|
| 107 |
+
|
| 108 |
+
ESFM uses variable-specific tokenization, axial attention across variables, perceiver aggregation, a 3D Swin-UNet backbone, and a decoder queried at target pressure levels. Missing patches are represented by learnable NaN tokens. Resolution-specific tokenizers and station mapping are enabled for the relevant sparse-data configs. The ensemble checkpoint additionally applies member-conditioned AdaLN-Zero after the backbone.
|
| 109 |
+
|
| 110 |
+
## Environmental Impact
|
| 111 |
+
|
| 112 |
+
- **Hardware type:** NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
|
| 113 |
+
- **Total training time:** 20 hours
|
| 114 |
+
- **Compute location:** Training used CSCS Alps infrastructure.
|
| 115 |
+
|
| 116 |
+
## Citation
|
| 117 |
+
|
| 118 |
+
```bibtex
|
| 119 |
+
@misc{ozdemir2026esfm,
|
| 120 |
+
title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
|
| 121 |
+
author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
|
| 122 |
+
year={2026},
|
| 123 |
+
eprint={2605.00850},
|
| 124 |
+
archivePrefix={arXiv},
|
| 125 |
+
primaryClass={physics.ao-ph},
|
| 126 |
+
url={https://arxiv.org/abs/2605.00850}
|
| 127 |
+
}
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
## More Information
|
| 131 |
+
|
| 132 |
+
- [ESFM code and configurations](https://github.com/swiss-ai/ESFM)
|
| 133 |
+
- [Current ESFM preprint](https://arxiv.org/abs/2605.00850)
|
| 134 |
+
- [Data preprocessing scripts](https://github.com/swiss-ai/SwissClim_data_processing_scripts)
|
| 135 |
+
- [Project page](https://swiss-ai.github.io/ESFM/)
|
| 136 |
+
|
| 137 |
+
## Model Card Contact
|
| 138 |
+
|
| 139 |
+
Firat Ozdemir: `firat.ozdemir@sdsc.ethz.ch`
|