---
license: other
license_name: 8f-ai-license-v1.0
license_link: https://huggingface.co/8Fai/license
tags:
- time-series-forecasting
- pytorch
- gqa
- rope
- swiglu
- revin
- patch-transformer
language:
- en
library_name: torch
---

# Chorous1

<p align="center">
  <img src="https://img.shields.io/badge/Parameters-100M%20%7C%2050M%20%7C%2027M-brightgreen?style=flat-square" />
  <img src="https://img.shields.io/badge/Architecture-Patch--Transformer-purple?style=flat-square" />
  <img src="https://img.shields.io/badge/License-8f--ai--license--v1.0-red?style=flat-square" />
</p>

> **Chorous1** is a suite of three high-performance, patch-based transformer models for multivariate time-series forecasting. Combining RevIN, MAE-style patch masking, and a Flatten Head architecture, Chorous1 delivers state-of-the-art accuracy on real-world benchmark data.

---

## Table of Contents

- [Model Variants](#model-variants)
- [Architecture](#architecture)
- [Quickstart](#quickstart)
- [Performance](#performance)
- [Limitations](#limitations)
- [License](#license)

---

## Model Variants

| Variant | Parameters | Hidden Size | Layers | Query Heads / KV Heads |
|---|---|---|---|---|
| `chorous1-100m` | ~100M | 768 | 12 | 12 / 4 |
| `chorous1-50m` | ~50M | 512 | 16 | 8 / 2 |
| `chorous1-27m` | ~27M | 384 | 16 | 6 / 2 |

---

## Architecture

| Component | Specification |
|---|---|
| Context Length | 512 steps |
| Forecast Horizon | 96 steps |
| Patch Size | 16 (non-overlapping) |
| Number of Patches | 32 |
| FFN Multiplier | 2.667× |
| Activation | SwiGLU |
| Positional Encoding | RoPE (θ = 500,000) |
| Normalization | RMSNorm |
| Masking Ratio | 25% (training only) |
| Loss Function | Huber Loss + MAE |
| Precision | bfloat16 |

### How It Works

**Stage 1 — Neural Encoding.** The transformer encoder processes patches of time-series data using RoPE and GQA to capture long-range temporal dependencies and periodic structure.

**Stage 2 — RevIN Normalization.** A reversible instance normalization layer removes mean and variance shifts from the input prior to processing, then restores them on the output — eliminating the distribution mismatch problem common in real-world deployments.

---

## Quickstart

```python
import torch
from safetensors.torch import load_file

# Replace "100m" with "50m" or "27m" as needed
weights = load_file("./chorous_checkpoint/100m/model.safetensors")
model.load_state_dict(weights)
model.eval()

# Input shape: [Batch, Channels, Time]
x = torch.randn(1, 7, 512)

with torch.no_grad():
    forecast = model(x)  # Output shape: [1, 7, 96]
```

---

## Performance

| Metric | `chorous1-100m` | `chorous1-50m` | `chorous1-27m` |
|---|---|---|---|
| Weights Size | ~200 MB | ~110 MB | ~65 MB |
| VRAM (Inference) | ~12 GB | ~8 GB | ~6 GB |

---

## Limitations

- **Fixed Forecast Horizon** — Optimized for 96-step forecasting. Modifying the output head for longer horizons may reduce accuracy.
- **Channel Count Constraint** — The RevIN layer is initialized using the maximum channel count from the training suite. Inputs exceeding this limit are not supported out of the box.
- **Patch Alignment Requirement** — Input context length must be an exact multiple of the patch size (16).

---

## License

Chorous1 is released under the [8f-ai-license-v1.0](https://huggingface.co/8Fai/license). Please review the full terms before use in production or commercial applications.