---
license: mit
language:
  - en
tags:
  - chess
  - ocr
  - handwriting-recognition
  - bilstm
  - ctc
  - pytorch
  - onnx
datasets:
  - BenjaminKost/processed_hcs
metrics:
  - accuracy
pipeline_tag: image-to-text
---

# Chess OCR BiLSTM

A BiLSTM neural network for recognizing handwritten chess moves from scoresheet images.

## Model Description

This model recognizes handwritten chess moves in Standard Algebraic Notation (SAN) from cropped move-box images extracted from chess scoresheets.

- **Architecture:** CNN frontend + Bidirectional LSTM + CTC loss
- **Input:** 64×256 grayscale image, normalized to [-1, 1]
- **Output:** Chess move string (e.g., "e4", "Nf3", "O-O", "Qxd7+")
- **Character set:** 28 classes (blank + K, Q, R, B, N, a-h, 1-8, x, +, #, =, O, -)

## Model Variants

| File | Size | Use Case |
|------|------|----------|
| `chess_ocr.onnx` | 41 MB | Browser inference (ONNX Runtime Web) |
| `chess_ocr_v47_complete.pth` | ~47 MB | Python/PyTorch inference |
| `best_chess_ocr_v47.pth` | ~127 MB | Resume training (includes optimizer state) |

## Usage (Browser - ONNX Runtime Web)

```javascript
import * as ort from 'onnxruntime-web';

// Load model from HuggingFace
const modelUrl = 'https://huggingface.co/GerhardTrippen/chess-ocr-bilstm/resolve/main/chess_ocr.onnx';
const session = await ort.InferenceSession.create(modelUrl);

// Preprocess image to 64x256 grayscale, normalize to [-1, 1]
const inputTensor = new ort.Tensor('float32', imageData, [1, 1, 64, 256]);

// Run inference
const results = await session.run({ input: inputTensor });
const output = results.output.data;

// Decode with CTC (greedy or beam search)
const move = ctcDecode(output, charset);
```

## Usage (Python/PyTorch)

```python
import torch
from PIL import Image
import numpy as np

# Load model
checkpoint = torch.load('chess_ocr_v47_complete.pth', map_location='cpu')
model = ChessOCRModel()  # See training notebook for architecture
model.load_state_dict(checkpoint['model_state_dict'])
model.eval()

idx_to_char = checkpoint['idx_to_char']

# Preprocess: resize to 256x64, grayscale, normalize
image = Image.open('move_cell.png').convert('L')
image = image.resize((256, 64))
img_array = np.array(image, dtype=np.float32)
img_array = (img_array / 255.0 - 0.5) / 0.5  # Normalize to [-1, 1]
tensor = torch.FloatTensor(img_array).unsqueeze(0).unsqueeze(0)

# Inference
with torch.no_grad():
    output = model(tensor)
    
# CTC greedy decode
predictions = output.argmax(dim=2).squeeze()
move = ''.join([idx_to_char[idx.item()] for idx in predictions if idx.item() != 0])
```

## Training

Trained on the [HCS (Handwritten Chess Scoresheet) dataset](https://huggingface.co/datasets/BenjaminKost/processed_hcs).

See `BiLSTM_v7.ipynb` for the complete training code.

**Training configuration:**
- Epochs: 50
- Batch size: 32
- Learning rate: 0.0005
- Optimizer: Adam
- Augmentation: 10:1 ratio (rotation ±5°, horizontal scale, shear, vertical shift, brightness/contrast)

**Key differences from original Eicher et al. approach:**
- Trained from scratch (no IAM pretraining)
- Modified preprocessing: no horizontal line removal, center-padded alignment
- Achieves comparable results: **82.9% MRA** vs 82.5% MRA (Eicher 2021)

## Acknowledgments & Citations

This model implements the architecture described in:

**Eicher, O., Farmer, D., Li, Y., & Majid, N. (2021).** "Handwritten Chess Scoresheet Recognition Using a Convolutional BiLSTM Network." *ICDAR 2021 Workshops*, pp. 245-259.  
[DOI: 10.1007/978-3-030-86198-8_18](https://doi.org/10.1007/978-3-030-86198-8_18)

**Majid, N., Eicher, O. (2022).** "Digitization of Handwritten Chess Scoresheets with a BiLSTM Network." *J. Imaging* 2022, 8, 31.  
[DOI: 10.3390/jimaging8020031](https://doi.org/10.3390/jimaging8020031)

**Dataset:**
- Original HCS dataset by Eicher, Farmer, Li, Majid
- HuggingFace hosting by [Benjamin Kost](https://huggingface.co/datasets/BenjaminKost/processed_hcs)

## License

MIT License - Free for personal and commercial use.

## Author

Trained and uploaded by **Gerhard Trippen** (2025).

## Citation

```bibtex
@misc{trippen2025chessocr,
  author = {Trippen, Gerhard},
  title = {Chess OCR BiLSTM: Handwritten Chess Scoresheet Recognition},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/GerhardTrippen/chess-ocr-bilstm}
}
```