Mask Generation
Transformers
Safetensors
falcon_perception
text-generation
falcon
segmentation
vision-language
open-vocabulary
custom_code
Eval Results
Instructions to use tiiuae/Falcon-Perception with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tiiuae/Falcon-Perception with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("mask-generation", model="tiiuae/Falcon-Perception", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("tiiuae/Falcon-Perception", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
update readme for RL post-trained version
Browse files
README.md
CHANGED
|
@@ -25,9 +25,9 @@ The model is built around a simple interface. Image patches and text tokens are
|
|
| 25 |
- PBench dataset: `tiiuae/PBench`
|
| 26 |
- OCR model: `tiiuae/Falcon-OCR`
|
| 27 |
|
| 28 |
-
## Falcon Perception
|
| 29 |
|
| 30 |
-
This revision
|
| 31 |
|
| 32 |
RL post-training brings two main benefits:
|
| 33 |
|
|
@@ -61,6 +61,19 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
| 61 |
)
|
| 62 |
```
|
| 63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
The inference API is identical to Falcon Perception (see below). The architecture and tokenizer are unchanged; only the weights differ.
|
| 65 |
|
| 66 |
## Quickstart
|
|
|
|
| 25 |
- PBench dataset: `tiiuae/PBench`
|
| 26 |
- OCR model: `tiiuae/Falcon-OCR`
|
| 27 |
|
| 28 |
+
## RL post-training for Falcon Perception (this revision)
|
| 29 |
|
| 30 |
+
This revision is an RL post-trained version of Falcon Perception, obtained by reinforcement learning (GRPO) post-training of Falcon Perception. Instead of the per-token likelihood used in SFT, the policy is aligned directly with detection metrics through a simple set-matching reward (a Hungarian-matched count that penalizes false negatives and false positives).
|
| 31 |
|
| 32 |
RL post-training brings two main benefits:
|
| 33 |
|
|
|
|
| 61 |
)
|
| 62 |
```
|
| 63 |
|
| 64 |
+
### Citation
|
| 65 |
+
|
| 66 |
+
If you use this version, please cite:
|
| 67 |
+
|
| 68 |
+
```bibtex
|
| 69 |
+
@article{chaybouti2026rlperception,
|
| 70 |
+
title={Falcon Perception-HD: High Density Perception via Reinforcement Learning},
|
| 71 |
+
author={Chaybouti, Sofian and Dahou, Yasser and Huynh, Ngoc Dung and Alami, Reda and Kuehne, Hilde},
|
| 72 |
+
journal={arXiv preprint arXiv:2608.18881},
|
| 73 |
+
year={2026}
|
| 74 |
+
}
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
The inference API is identical to Falcon Perception (see below). The architecture and tokenizer are unchanged; only the weights differ.
|
| 78 |
|
| 79 |
## Quickstart
|