Image-to-Image
Diffusers
flux
flux-kontext
lora
thermal
face-recognition
biometrics
rgb-to-thermal
synthetic-data
Instructions to use GbrlOl/thermal-fr-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use GbrlOl/thermal-fr-lora with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-Kontext-dev", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("GbrlOl/thermal-fr-lora") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Update README.md
Browse files
README.md
CHANGED
|
@@ -42,26 +42,36 @@ extra_gated_fields:
|
|
| 42 |
extra_gated_button_content: "Acknowledge terms and request access"
|
| 43 |
---
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
|
| 48 |
-
(RGB-to-TIR) facial translation** under an explicit image-to-image (I2I)
|
| 49 |
-
conditioning scheme. A real visible face is used as the structural and identity
|
| 50 |
-
anchor, while a fixed text prompt steers the thermal appearance, so that the
|
| 51 |
-
generated thermal face preserves the subject's identity rather than merely
|
| 52 |
-
looking realistic.
|
| 53 |
|
| 54 |
-
|
| 55 |
-
Data: FLUX.1 Kontext Image-to-Image Adaptation and Cross-Sensor Evaluation"*
|
| 56 |
-
and were used to generate **PUCV-TER-S**, a synthetic thermal face dataset.
|
| 57 |
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
> [License & intended use](#license--intended-use) below.
|
| 63 |
|
| 64 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
| | |
|
| 67 |
|---|---|
|
|
@@ -72,29 +82,29 @@ and were used to generate **PUCV-TER-S**, a synthetic thermal face dataset.
|
|
| 72 |
| **Adapter target** | All linear projections (attention + feed-forward) of the double-stream and single-stream blocks |
|
| 73 |
| **LoRA rank / alpha** | r = 128 / α = 128 |
|
| 74 |
| **Training pairs** | 50 synchronized visible–thermal pairs (PUCV-VTF) |
|
| 75 |
-
| **
|
| 76 |
|
| 77 |
-
## Intended use
|
| 78 |
|
| 79 |
-
- Generating **synthetic thermal face datasets** from consented visible-face
|
| 80 |
-
|
| 81 |
-
- Studying **synthetic-to-real transfer** and **cross-sensor** robustness in
|
| 82 |
-
|
| 83 |
|
| 84 |
-
**Out of scope / prohibited:** any commercial or production use; generating
|
| 85 |
-
|
| 86 |
-
|
| 87 |
|
| 88 |
-
## How to use
|
| 89 |
|
| 90 |
-
You must be authenticated on the Hugging Face Hub and have accepted the terms of
|
| 91 |
-
|
| 92 |
[`black-forest-labs/FLUX.1-Kontext-dev`](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
|
| 93 |
(which is itself gated).
|
| 94 |
|
| 95 |
```bash
|
| 96 |
huggingface-cli login
|
| 97 |
-
pip install -U diffusers transformers accelerate peft safetensors
|
| 98 |
```
|
| 99 |
|
| 100 |
```python
|
|
@@ -102,32 +112,33 @@ import torch
|
|
| 102 |
from diffusers import FluxKontextPipeline
|
| 103 |
from diffusers.utils import load_image
|
| 104 |
|
| 105 |
-
# 1)
|
| 106 |
pipe = FluxKontextPipeline.from_pretrained(
|
| 107 |
"black-forest-labs/FLUX.1-Kontext-dev",
|
| 108 |
torch_dtype=torch.bfloat16,
|
| 109 |
).to("cuda")
|
| 110 |
|
| 111 |
-
# 2) Attach this LoRA adapter
|
| 112 |
pipe.load_lora_weights(
|
| 113 |
-
"GbrlOl/
|
| 114 |
-
weight_name="
|
| 115 |
)
|
| 116 |
|
| 117 |
-
# 3)
|
| 118 |
visible = load_image("visible_face_512.png")
|
| 119 |
|
| 120 |
prompt = (
|
| 121 |
-
"convert this image to a high-contrast thermal (8-12
|
| 122 |
"monochrome grayscale, black background, preserve face structure, "
|
| 123 |
-
"no visible-spectrum colours
|
| 124 |
)
|
| 125 |
|
|
|
|
| 126 |
thermal = pipe(
|
| 127 |
image=visible,
|
| 128 |
prompt=prompt,
|
| 129 |
-
guidance_scale=
|
| 130 |
-
num_inference_steps=
|
| 131 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 132 |
).images[0]
|
| 133 |
|
|
@@ -136,81 +147,59 @@ thermal.save("thermal_face.png")
|
|
| 136 |
|
| 137 |
### Input preprocessing
|
| 138 |
|
| 139 |
-
For results consistent with the paper, preprocess visible images
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
|
|
|
| 143 |
|
| 144 |
-
## Training details
|
| 145 |
|
| 146 |
- **Base model:** FLUX.1 Kontext [dev] (flow-matching, I2I conditioning).
|
| 147 |
- **Adaptation:** LoRA (rank 128, α 128) on all attention and feed-forward linear
|
| 148 |
projections of the double-stream and single-stream blocks; base weights frozen.
|
| 149 |
-
- **Optimization:** learning rate 1e-4, batch size 1, 3000 steps, single fixed prompt.
|
| 150 |
-
- **Data:** 50 synchronized visible–thermal pairs from PUCV-VTF;
|
| 151 |
-
|
| 152 |
-
contrast-normalized to 8-bit via per-image min–max scaling.
|
| 153 |
- **Hardware:** 2× NVIDIA H100 (80 GB).
|
| 154 |
|
| 155 |
-
## Evaluation (summary)
|
| 156 |
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
paper for the full protocol, confidence intervals, and open-set results.
|
| 163 |
|
| 164 |
-
## Reproducibility & code
|
| 165 |
|
| 166 |
-
Training, generation
|
| 167 |
-
|
| 168 |
|
| 169 |
-
|
|
|
|
|
|
|
|
|
|
| 170 |
|
| 171 |
-
|
| 172 |
-
available from the corresponding author upon reasonable request, subject to the
|
| 173 |
-
consent conditions of the original captures.
|
| 174 |
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
This repository is labeled **`cc-by-nc-4.0`** (attribution, non-commercial).
|
| 178 |
-
Because the weights are a **Derivative of FLUX.1 Kontext [dev]**, the upstream
|
| 179 |
**[FLUX.1 [dev] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)**
|
| 180 |
-
and its Acceptable Use Policy also apply, and its restrictions (including
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
separate license from Black Forest Labs.
|
| 184 |
-
|
| 185 |
-
## Ethical considerations
|
| 186 |
-
|
| 187 |
-
The synthetic images derive from **consented** captures (PUCV-VTF) and are
|
| 188 |
-
intended solely for research on data scarcity in thermal biometrics.
|
| 189 |
-
Identity-preserving generative models can, in principle, be misused to fabricate
|
| 190 |
-
biometric samples; access is therefore gated, redistribution of the weights is
|
| 191 |
-
non-commercial and attribution-bound, and downstream users are expected to adopt
|
| 192 |
-
equivalent safeguards. No model capable of impersonating individuals outside the
|
| 193 |
-
enrolled cohort is distributed.
|
| 194 |
|
| 195 |
-
##
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
Image-to-Image Adaptation and Cross-Sensor Evaluation},
|
| 204 |
-
author = {Hermosilla, Gabriel and Jara, Ashley and Olmos, Gabriel and
|
| 205 |
-
Soto, Lisa and Cocca-Guardia, Giovanni},
|
| 206 |
-
journal = {IEEE Access (under review)},
|
| 207 |
-
year = {2026},
|
| 208 |
-
note = {Manuscript ID: Access-2026-19915}
|
| 209 |
-
}
|
| 210 |
-
```
|
| 211 |
|
| 212 |
-
## Acknowledgements
|
| 213 |
|
| 214 |
-
Funded by ANID (FONDECYT Grant 1240573; ANID Magíster Nacional 2025-22251171;
|
| 215 |
-
|
| 216 |
-
Valparaíso (PUCV), Chile.
|
|
|
|
| 42 |
extra_gated_button_content: "Acknowledge terms and request access"
|
| 43 |
---
|
| 44 |
|
| 45 |
+
<div align="center">
|
| 46 |
|
| 47 |
+
# 🔥 Thermal Face LoRA — FLUX.1 Kontext (I2I)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
+
**Visible → Thermal (RGB-to-TIR) facial translation that preserves subject identity**
|
|
|
|
|
|
|
| 50 |
|
| 51 |
+
[](https://creativecommons.org/licenses/by-nc/4.0/)
|
| 52 |
+
[](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
|
| 53 |
+
[](https://github.com/GbrlOl/thermal-fr-lora)
|
| 54 |
+
[-orange)](#-license--intended-use)
|
|
|
|
| 55 |
|
| 56 |
+
</div>
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
A LoRA adapter that specializes **FLUX.1 Kontext [dev]** for **visible-to-thermal
|
| 61 |
+
facial translation** under an explicit image-to-image (I2I) scheme. A real visible
|
| 62 |
+
face acts as the structural and identity anchor while a fixed text prompt steers the
|
| 63 |
+
thermal appearance — so the generated thermal face keeps the subject's identity
|
| 64 |
+
instead of merely looking realistic.
|
| 65 |
+
|
| 66 |
+
These weights were used to generate **PUCV-TER-S**, a synthetic thermal face dataset,
|
| 67 |
+
and accompany the code released at 👉 **[github.com/GbrlOl/thermal-fr-lora](https://github.com/GbrlOl/thermal-fr-lora)**.
|
| 68 |
+
|
| 69 |
+
> ⚠️ **Gated, non-commercial research release.** These weights are a **Derivative of
|
| 70 |
+
> FLUX.1 Kontext [dev]**; the upstream **FLUX.1 [dev] Non-Commercial License** and its
|
| 71 |
+
> Acceptable Use Policy apply in addition to the `cc-by-nc-4.0` label on this repo.
|
| 72 |
+
> See [License & intended use](#-license--intended-use).
|
| 73 |
+
|
| 74 |
+
## 📦 Model summary
|
| 75 |
|
| 76 |
| | |
|
| 77 |
|---|---|
|
|
|
|
| 82 |
| **Adapter target** | All linear projections (attention + feed-forward) of the double-stream and single-stream blocks |
|
| 83 |
| **LoRA rank / alpha** | r = 128 / α = 128 |
|
| 84 |
| **Training pairs** | 50 synchronized visible–thermal pairs (PUCV-VTF) |
|
| 85 |
+
| **Weights file** | `Flux_ter_50_redo_seed42.safetensors` (≈1.37 GB, bf16) |
|
| 86 |
|
| 87 |
+
## 🎯 Intended use
|
| 88 |
|
| 89 |
+
- Generating **synthetic thermal face datasets** from consented visible-face images,
|
| 90 |
+
for research on data scarcity in thermal-face recognition.
|
| 91 |
+
- Studying **synthetic-to-real transfer** and **cross-sensor** robustness in thermal
|
| 92 |
+
biometrics.
|
| 93 |
|
| 94 |
+
**Out of scope / prohibited:** any commercial or production use; generating thermal
|
| 95 |
+
representations of individuals without consent; impersonation; fabrication of biometric
|
| 96 |
+
samples; any unlawful, defamatory, or abusive use.
|
| 97 |
|
| 98 |
+
## 🚀 How to use
|
| 99 |
|
| 100 |
+
You must be authenticated on the Hugging Face Hub and have accepted the terms of **both**
|
| 101 |
+
this repository **and** the base model
|
| 102 |
[`black-forest-labs/FLUX.1-Kontext-dev`](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
|
| 103 |
(which is itself gated).
|
| 104 |
|
| 105 |
```bash
|
| 106 |
huggingface-cli login
|
| 107 |
+
pip install -U diffusers transformers accelerate peft safetensors facenet-pytorch
|
| 108 |
```
|
| 109 |
|
| 110 |
```python
|
|
|
|
| 112 |
from diffusers import FluxKontextPipeline
|
| 113 |
from diffusers.utils import load_image
|
| 114 |
|
| 115 |
+
# 1) Base FLUX.1 Kontext model
|
| 116 |
pipe = FluxKontextPipeline.from_pretrained(
|
| 117 |
"black-forest-labs/FLUX.1-Kontext-dev",
|
| 118 |
torch_dtype=torch.bfloat16,
|
| 119 |
).to("cuda")
|
| 120 |
|
| 121 |
+
# 2) Attach this LoRA adapter
|
| 122 |
pipe.load_lora_weights(
|
| 123 |
+
"GbrlOl/thermal-fr-lora",
|
| 124 |
+
weight_name="Flux_ter_50_redo_seed42.safetensors",
|
| 125 |
)
|
| 126 |
|
| 127 |
+
# 3) Preprocessed visible face (MTCNN-cropped, 512x512) as the I2I condition
|
| 128 |
visible = load_image("visible_face_512.png")
|
| 129 |
|
| 130 |
prompt = (
|
| 131 |
+
"convert this image to a high-contrast thermal (8-12 µm) infrared view, "
|
| 132 |
"monochrome grayscale, black background, preserve face structure, "
|
| 133 |
+
"no visible-spectrum colours"
|
| 134 |
)
|
| 135 |
|
| 136 |
+
# Deterministic generation settings used to build PUCV-TER-S
|
| 137 |
thermal = pipe(
|
| 138 |
image=visible,
|
| 139 |
prompt=prompt,
|
| 140 |
+
guidance_scale=4.0,
|
| 141 |
+
num_inference_steps=20,
|
| 142 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 143 |
).images[0]
|
| 144 |
|
|
|
|
| 147 |
|
| 148 |
### Input preprocessing
|
| 149 |
|
| 150 |
+
For results consistent with the paper, preprocess visible images as in training: detect
|
| 151 |
+
the face with **MTCNN** (first / highest-confidence detection, 10-pixel margin), crop, and
|
| 152 |
+
resize to **512×512**. The prompt above is fixed and was used for all generations. The full
|
| 153 |
+
generation and evaluation pipeline is on
|
| 154 |
+
[GitHub](https://github.com/GbrlOl/thermal-fr-lora).
|
| 155 |
|
| 156 |
+
## 🧪 Training details
|
| 157 |
|
| 158 |
- **Base model:** FLUX.1 Kontext [dev] (flow-matching, I2I conditioning).
|
| 159 |
- **Adaptation:** LoRA (rank 128, α 128) on all attention and feed-forward linear
|
| 160 |
projections of the double-stream and single-stream blocks; base weights frozen.
|
| 161 |
+
- **Optimization:** learning rate 1e-4, batch size 1, 3000 steps, single fixed prompt, seed 42.
|
| 162 |
+
- **Data:** 50 synchronized visible–thermal pairs from PUCV-VTF; thermal captured with a
|
| 163 |
+
FLIR Tau 2 640 (LWIR, 14-bit), contrast-normalized to 8-bit via per-image min–max scaling.
|
|
|
|
| 164 |
- **Hardware:** 2× NVIDIA H100 (80 GB).
|
| 165 |
|
| 166 |
+
## 📊 Evaluation (summary)
|
| 167 |
|
| 168 |
+
Synthetic thermal faces from this adapter preserve subject identity at **99.87% rank-1**
|
| 169 |
+
(d′ = 13.6) against real-thermal templates, versus 11.4% (Pix2Pix) and 3.5% (CycleGAN)
|
| 170 |
+
under an identical 50-pair budget. An ArcFace recognizer trained **only** on the synthetic
|
| 171 |
+
data transfers to real thermal benchmarks, including unseen-identity cross-sensor settings.
|
| 172 |
+
Full protocol, confidence intervals and open-set results are in the code repository.
|
|
|
|
| 173 |
|
| 174 |
+
## 🔁 Reproducibility & code
|
| 175 |
|
| 176 |
+
Training, generation and evaluation code, plus the generation prompt and configuration:
|
| 177 |
+
**[github.com/GbrlOl/thermal-fr-lora](https://github.com/GbrlOl/thermal-fr-lora)**
|
| 178 |
|
| 179 |
+
The synthetic **PUCV-TER-S** dataset and the source **PUCV-VTF** recordings are available
|
| 180 |
+
from the corresponding author upon reasonable request, subject to the consent conditions of
|
| 181 |
+
the original captures. With the released code and these weights, the synthetic dataset can
|
| 182 |
+
be regenerated exactly from the source visible images.
|
| 183 |
|
| 184 |
+
## 📜 License & intended use
|
|
|
|
|
|
|
| 185 |
|
| 186 |
+
This repository is labeled **`cc-by-nc-4.0`** (attribution, non-commercial). Because the
|
| 187 |
+
weights are a **Derivative of FLUX.1 Kontext [dev]**, the upstream
|
|
|
|
|
|
|
| 188 |
**[FLUX.1 [dev] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)**
|
| 189 |
+
and its Acceptable Use Policy also apply, and its restrictions (including non-commercial use
|
| 190 |
+
only) propagate to these weights. In case of any conflict, the more restrictive terms govern.
|
| 191 |
+
Commercial use of the base model requires a separate license from Black Forest Labs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 192 |
|
| 193 |
+
## 🔒 Ethical considerations
|
| 194 |
+
|
| 195 |
+
The synthetic images derive from **consented** captures (PUCV-VTF) and are intended solely
|
| 196 |
+
for research on data scarcity in thermal biometrics. Identity-preserving generative models
|
| 197 |
+
can, in principle, be misused to fabricate biometric samples; access is therefore gated,
|
| 198 |
+
redistribution is non-commercial and attribution-bound, and downstream users are expected to
|
| 199 |
+
adopt equivalent safeguards. No model capable of impersonating individuals outside the
|
| 200 |
+
enrolled cohort is distributed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 201 |
|
| 202 |
+
## 🙏 Acknowledgements
|
| 203 |
|
| 204 |
+
Funded by ANID (FONDECYT Grant 1240573; ANID Magíster Nacional 2025-22251171; ANID Doctorado
|
| 205 |
+
Nacional 2025-21251472), Pontificia Universidad Católica de Valparaíso (PUCV), Chile.
|
|
|