GbrlOl commited on
Commit
afc40da
·
verified ·
1 Parent(s): 8549538

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +88 -99
README.md CHANGED
@@ -42,26 +42,36 @@ extra_gated_fields:
42
  extra_gated_button_content: "Acknowledge terms and request access"
43
  ---
44
 
45
- # PUCV Thermal FR — FLUX.1 Kontext I2I LoRA
46
 
47
- LoRA adapter that specializes **FLUX.1 Kontext [dev]** for **visible-to-thermal
48
- (RGB-to-TIR) facial translation** under an explicit image-to-image (I2I)
49
- conditioning scheme. A real visible face is used as the structural and identity
50
- anchor, while a fixed text prompt steers the thermal appearance, so that the
51
- generated thermal face preserves the subject's identity rather than merely
52
- looking realistic.
53
 
54
- These weights accompany the paper *"Thermal Face Recognition from Synthetic
55
- Data: FLUX.1 Kontext Image-to-Image Adaptation and Cross-Sensor Evaluation"*
56
- and were used to generate **PUCV-TER-S**, a synthetic thermal face dataset.
57
 
58
- > ⚠️ **Access & license.** This is a **gated, non-commercial research** release.
59
- > The weights are a **Derivative of FLUX.1 Kontext [dev]**; the upstream
60
- > **FLUX.1 [dev] Non-Commercial License** and its Acceptable Use Policy apply in
61
- > addition to the `cc-by-nc-4.0` label on this repository. See
62
- > [License & intended use](#license--intended-use) below.
63
 
64
- ## Model summary
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
  | | |
67
  |---|---|
@@ -72,29 +82,29 @@ and were used to generate **PUCV-TER-S**, a synthetic thermal face dataset.
72
  | **Adapter target** | All linear projections (attention + feed-forward) of the double-stream and single-stream blocks |
73
  | **LoRA rank / alpha** | r = 128 / α = 128 |
74
  | **Training pairs** | 50 synchronized visible–thermal pairs (PUCV-VTF) |
75
- | **Precision** | bf16 |
76
 
77
- ## Intended use
78
 
79
- - Generating **synthetic thermal face datasets** from consented visible-face
80
- images, for research on data scarcity in thermal-face recognition.
81
- - Studying **synthetic-to-real transfer** and **cross-sensor** robustness in
82
- thermal biometrics.
83
 
84
- **Out of scope / prohibited:** any commercial or production use; generating
85
- thermal representations of individuals without consent; impersonation;
86
- fabrication of biometric samples; any unlawful, defamatory, or abusive use.
87
 
88
- ## How to use
89
 
90
- You must be authenticated on the Hugging Face Hub and have accepted the terms of
91
- **both** this repository **and** the base model
92
  [`black-forest-labs/FLUX.1-Kontext-dev`](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
93
  (which is itself gated).
94
 
95
  ```bash
96
  huggingface-cli login
97
- pip install -U diffusers transformers accelerate peft safetensors
98
  ```
99
 
100
  ```python
@@ -102,32 +112,33 @@ import torch
102
  from diffusers import FluxKontextPipeline
103
  from diffusers.utils import load_image
104
 
105
- # 1) Load the base FLUX.1 Kontext model
106
  pipe = FluxKontextPipeline.from_pretrained(
107
  "black-forest-labs/FLUX.1-Kontext-dev",
108
  torch_dtype=torch.bfloat16,
109
  ).to("cuda")
110
 
111
- # 2) Attach this LoRA adapter (replace with your actual repo id / filename)
112
  pipe.load_lora_weights(
113
- "GbrlOl/pucv-thermal-flux-kontext-lora",
114
- weight_name="pucv_thermal_kontext_lora.safetensors",
115
  )
116
 
117
- # 3) Provide a preprocessed visible face (MTCNN-cropped, 512x512) as the I2I condition
118
  visible = load_image("visible_face_512.png")
119
 
120
  prompt = (
121
- "convert this image to a high-contrast thermal (8-12 um) infrared view, "
122
  "monochrome grayscale, black background, preserve face structure, "
123
- "no visible-spectrum colours."
124
  )
125
 
 
126
  thermal = pipe(
127
  image=visible,
128
  prompt=prompt,
129
- guidance_scale=2.5,
130
- num_inference_steps=28,
131
  generator=torch.Generator("cuda").manual_seed(42),
132
  ).images[0]
133
 
@@ -136,81 +147,59 @@ thermal.save("thermal_face.png")
136
 
137
  ### Input preprocessing
138
 
139
- For results consistent with the paper, preprocess visible images exactly as in
140
- training: detect the face with **MTCNN** (first / highest-confidence detection,
141
- 10-pixel margin), crop, and resize to **512×512**. The prompt above is fixed and
142
- was used for all generations.
 
143
 
144
- ## Training details
145
 
146
  - **Base model:** FLUX.1 Kontext [dev] (flow-matching, I2I conditioning).
147
  - **Adaptation:** LoRA (rank 128, α 128) on all attention and feed-forward linear
148
  projections of the double-stream and single-stream blocks; base weights frozen.
149
- - **Optimization:** learning rate 1e-4, batch size 1, 3000 steps, single fixed prompt.
150
- - **Data:** 50 synchronized visible–thermal pairs from PUCV-VTF; visible images
151
- captured with a PS3Eye sensor and thermal with a FLIR Tau 2 640 (LWIR, 14-bit),
152
- contrast-normalized to 8-bit via per-image min–max scaling.
153
  - **Hardware:** 2× NVIDIA H100 (80 GB).
154
 
155
- ## Evaluation (summary)
156
 
157
- Using the selected 50-pair configuration, synthetic thermal faces preserve
158
- subject identity at **99.87% rank-1** (d′ = 13.6) against real-thermal templates,
159
- versus 11.4% (Pix2Pix) and 3.5% (CycleGAN) under an identical 50-pair budget. An
160
- ArcFace recognizer trained **only** on the synthetic data transfers to real
161
- thermal benchmarks, including unseen-identity cross-sensor settings. See the
162
- paper for the full protocol, confidence intervals, and open-set results.
163
 
164
- ## Reproducibility & code
165
 
166
- Training, generation, and evaluation code, together with the generation prompt
167
- and configuration, are released here:
168
 
169
- - **Code:** `https://github.com/<your-org>/<your-repo>` ← *replace with your GitHub URL*
 
 
 
170
 
171
- The synthetic **PUCV-TER-S** dataset and the source **PUCV-VTF** recordings are
172
- available from the corresponding author upon reasonable request, subject to the
173
- consent conditions of the original captures.
174
 
175
- ## License & intended use
176
-
177
- This repository is labeled **`cc-by-nc-4.0`** (attribution, non-commercial).
178
- Because the weights are a **Derivative of FLUX.1 Kontext [dev]**, the upstream
179
  **[FLUX.1 [dev] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)**
180
- and its Acceptable Use Policy also apply, and its restrictions (including
181
- non-commercial use only) propagate to these weights. In case of any conflict,
182
- the more restrictive terms govern. Commercial use of the base model requires a
183
- separate license from Black Forest Labs.
184
-
185
- ## Ethical considerations
186
-
187
- The synthetic images derive from **consented** captures (PUCV-VTF) and are
188
- intended solely for research on data scarcity in thermal biometrics.
189
- Identity-preserving generative models can, in principle, be misused to fabricate
190
- biometric samples; access is therefore gated, redistribution of the weights is
191
- non-commercial and attribution-bound, and downstream users are expected to adopt
192
- equivalent safeguards. No model capable of impersonating individuals outside the
193
- enrolled cohort is distributed.
194
 
195
- ## Citation
196
-
197
- > Paper under review at IEEE Access (Manuscript ID: Access-2026-19915). Please
198
- > check this card for the final reference once published.
199
-
200
- ```bibtex
201
- @article{hermosilla2026thermalflux,
202
- title = {Thermal Face Recognition from Synthetic Data: FLUX.1 Kontext
203
- Image-to-Image Adaptation and Cross-Sensor Evaluation},
204
- author = {Hermosilla, Gabriel and Jara, Ashley and Olmos, Gabriel and
205
- Soto, Lisa and Cocca-Guardia, Giovanni},
206
- journal = {IEEE Access (under review)},
207
- year = {2026},
208
- note = {Manuscript ID: Access-2026-19915}
209
- }
210
- ```
211
 
212
- ## Acknowledgements
213
 
214
- Funded by ANID (FONDECYT Grant 1240573; ANID Magíster Nacional 2025-22251171;
215
- ANID Doctorado Nacional 2025-21251472), Pontificia Universidad Católica de
216
- Valparaíso (PUCV), Chile.
 
42
  extra_gated_button_content: "Acknowledge terms and request access"
43
  ---
44
 
45
+ <div align="center">
46
 
47
+ # 🔥 Thermal Face LoRA — FLUX.1 Kontext (I2I)
 
 
 
 
 
48
 
49
+ **Visible → Thermal (RGB-to-TIR) facial translation that preserves subject identity**
 
 
50
 
51
+ [![License: CC BY-NC 4.0](https://img.shields.io/badge/License-CC%20BY--NC%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc/4.0/)
52
+ [![Base model](https://img.shields.io/badge/base-FLUX.1--Kontext--dev-blue)](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
53
+ [![Code](https://img.shields.io/badge/code-GitHub-black?logo=github)](https://github.com/GbrlOl/thermal-fr-lora)
54
+ [![Gated](https://img.shields.io/badge/access-gated%20(research%20only)-orange)](#-license--intended-use)
 
55
 
56
+ </div>
57
+
58
+ ---
59
+
60
+ A LoRA adapter that specializes **FLUX.1 Kontext [dev]** for **visible-to-thermal
61
+ facial translation** under an explicit image-to-image (I2I) scheme. A real visible
62
+ face acts as the structural and identity anchor while a fixed text prompt steers the
63
+ thermal appearance — so the generated thermal face keeps the subject's identity
64
+ instead of merely looking realistic.
65
+
66
+ These weights were used to generate **PUCV-TER-S**, a synthetic thermal face dataset,
67
+ and accompany the code released at 👉 **[github.com/GbrlOl/thermal-fr-lora](https://github.com/GbrlOl/thermal-fr-lora)**.
68
+
69
+ > ⚠️ **Gated, non-commercial research release.** These weights are a **Derivative of
70
+ > FLUX.1 Kontext [dev]**; the upstream **FLUX.1 [dev] Non-Commercial License** and its
71
+ > Acceptable Use Policy apply in addition to the `cc-by-nc-4.0` label on this repo.
72
+ > See [License & intended use](#-license--intended-use).
73
+
74
+ ## 📦 Model summary
75
 
76
  | | |
77
  |---|---|
 
82
  | **Adapter target** | All linear projections (attention + feed-forward) of the double-stream and single-stream blocks |
83
  | **LoRA rank / alpha** | r = 128 / α = 128 |
84
  | **Training pairs** | 50 synchronized visible–thermal pairs (PUCV-VTF) |
85
+ | **Weights file** | `Flux_ter_50_redo_seed42.safetensors` (≈1.37 GB, bf16) |
86
 
87
+ ## 🎯 Intended use
88
 
89
+ - Generating **synthetic thermal face datasets** from consented visible-face images,
90
+ for research on data scarcity in thermal-face recognition.
91
+ - Studying **synthetic-to-real transfer** and **cross-sensor** robustness in thermal
92
+ biometrics.
93
 
94
+ **Out of scope / prohibited:** any commercial or production use; generating thermal
95
+ representations of individuals without consent; impersonation; fabrication of biometric
96
+ samples; any unlawful, defamatory, or abusive use.
97
 
98
+ ## 🚀 How to use
99
 
100
+ You must be authenticated on the Hugging Face Hub and have accepted the terms of **both**
101
+ this repository **and** the base model
102
  [`black-forest-labs/FLUX.1-Kontext-dev`](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
103
  (which is itself gated).
104
 
105
  ```bash
106
  huggingface-cli login
107
+ pip install -U diffusers transformers accelerate peft safetensors facenet-pytorch
108
  ```
109
 
110
  ```python
 
112
  from diffusers import FluxKontextPipeline
113
  from diffusers.utils import load_image
114
 
115
+ # 1) Base FLUX.1 Kontext model
116
  pipe = FluxKontextPipeline.from_pretrained(
117
  "black-forest-labs/FLUX.1-Kontext-dev",
118
  torch_dtype=torch.bfloat16,
119
  ).to("cuda")
120
 
121
+ # 2) Attach this LoRA adapter
122
  pipe.load_lora_weights(
123
+ "GbrlOl/thermal-fr-lora",
124
+ weight_name="Flux_ter_50_redo_seed42.safetensors",
125
  )
126
 
127
+ # 3) Preprocessed visible face (MTCNN-cropped, 512x512) as the I2I condition
128
  visible = load_image("visible_face_512.png")
129
 
130
  prompt = (
131
+ "convert this image to a high-contrast thermal (8-12 µm) infrared view, "
132
  "monochrome grayscale, black background, preserve face structure, "
133
+ "no visible-spectrum colours"
134
  )
135
 
136
+ # Deterministic generation settings used to build PUCV-TER-S
137
  thermal = pipe(
138
  image=visible,
139
  prompt=prompt,
140
+ guidance_scale=4.0,
141
+ num_inference_steps=20,
142
  generator=torch.Generator("cuda").manual_seed(42),
143
  ).images[0]
144
 
 
147
 
148
  ### Input preprocessing
149
 
150
+ For results consistent with the paper, preprocess visible images as in training: detect
151
+ the face with **MTCNN** (first / highest-confidence detection, 10-pixel margin), crop, and
152
+ resize to **512×512**. The prompt above is fixed and was used for all generations. The full
153
+ generation and evaluation pipeline is on
154
+ [GitHub](https://github.com/GbrlOl/thermal-fr-lora).
155
 
156
+ ## 🧪 Training details
157
 
158
  - **Base model:** FLUX.1 Kontext [dev] (flow-matching, I2I conditioning).
159
  - **Adaptation:** LoRA (rank 128, α 128) on all attention and feed-forward linear
160
  projections of the double-stream and single-stream blocks; base weights frozen.
161
+ - **Optimization:** learning rate 1e-4, batch size 1, 3000 steps, single fixed prompt, seed 42.
162
+ - **Data:** 50 synchronized visible–thermal pairs from PUCV-VTF; thermal captured with a
163
+ FLIR Tau 2 640 (LWIR, 14-bit), contrast-normalized to 8-bit via per-image min–max scaling.
 
164
  - **Hardware:** 2× NVIDIA H100 (80 GB).
165
 
166
+ ## 📊 Evaluation (summary)
167
 
168
+ Synthetic thermal faces from this adapter preserve subject identity at **99.87% rank-1**
169
+ (d′ = 13.6) against real-thermal templates, versus 11.4% (Pix2Pix) and 3.5% (CycleGAN)
170
+ under an identical 50-pair budget. An ArcFace recognizer trained **only** on the synthetic
171
+ data transfers to real thermal benchmarks, including unseen-identity cross-sensor settings.
172
+ Full protocol, confidence intervals and open-set results are in the code repository.
 
173
 
174
+ ## 🔁 Reproducibility & code
175
 
176
+ Training, generation and evaluation code, plus the generation prompt and configuration:
177
+ **[github.com/GbrlOl/thermal-fr-lora](https://github.com/GbrlOl/thermal-fr-lora)**
178
 
179
+ The synthetic **PUCV-TER-S** dataset and the source **PUCV-VTF** recordings are available
180
+ from the corresponding author upon reasonable request, subject to the consent conditions of
181
+ the original captures. With the released code and these weights, the synthetic dataset can
182
+ be regenerated exactly from the source visible images.
183
 
184
+ ## 📜 License & intended use
 
 
185
 
186
+ This repository is labeled **`cc-by-nc-4.0`** (attribution, non-commercial). Because the
187
+ weights are a **Derivative of FLUX.1 Kontext [dev]**, the upstream
 
 
188
  **[FLUX.1 [dev] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)**
189
+ and its Acceptable Use Policy also apply, and its restrictions (including non-commercial use
190
+ only) propagate to these weights. In case of any conflict, the more restrictive terms govern.
191
+ Commercial use of the base model requires a separate license from Black Forest Labs.
 
 
 
 
 
 
 
 
 
 
 
192
 
193
+ ## 🔒 Ethical considerations
194
+
195
+ The synthetic images derive from **consented** captures (PUCV-VTF) and are intended solely
196
+ for research on data scarcity in thermal biometrics. Identity-preserving generative models
197
+ can, in principle, be misused to fabricate biometric samples; access is therefore gated,
198
+ redistribution is non-commercial and attribution-bound, and downstream users are expected to
199
+ adopt equivalent safeguards. No model capable of impersonating individuals outside the
200
+ enrolled cohort is distributed.
 
 
 
 
 
 
 
 
201
 
202
+ ## 🙏 Acknowledgements
203
 
204
+ Funded by ANID (FONDECYT Grant 1240573; ANID Magíster Nacional 2025-22251171; ANID Doctorado
205
+ Nacional 2025-21251472), Pontificia Universidad Católica de Valparaíso (PUCV), Chile.