Instructions to use CIawevy/QwenImage-TextPecker-SQPA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use CIawevy/QwenImage-TextPecker-SQPA with PEFT:
Task type is invalid.
- Inference
- Notebooks
- Google Colab
- Kaggle
Improve model card for TextPecker Qwen-Image adapter
Browse filesHi! I'm Niels from the Hugging Face community science team. This PR improves the model card for the TextPecker-optimized Qwen-Image model.
Changes include:
- Added the `text-to-image` pipeline tag.
- Added the `apache-2.0` license based on the official repository.
- Linked the research paper and GitHub repository.
- Structured the description and usage information for better readability.
- Retained the existing sample usage code which demonstrates how to load the LoRA weights using `peft` and `diffusers`.
README.md
CHANGED
|
@@ -1,21 +1,30 @@
|
|
| 1 |
---
|
| 2 |
base_model: Qwen/Qwen-Image
|
| 3 |
library_name: peft
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
-
# Model Card for Model ID
|
| 6 |
|
| 7 |
-
|
| 8 |
-
This model is trained using Flow-GRPO with LoRA. We provide only the LoRA weights here, so you will need to download the Qwen-Image base model first.
|
| 9 |
|
| 10 |
-
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
- **Repository:** https://github.com/CIawevy/TextPecker/tree/main
|
| 17 |
-
- **Paper:** https://www.arxiv.org/pdf/2602.20903
|
| 18 |
-
## Uses
|
| 19 |
```python
|
| 20 |
import os
|
| 21 |
import torch
|
|
@@ -27,8 +36,8 @@ os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
|
|
| 27 |
os.environ["DIFFUSERS_DISABLE_NATIVE_ATTENTION"] = "1"
|
| 28 |
|
| 29 |
def load_model(model_path, ckpt_path=None, use_lora=True):
|
| 30 |
-
torch_dtype = torch.bfloat16
|
| 31 |
-
device = "cuda"
|
| 32 |
|
| 33 |
pipe = DiffusionPipeline.from_pretrained(
|
| 34 |
model_path,
|
|
@@ -60,7 +69,7 @@ def load_model(model_path, ckpt_path=None, use_lora=True):
|
|
| 60 |
|
| 61 |
model_id = "Qwen/Qwen-Image"
|
| 62 |
lora_ckpt_path = "CIawevy/QwenImage-TextPecker-SQPA"
|
| 63 |
-
device = "cuda"
|
| 64 |
|
| 65 |
negative_prompt = " "
|
| 66 |
aspect_ratios = {
|
|
@@ -87,4 +96,15 @@ image = pipe(
|
|
| 87 |
|
| 88 |
image.save("TextPecker_qwen_demo.png")
|
| 89 |
print("img has been saved to: TextPecker_qwen_demo.png")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
```
|
|
|
|
| 1 |
---
|
| 2 |
base_model: Qwen/Qwen-Image
|
| 3 |
library_name: peft
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
pipeline_tag: text-to-image
|
| 6 |
+
tags:
|
| 7 |
+
- lora
|
| 8 |
+
- visual-text-rendering
|
| 9 |
+
- text-to-image
|
| 10 |
---
|
|
|
|
| 11 |
|
| 12 |
+
# QwenImage-TextPecker-SQPA
|
|
|
|
| 13 |
|
| 14 |
+
This model is a LoRA adapter for [Qwen/Qwen-Image](https://huggingface.co/Qwen/Qwen-Image), trained using the TextPecker structural anomaly perceptive RL strategy. TextPecker is designed to enhance Visual Text Rendering (VTR) by quantifying and rewarding structural anomalies like distortion and misalignment.
|
| 15 |
|
| 16 |
+
- **Paper:** [TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering](https://arxiv.org/abs/2602.20903)
|
| 17 |
+
- **Repository:** [https://github.com/CIawevy/TextPecker](https://github.com/CIawevy/TextPecker)
|
| 18 |
+
- **Base Model:** [Qwen/Qwen-Image](https://huggingface.co/Qwen/Qwen-Image)
|
| 19 |
|
| 20 |
+
## Model Description
|
| 21 |
+
|
| 22 |
+
Visual Text Rendering (VTR) remains a critical challenge in text-to-image generation. Even advanced models frequently produce text with structural anomalies. TextPecker addresses this using a structural anomaly perceptive RL strategy that works with any text-to-image generator. When applied to Qwen-Image, it yields significant gains in structural fidelity and semantic alignment for text rendering.
|
| 23 |
+
|
| 24 |
+
## Usage
|
| 25 |
+
|
| 26 |
+
This repository provides only the LoRA weights (SQPA). You must download the `Qwen-Image` base model first to use this adapter.
|
| 27 |
|
|
|
|
|
|
|
|
|
|
| 28 |
```python
|
| 29 |
import os
|
| 30 |
import torch
|
|
|
|
| 36 |
os.environ["DIFFUSERS_DISABLE_NATIVE_ATTENTION"] = "1"
|
| 37 |
|
| 38 |
def load_model(model_path, ckpt_path=None, use_lora=True):
|
| 39 |
+
torch_dtype = torch.get_default_dtype() if not torch.cuda.is_available() else torch.bfloat16
|
| 40 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 41 |
|
| 42 |
pipe = DiffusionPipeline.from_pretrained(
|
| 43 |
model_path,
|
|
|
|
| 69 |
|
| 70 |
model_id = "Qwen/Qwen-Image"
|
| 71 |
lora_ckpt_path = "CIawevy/QwenImage-TextPecker-SQPA"
|
| 72 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 73 |
|
| 74 |
negative_prompt = " "
|
| 75 |
aspect_ratios = {
|
|
|
|
| 96 |
|
| 97 |
image.save("TextPecker_qwen_demo.png")
|
| 98 |
print("img has been saved to: TextPecker_qwen_demo.png")
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
## Citation
|
| 102 |
+
|
| 103 |
+
```bibtex
|
| 104 |
+
@article{zhu2026TextPecker,
|
| 105 |
+
title = {TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering},
|
| 106 |
+
author = {Zhu, Hanshen and Liu, Yuliang and Wu, Xuecheng and Wang, An-Lan and Feng, Hao and Yang, Dingkang and Feng, Chao and Huang, Can and Tang, Jingqun and Bai, Xiang},
|
| 107 |
+
journal = {arXiv preprint arXiv:2602.20903},
|
| 108 |
+
year = {2026}
|
| 109 |
+
}
|
| 110 |
```
|