Upload 16 files

Flash-README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- text-to-image
+- stable-diffusion
+- lora
+- diffusers
+- template:sd-lora
+base_model: stabilityai/stable-diffusion-xl-base-1.0
+license: cc-by-nc-nd-4.0
+---
+# ⚡ Flash Diffusion: FlashSDXL ⚡
+
+
+Flash Diffusion is a diffusion distillation method proposed in [Flash Diffusion: Accelerating Any Conditional
+Diffusion Model for Few Steps Image Generation](http://arxiv.org/abs/2406.02347) *by Clément Chadebec, Onur Tasar, Eyal Benaroche, and Benjamin Aubin.*
+This model is a **108M** LoRA distilled version of [SDXL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) model that is able to generate images in **4 steps**. The main purpose of this model is to reproduce the main results of the paper.
+See our [live demo](https://huggingface.co/spaces/jasperai/FlashPixart) and [official implementation](https://github.com/gojasper/flash-diffusion).
+
+
+<p align="center">
+   <img style="width:700px;" src="images/flash_sdxl.jpg">
+</p>
+
+# How to use?
+
+The model can be used using the `DiffusionPipeline` from `diffusers` library directly. It can allow reducing the number of required sampling steps to **4 steps**.
+
+```python
+from diffusers import DiffusionPipeline, LCMScheduler
+
+adapter_id = "jasperai/flash-sdxl"
+
+pipe = DiffusionPipeline.from_pretrained(
+  "stabilityai/stable-diffusion-xl-base-1.0",
+  use_safetensors=True,
+)
+
+pipe.scheduler = LCMScheduler.from_pretrained(
+  "stabilityai/stable-diffusion-xl-base-1.0",
+  subfolder="scheduler",
+  timestep_spacing="trailing",
+)
+pipe.to("cuda")
+
+# Fuse and load LoRA weights
+pipe.load_lora_weights(adapter_id)
+pipe.fuse_lora()
+
+prompt = "A raccoon reading a book in a lush forest."
+
+image = pipe(prompt, num_inference_steps=4, guidance_scale=0).images[0]
+```
+<p align="center">
+   <img style="width:400px;" src="images/raccoon.png">
+</p>
+
+# Training Details
+The model was trained for 20k iterations on 4 H100 GPUs (representing approximately a total of 176 GPU hours of training). Please refer to the [paper](http://arxiv.org/abs/2406.02347) for further parameters details. 
+
+**Metrics on COCO 2014 validation (Table 3)**
+  - FID-10k: 21.62 (4 NFE)
+  - CLIP Score: 0.327 (4 NFE)
+
+## Citation
+If you find this work useful or use it in your research, please consider citing us
+
+```bibtex
+@misc{chadebec2024flash,
+      title={Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation}, 
+      author={Clement Chadebec and Onur Tasar and Eyal Benaroche and Benjamin Aubin},
+      year={2024},
+      eprint={2406.02347},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+   
+## License
+This model is released under the the Creative Commons BY-NC license.

Hyper-SD(XL).md

@@ -0,0 +1,346 @@
+---
+license: openrail++
+library_name: diffusers
+inference: false
+tags:
+  - lora
+  - text-to-image
+  - stable-diffusion
+---
+
+# Hyper-SD
+Official Repository of the paper: *[Hyper-SD](https://arxiv.org/abs/2404.13686)*.
+
+Project Page: https://hyper-sd.github.io/
+
+![](./hypersd_tearser.jpg)
+
+
+## News🔥🔥🔥
+
+* May.13, 2024. 💥💥💥 The **12-Steps CFG-Preserved** [Hyper-SDXL-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-12steps-CFG-lora.safetensors) and [Hyper-SD15-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-12steps-CFG-lora.safetensors) is also available now(support 5~8 guidance scales), this could be more practical with better trade-off between performance and speed. Enjoy! 💥💥💥
+* Apr.30, 2024. Our **8-Steps CFG-Preserved** [Hyper-SDXL-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-8steps-CFG-lora.safetensors) and [Hyper-SD15-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-8steps-CFG-lora.safetensors) is available now(support 5~8 guidance scales), we strongly recommend making the 8-step CFGLora a standard configuration for all SDXL and SD15 models!!!
+* Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA 🥰 with TCDScheduler to inference on different steps are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Remember to install ⭕️ [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) in your `ComfyUI/custom_nodes` folder!!! You're encouraged to adjust the eta parameter to get better results 🌟!
+* Apr.26, 2024. Thanks to @[Pete](https://huggingface.co/pngwn) for contributing to our [scribble demo](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) with larger canvas right now 👏.
+* Apr.24, 2024. The ComfyUI [workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json) and [checkpoint](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors) on 1-Step SDXL UNet ✨ is also available! Don't forget ⭕️ to install the custom [scheduler](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) in your `ComfyUI/custom_nodes` folder!!!
+* Apr.23, 2024. ComfyUI workflows on N-Steps LoRAs are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Worth a try for creators 💥!
+* Apr.23, 2024. Our technical report 📚 is uploaded to [arXiv](https://arxiv.org/abs/2404.13686)! Many implementation details are provided and we welcome more discussions👏.
+* Apr.21, 2024. Hyper-SD ⚡️ is highly compatible and work well with different base models and controlnets. To clarify, we also append the usage example of controlnet [here](https://huggingface.co/ByteDance/Hyper-SD#controlnet-usage).
+* Apr.20, 2024. Our checkpoints and two demos 🤗 (i.e. [SD15-Scribble](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) and [SDXL-T2I](https://huggingface.co/spaces/ByteDance/Hyper-SDXL-1Step-T2I)) are publicly available on [HuggingFace Repo](https://huggingface.co/ByteDance/Hyper-SD).
+
+## Try our Hugging Face demos: 
+Hyper-SD Scribble demo host on [🤗 scribble](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) 
+
+Hyper-SDXL One-step Text-to-Image demo host on [🤗 T2I](https://huggingface.co/spaces/ByteDance/Hyper-SDXL-1Step-T2I)
+
+## Introduction
+
+Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques.
+In this repository, we release the models distilled from [SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [Stable-Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)。
+
+## Checkpoints
+
+* `Hyper-SDXL-Nstep-lora.safetensors`: Lora checkpoint, for SDXL-related models.
+* `Hyper-SD15-Nstep-lora.safetensors`: Lora checkpoint, for SD1.5-related models.
+* `Hyper-SDXL-1step-unet.safetensors`: Unet checkpoint distilled from SDXL-Base.
+
+## Text-to-Image Usage
+### SDXL-related models
+#### 2-Steps, 4-Steps, 8-steps LoRA
+Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.  
+```python
+import torch
+from diffusers import DiffusionPipeline, DDIMScheduler
+from huggingface_hub import hf_hub_download
+base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"
+repo_name = "ByteDance/Hyper-SD"
+# Take 2-steps lora as an example
+ckpt_name = "Hyper-SDXL-2steps-lora.safetensors"
+# Load model.
+pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")
+pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
+pipe.fuse_lora()
+# Ensure ddim scheduler timestep spacing set as trailing !!!
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
+# lower eta results in more detail
+prompt="a photo of a cat"
+image=pipe(prompt=prompt, num_inference_steps=2, guidance_scale=0).images[0]
+```
+
+#### Unified LoRA (support 1 to 8 steps inference)
+You can flexibly adjust the number of inference steps and eta value to achieve best performance. 
+```python
+import torch
+from diffusers import DiffusionPipeline, TCDScheduler
+from huggingface_hub import hf_hub_download
+base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"
+repo_name = "ByteDance/Hyper-SD"
+ckpt_name = "Hyper-SDXL-1step-lora.safetensors"
+# Load model.
+pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")
+pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
+pipe.fuse_lora()
+# Use TCD scheduler to achieve better image quality
+pipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)
+# Lower eta results in more detail for multi-steps inference
+eta=1.0
+prompt="a photo of a cat"
+image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, eta=eta).images[0]
+```
+
+#### 1-step SDXL Unet
+Only for the single step inference.
+```python
+import torch
+from diffusers import DiffusionPipeline, UNet2DConditionModel, LCMScheduler
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"
+repo_name = "ByteDance/Hyper-SD"
+ckpt_name = "Hyper-SDXL-1step-Unet.safetensors"
+# Load model.
+unet = UNet2DConditionModel.from_config(base_model_id, subfolder="unet").to("cuda", torch.float16)
+unet.load_state_dict(load_file(hf_hub_download(repo_name, ckpt_name), device="cuda"))
+pipe = DiffusionPipeline.from_pretrained(base_model_id, unet=unet, torch_dtype=torch.float16, variant="fp16").to("cuda")
+# Use LCM scheduler instead of ddim scheduler to support specific timestep number inputs
+pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
+# Set start timesteps to 800 in the one-step inference to get better results
+prompt="a photo of a cat"
+image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, timesteps=[800]).images[0]
+```
+
+
+### SD1.5-related models
+
+#### 2-Steps, 4-Steps, 8-steps LoRA
+Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
+```python
+import torch
+from diffusers import DiffusionPipeline, DDIMScheduler
+from huggingface_hub import hf_hub_download
+base_model_id = "runwayml/stable-diffusion-v1-5"
+repo_name = "ByteDance/Hyper-SD"
+# Take 2-steps lora as an example
+ckpt_name = "Hyper-SD15-2steps-lora.safetensors"
+# Load model.
+pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")
+pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
+pipe.fuse_lora()
+# Ensure ddim scheduler timestep spacing set as trailing !!!
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
+prompt="a photo of a cat"
+image=pipe(prompt=prompt, num_inference_steps=2, guidance_scale=0).images[0]
+```
+
+
+#### Unified LoRA (support 1 to 8 steps inference)
+You can flexibly adjust the number of inference steps and eta value to achieve best performance.
+```python
+import torch
+from diffusers import DiffusionPipeline, TCDScheduler
+from huggingface_hub import hf_hub_download
+base_model_id = "runwayml/stable-diffusion-v1-5"
+repo_name = "ByteDance/Hyper-SD"
+ckpt_name = "Hyper-SD15-1step-lora.safetensors"
+# Load model.
+pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")
+pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
+pipe.fuse_lora()
+# Use TCD scheduler to achieve better image quality
+pipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)
+# Lower eta results in more detail for multi-steps inference
+eta=1.0
+prompt="a photo of a cat"
+image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, eta=eta).images[0]
+```
+
+## ControlNet Usage
+### SDXL-related models
+
+#### 2-Steps, 4-Steps, 8-steps LoRA
+Take Canny Controlnet and 2-steps inference as an example:
+```python
+import torch
+from diffusers.utils import load_image
+import numpy as np
+import cv2
+from PIL import Image
+from diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline, AutoencoderKL, DDIMScheduler
+from huggingface_hub import hf_hub_download
+
+# Load original image
+image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/hf-logo.png")
+image = np.array(image)
+# Prepare Canny Control Image
+low_threshold = 100
+high_threshold = 200
+image = cv2.Canny(image, low_threshold, high_threshold)
+image = image[:, :, None]
+image = np.concatenate([image, image, image], axis=2)
+control_image = Image.fromarray(image)
+control_image.save("control.png")
+control_weight = 0.5  # recommended for good generalization
+
+# Initialize pipeline
+controlnet = ControlNetModel.from_pretrained(
+    "diffusers/controlnet-canny-sdxl-1.0",
+    torch_dtype=torch.float16
+)
+vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
+pipe = StableDiffusionXLControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, vae=vae, torch_dtype=torch.float16).to("cuda")
+
+pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SDXL-2steps-lora.safetensors"))
+# Ensure ddim scheduler timestep spacing set as trailing !!!
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
+pipe.fuse_lora()
+image = pipe("A chocolate cookie", num_inference_steps=2, image=control_image, guidance_scale=0, controlnet_conditioning_scale=control_weight).images[0]
+image.save('image_out.png')
+```
+
+#### Unified LoRA (support 1 to 8 steps inference)
+Take Canny Controlnet as an example:
+```python
+import torch
+from diffusers.utils import load_image
+import numpy as np
+import cv2
+from PIL import Image
+from diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline, AutoencoderKL, TCDScheduler
+from huggingface_hub import hf_hub_download
+
+# Load original image
+image = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/hf-logo.png")
+image = np.array(image)
+# Prepare Canny Control Image
+low_threshold = 100
+high_threshold = 200
+image = cv2.Canny(image, low_threshold, high_threshold)
+image = image[:, :, None]
+image = np.concatenate([image, image, image], axis=2)
+control_image = Image.fromarray(image)
+control_image.save("control.png")
+control_weight = 0.5  # recommended for good generalization
+
+# Initialize pipeline
+controlnet = ControlNetModel.from_pretrained(
+    "diffusers/controlnet-canny-sdxl-1.0",
+    torch_dtype=torch.float16
+)
+vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
+pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
+    "stabilityai/stable-diffusion-xl-base-1.0",
+    controlnet=controlnet, vae=vae, torch_dtype=torch.float16).to("cuda")
+
+# Load Hyper-SD15-1step lora
+pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SDXL-1step-lora.safetensors"))
+pipe.fuse_lora()
+# Use TCD scheduler to achieve better image quality
+pipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)
+# Lower eta results in more detail for multi-steps inference
+eta=1.0
+image = pipe("A chocolate cookie", num_inference_steps=4, image=control_image, guidance_scale=0, controlnet_conditioning_scale=control_weight, eta=eta).images[0]
+image.save('image_out.png')
+```
+
+### SD1.5-related models
+
+#### 2-Steps, 4-Steps, 8-steps LoRA
+Take Canny Controlnet and 2-steps inference as an example:
+```python
+import torch
+from diffusers.utils import load_image
+import numpy as np
+import cv2
+from PIL import Image
+from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, DDIMScheduler
+
+from huggingface_hub import hf_hub_download
+
+controlnet_checkpoint = "lllyasviel/control_v11p_sd15_canny"
+
+# Load original image
+image = load_image("https://huggingface.co/lllyasviel/control_v11p_sd15_canny/resolve/main/images/input.png")
+image = np.array(image)
+# Prepare Canny Control Image
+low_threshold = 100
+high_threshold = 200
+image = cv2.Canny(image, low_threshold, high_threshold)
+image = image[:, :, None]
+image = np.concatenate([image, image, image], axis=2)
+control_image = Image.fromarray(image)
+control_image.save("control.png")
+
+# Initialize pipeline
+controlnet = ControlNetModel.from_pretrained(controlnet_checkpoint, torch_dtype=torch.float16)
+pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16).to("cuda")
+pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SD15-2steps-lora.safetensors"))
+pipe.fuse_lora()
+# Ensure ddim scheduler timestep spacing set as trailing !!!
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
+image = pipe("a blue paradise bird in the jungle", num_inference_steps=2, image=control_image, guidance_scale=0).images[0]
+image.save('image_out.png')
+```
+
+
+#### Unified LoRA (support 1 to 8 steps inference)
+Take Canny Controlnet as an example:
+```python
+import torch
+from diffusers.utils import load_image
+import numpy as np
+import cv2
+from PIL import Image
+from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, TCDScheduler
+from huggingface_hub import hf_hub_download
+
+controlnet_checkpoint = "lllyasviel/control_v11p_sd15_canny"
+
+# Load original image
+image = load_image("https://huggingface.co/lllyasviel/control_v11p_sd15_canny/resolve/main/images/input.png")
+image = np.array(image)
+# Prepare Canny Control Image
+low_threshold = 100
+high_threshold = 200
+image = cv2.Canny(image, low_threshold, high_threshold)
+image = image[:, :, None]
+image = np.concatenate([image, image, image], axis=2)
+control_image = Image.fromarray(image)
+control_image.save("control.png")
+
+# Initialize pipeline
+controlnet = ControlNetModel.from_pretrained(controlnet_checkpoint, torch_dtype=torch.float16)
+pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16).to("cuda")
+# Load Hyper-SD15-1step lora
+pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SD15-1step-lora.safetensors"))
+pipe.fuse_lora()
+# Use TCD scheduler to achieve better image quality
+pipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)
+# Lower eta results in more detail for multi-steps inference
+eta=1.0
+image = pipe("a blue paradise bird in the jungle", num_inference_steps=1, image=control_image, guidance_scale=0, eta=eta).images[0]
+image.save('image_out.png')
+```
+## Comfyui Usage
+* `Hyper-SDXL-Nsteps-lora.safetensors`: [text-to-image workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-Nsteps-lora-workflow.json)
+* `Hyper-SD15-Nsteps-lora.safetensors`: [text-to-image workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SD15-Nsteps-lora-workflow.json)
+* `Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors`: [text-to-image workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json)
+  * **REQUIREMENT / INSTALL** for 1-Step SDXL UNet: Please install our [scheduler folder](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) into your `ComfyUI/custom_nodes` to enable sampling from 800 timestep instead of 999. 
+  * i.e. making sure the `ComfyUI/custom_nodes/ComfyUI-HyperSDXL1StepUnetScheduler` folder exist.
+  * For more details, please refer to our [technical report](https://arxiv.org/abs/2404.13686).
+* `Hyper-SD15-1step-lora.safetensors`: [text-to-image workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SD15-1step-unified-lora-workflow.json)
+* `Hyper-SDXL-1step-lora.safetensors`: [text-to-image workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-unified-lora-workflow.json)
+  * **REQUIREMENT / INSTALL** for 1-Step Unified LoRAs: Please install the [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) into your `ComfyUI/custom_nodes` to enable TCDScheduler with support of different inference steps (1~8) using single checkpoint.
+  * i.e. making sure the `ComfyUI/custom_nodes/ComfyUI-TCD` folder exist.
+  * You're encouraged to adjust the eta parameter in TCDScheduler to get better results.
+
+## Citation
+```bibtex
+@misc{ren2024hypersd,
+      title={Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis}, 
+      author={Yuxi Ren and Xin Xia and Yanzuo Lu and Jiacheng Zhang and Jie Wu and Pan Xie and Xing Wang and Xuefeng Xiao},
+      year={2024},
+      eprint={2404.13686},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```

Hyper-SDXL-1step-lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a75acf70ca40a9c8ab0a2dd1bf76174c64f9636e98809fe87a223616e3cc4d9
+size 393854592

Hyper-SDXL-2steps-lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ef7e0cc5aa10d89eb7cbb77ec15b83e6ae1a596f75a31bccb227cc6043702e1
+size 393854592

Hyper-SDXL-4steps-lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06c37b7cd5f5c5c2aa21deeb69c7b49b6124c4402b8dbd9e78fbf36ca3243b82
+size 393854592

PCM-README.md

@@ -0,0 +1,9 @@
+---
+library_name: diffusers
+pipeline_tag: text-to-image
+---
+# Phased Consistency Model
+
+LoRA weights of Stable Diffusion v1-5 for fast text-to-image generation.
+
+[[paper](https://huggingface.co/papers/2405.18407)] [[arXiv](https://arxiv.org/abs/2405.18407)]  [[code](https://github.com/G-U-N/Phased-Consistency-Model)] [[project page](https://g-u-n.github.io/projects/pcm)]

SLAM-README.md

@@ -0,0 +1,51 @@
+---
+library_name: diffusers
+base_model: stabilityai/stable-diffusion-xl-base-1.0
+tags:
+  - text-to-image
+license: apache-2.0
+inference: false
+---
+# Sub-path Linear Approximation Model (SLAM) LoRA: SDXL
+Paper: [https://arxiv.org/abs/2404.13903](https://arxiv.org/abs/2404.13903)<br/>
+Project Page: [https://subpath-linear-approx-model.github.io/](https://subpath-linear-approx-model.github.io/)<br/>
+The checkpoint is a distilled from [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) with our proposed Sub-path Linear Approximation Model, which reduces the number of inference steps to only between 2-4 steps.
+## Usage
+First, install the latest version of the Diffusers library as well as peft, accelerate and transformers.
+```bash
+pip install --upgrade pip
+pip install --upgrade diffusers transformers accelerate peft
+```
+We implement SLAM to be compatible with [LCMScheduler](https://huggingface.co/docs/diffusers/v0.22.3/en/api/schedulers/lcm#diffusers.LCMScheduler). You can use SLAM-LoRA just like you use LCM-LoRA.
+```python
+import torch
+from diffusers import LCMScheduler, AutoPipelineForText2Image
+
+model_id = "stabilityai/stable-diffusion-xl-base-1.0"
+adapter_id = "alimama-creative/slam-lora-sdxl"
+
+pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
+pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
+pipe.to("cuda")
+
+# load and fuse lcm lora
+pipe.load_lora_weights(adapter_id)
+pipe.fuse_lora()
+
+prompt = "A brown teddy bear holding a glass vase in front of a grave."
+
+image = pipe(prompt=prompt, num_inference_steps=4, guidance_scale=1.0).images[0]
+
+```
+
+
+Compare with latent-consistency/lcm-lora-sdxl.
+<img src='https://huggingface.co/alimama-creative/slam-lora-sdxl/resolve/main/sdxl_cmp.jpg'>
+
+---
+
+More examples:
+<img src='https://huggingface.co/alimama-creative/slam-lora-sdxl/resolve/main/slam-lora-sdxl.jpg'>
+
+
+

TCD-SDXL.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c777bc60abf41d3eb0fe405d23d73c280a020eea5adf97a82a141592c33feba
+size 393854624

flash-sdxl.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe2ca6e27c4c6087f50ef42772c45d7b0efbc471b76e422492403f9cae724d7
+size 371758976

pcm_sdxl_smallcfg_2step.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:242cbe4695fe3f2e248faa71cf53f2ccbf248a316973e4b2f38ab9e34f35a5ab
+size 393854624

pcm_sdxl_smallcfg_4step.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0bf40a7f280829195563486bec7253f043a06b1f218602b20901c367641023e
+size 393854624

sd_xl_turbo_lora_v1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a599c42a9f4f7494c7f410dbc0fd432cf0242720509e9d52fa41aac7a88d1b69
+size 787342192

sdxl-turbo-dpo-lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c31b841d4574f724fa9f2600b69fd30c81d2593d2e11a3b6bdf51ec8b8761e6d
+size 46615272

sdxl_lightning_2step_lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04fafc778385b24144498a8247498ae4fb7a69f702ea0566bdce2845a31fcc43
+size 393854592

sdxl_lightning_4step_lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf56cf2657efb15e465d81402ed481d1e11c4677e4bcce1bc11fe71ad8506b79
+size 393854592

slam-lora-sdxl.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22569a946b0db645aa3b8eb782c674c8e726a7cc0d655887c21fecf6dfe6ad91
+size 393854592