je-suis-tm/sayama_ai_lora_flux_nf4

Sayama Ai Lora Flux NF4

Prompt

Training With QLoRA: Sayama Ai smiles confidently for the camera in this outdoor portrait. Her striking red hair, long and wavy, complements a playful outfit: a white skirt and a pink and white striped top adorned with hearts. she radiates happiness. The blurred background hints at an urban location, possibly near a road. The photograph captures a moment of joy and self assuredness in a vibrant, natural setting.

Prompt

Training Without QLoRA: Sayama Ai smiles confidently for the camera in this outdoor portrait. Her striking red hair, long and wavy, complements a playful outfit: a white skirt and a pink and white striped top adorned with hearts. she radiates happiness. The blurred background hints at an urban location, possibly near a road. The photograph captures a moment of joy and self assuredness in a vibrant, natural setting.

Prompt

Testing With QLoRA: Sayama Ai looking directly at the viewer, wearing a champagne-colored silk camisole, matching silk pajama pants, and a white silk robe, holding a white teacup and saucer in both hands at chest level, standing confidently in a luxurious hotel suite. The setting features a large window behind her, offering a blurred view of a city skyline with bokeh lights, a neutral-toned armchair to her right, and a side table with a patterned vase holding pink flowers and a black book. To her left, there's a floor lamp with a cream shade, a sofa with a cushion, and a brown curtain. Photorealistic, High-definition photography Soft ambient lighting combining interior warmth and subtle outdoor bluish light Medium full shot, eye-level Elegant, sophisticated, serene, warm tones, natural glow --ar 16:9 --v 6. 0 --s 750 --c 0

Prompt

Testing Without QLoRA: Sayama Ai looking directly at the viewer, wearing a champagne-colored silk camisole, matching silk pajama pants, and a white silk robe, holding a white teacup and saucer in both hands at chest level, standing confidently in a luxurious hotel suite. The setting features a large window behind her, offering a blurred view of a city skyline with bokeh lights, a neutral-toned armchair to her right, and a side table with a patterned vase holding pink flowers and a black book. To her left, there's a floor lamp with a cream shade, a sofa with a cushion, and a brown curtain. Photorealistic, High-definition photography Soft ambient lighting combining interior warmth and subtle outdoor bluish light Medium full shot, eye-level Elegant, sophisticated, serene, warm tones, natural glow --ar 16:9 --v 6. 0 --s 750 --c 0

佐山愛 / さやまあい / Sayama Ai

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

The QLoRA fine-tuning process of sayama_ai_lora_flux_nf4 takes inspiration from this post (https://huggingface.co/blog/flux-qlora). The training was executed on a local computer with 1200 timesteps and the same parameters as the link mentioned above, which took around 8 hours on 8GB VRAM 4060. The peak VRAM usage was around 7.7GB. To avoid running low on VRAM, both transformers and text_encoder were quantized. The biggest challenge of training Japanese actresses is their photos used heavy filters to whiten and smoothen the skin. This practise severely distorts the training images which makes the result less convincing than Hollywood actresses. This training dataset contains a lot of face closeup which makes result more aligned with her actual face. The tradeoff is the overfitting problem of QLoRA which makes model more likely to ignore the prompt. All the images generated here are using the below parameters

• Height: 512
• Width: 512
• Guidance scale: 5
• Num inference steps: 20
• Max sequence length: 512
• Seed: 0

Usage

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel

text_encoder_4bit = T5EncoderModel.from_pretrained(
    "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="text_encoder_2",torch_dtype=torch.float16,)

transformer_4bit = FluxTransformer2DModel.from_pretrained(
        "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="transformer",torch_dtype=torch.float16,)

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16,
                                    transformer=transformer_4bit,text_encoder_2=text_encoder_4bit)

pipe.load_lora_weights("je-suis-tm/sayama_ai_lora_flux_nf4",
                       weight_name='pytorch_lora_weights.safetensors')

prompt="Sayama Ai looking directly at the viewer, wearing a champagne-colored silk camisole, matching silk pajama pants, and a white silk robe, holding a white teacup and saucer in both hands at chest level, standing confidently in a luxurious hotel suite. The setting features a large window behind her, offering a blurred view of a city skyline with bokeh lights, a neutral-toned armchair to her right, and a side table with a patterned vase holding pink flowers and a black book. To her left, there's a floor lamp with a cream shade, a sofa with a cushion, and a brown curtain. Photorealistic, High-definition photography Soft ambient lighting combining interior warmth and subtle outdoor bluish light Medium full shot, eye-level Elegant, sophisticated, serene, warm tones, natural glow --ar 16:9 --v 6. 0 --s 750 --c 0"

image = pipe(
            prompt,
            height=512,
            width=512,
            guidance_scale=5,
            num_inference_steps=20,
            max_sequence_length=512,
            generator=torch.Generator("cpu").manual_seed(0),            
        ).images[0]

image.save("sayama_ai_lora_flux_nf4.png")

Trigger words

You should use Sayama Ai to trigger the image generation.

Download model

Download them in the Files & versions tab.