laion/FLUX.2-klein-base-9B-vivarium-loras

Vivarium character and background LoRAs for FLUX.2-klein-base-9B

Two separate rank-64 style adapters for the undistilled FLUX.2-klein-base-9B:

Adapter	Trigger	Typical output	Strength	Selected training step
Characters	vvar_char	Warm painted animation characters on a solid green background;832×1248	1.0	1500
Backgrounds	vvar_bg	Painted empty locations;1376×768	1.0	2000

Use 40–50 inference steps and guidance_scale=4.0. All examples below use 50 steps,CFG4,and the undistilled base checkpoint. Load one adapter per pipeline according to the category.

Examples

The character composite preview has been omitted after the classifier recheck; the individual examples remain available below.

The example manifest contains exact prompts,seeds,dimensions,adapter checksums and generation settings. All images are model-generated examples, not source training images.

Install and authenticate

Create a Python environment and install requirements.txt. The pinned Diffusers source revision is the one tested for this model.

pip install -r requirements.txt
hf auth login

Accept the access conditions for the base model with the same account. The full base-model weights are downloaded separately. Our tests used a CUDA A100 80 GB with BF16 and no quantization.

Generate characters

import torch
from diffusers import Flux2KleinPipeline
from scripts.vivarium_prompt_templates import build_prompt

pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-base-9B",
    revision="32773329fbe7e81a90ef971740e8ba4b0364ecf3",
    torch_dtype=torch.bfloat16,
).to("cuda")
pipe.load_lora_weights(
    "laion/FLUX.2-klein-base-9B-vivarium-loras",
    subfolder="characters",
    weight_name="pytorch_lora_weights.safetensors",
    adapter_name="vivarium",
)
pipe.set_adapters("vivarium", adapter_weights=1.0)
image = pipe(
    prompt=build_prompt("characters"),
    width=832, height=1248,
    num_inference_steps=50, guidance_scale=4.0,
    max_sequence_length=256,
    generator=torch.Generator("cuda").manual_seed(28008),
).images[0]
image.save("character.png")

For locations, create a fresh pipeline, load subfolder="backgrounds", use trigger vvar_bg and size 1376×768. Example:

vvar_bg, a peaceful small railway station in winter, a snow-covered platform,
warm lit waiting room, wooden canopy and distant snowy hills.
Warm hand-painted anime environment illustration, soft painterly shading,
rich architectural details. Wide establishing view of an empty location.
No people, no humans, no characters, no text, no watermark.

The included infer.py supports both categories:

python infer.py characters --output character.png
python infer.py backgrounds --output location.png
python infer.py characters --description "a friendly elderly gardener with silver hair, a short beard, mustard cardigan and brown trousers" --output gardener.png
python infer.py backgrounds --description "an empty coastal library, wooden bookshelves and tall windows overlooking the sea" --output library.png

Recommended style: ghibli-warm-painted

The recommended default is ghibli-warm-painted, selected after comparing painted, soft-line, eye-proportion and CFG variants. Use this exact character style clause:

Studio Ghibli-inspired grounded coming-of-age animation, delicate expressive hand-drawn lines,
rich warm watercolor-like painted shading, gentle luminous colors,
restrained almond-shaped eyes and proportionate faces.

For a late teenager, use a mature-looking 17-year-old young man or a mature-looking 16-year-old young woman and describe an elongated oval face, modest almond-shaped eyes, a subtly defined jaw and a proportionate head. These examples keep LoRA strength1.0 and CFG4. The exact complete prompts are in the example manifest. This is a prompting choice on the same style adapter, not a separately trained Ghibli adapter.

The default location style clause is:

Studio Ghibli-inspired hand-painted background art, rich warm watercolor-like shading,
gentle luminous colors, atmospheric depth, delicate expressive architectural detail,
a welcoming lived-in storybook atmosphere.

Both infer.py and the optimized helper use scripts/vivarium_prompt_templates.py automatically if no full --prompt is supplied. --description changes the person or location and keeps the recommended style, framing and green-background/no-people instructions. The defaults are 50 steps, CFG4 and LoRA strength1.0. A full --prompt overrides the template. Neither path adds a negative prompt by default.

from infer import create_pipeline, generate
pipe = create_pipeline("characters")
# No prompt argument: the tested warm-painted teenager template is used.
generate(pipe).save("default-teenager.png")
generate(pipe, description="a curious 11-year-old girl with a black ponytail, coral hoodie and blue jeans, natural school-age anatomy and a proportionate head").save("schoolchild.png")

Complete tested prompts for different people and locations are included in prompt-examples.json; the templates and defaults are also exported as prompt-templates.json.

Warm painted and soft thin-line variants both retained this direction. CFG3 was softer but sometimes changed clothing details, so CFG4 remains the default. School-age children still have stylized eyes: the prompts reduced exaggeration only partly. Strong eye-size reduction through reference editing sometimes produced dot-like or closed eyes, so it is not recommended.

Complete warm-painted prompt examples

teen-boy

vvar_char, one fictional character, a mature-looking 17-year-old young Indian man, a late adolescent high-school senior with short dark hair, wearing a red hoodie and loose navy jeans, fully clothed and age-appropriate. The face reads as nearly adult: an elongated oval face, modest almond-shaped eyes, a subtly defined jaw, a proportionate head and long adolescent limbs. Calm friendly expression, facing the viewer. Standing three-quarter portrait, head to thighs, hands and shoulders inside the frame, margin around the silhouette. Studio Ghibli-inspired grounded coming-of-age animation, delicate expressive hand-drawn lines, rich warm watercolor-like painted shading, gentle luminous colors, restrained almond-shaped eyes and proportionate faces. Perfectly uniform solid bright chroma green background, no scenery, no props, no shadow or gradient. Clothing contains no green or teal. No text, no logo, no watermark.

Seed 28008;50 steps, CFG4, LoRA1.0; 832×1248.

school-age-girl

vvar_char, one fictional character, a curious 11-year-old Japanese girl with straight black hair in a ponytail, wearing a coral hoodie and blue jeans, fully clothed and age-appropriate. Clearly a school-age preadolescent, with a slender proportionate head, modest almond-shaped eyes, a more elongated face and natural child anatomy. Calm friendly expression, facing the viewer. Standing three-quarter portrait, head to thighs, hands and shoulders inside the frame, margin around the silhouette. Studio Ghibli-inspired grounded coming-of-age animation, delicate expressive hand-drawn lines, rich warm watercolor-like painted shading, gentle luminous colors, restrained almond-shaped eyes and proportionate faces. Perfectly uniform solid bright chroma green background, no scenery, no props, no shadow or gradient. Clothing contains no green or teal. No text, no logo, no watermark.

Seed 28011;50 steps, CFG4, LoRA1.0; 832×1248.

pottery-workshop

vvar_bg, A sunlit pottery workshop, clay bowls drying on wooden shelves, a potter's wheel, workbenches and a large window overlooking a garden. Studio Ghibli-inspired hand-painted background art, rich warm watercolor-like shading, gentle luminous colors, atmospheric depth, delicate expressive architectural detail, a welcoming lived-in storybook atmosphere. Wide establishing view of an empty location. No people, no humans, no characters, no text, no watermark.

Seed 18004;50 steps, CFG4, LoRA1.0; 1376×768.

For a new person, change the age, hair, skin tone and outfit description while retaining the style and green-background clause. For a new location, change the place description while retaining the warm background style and empty-location instruction. The station gallery uses an earlier Ghibli prompt because the later warm-painted station produced unwanted sign lettering despite the no-text clause.

Teenagers and less exaggerated eyes: negative CFG option

Simply adding "almost an adult" has a limited effect in our tests. More explicit face descriptions plus a negative CFG branch produced a clearer change for teenagers:

a mature-looking 17-year-old young Indian man, a late adolescent high-school senior.
The face reads as nearly adult: an elongated oval face, modest almond-shaped eyes,
a subtly defined jaw, a proportionate head and long adolescent limbs.

Suggested teenager negative prompt:

childish face, chibi, super deformed, oversized anime eyes, oversized head,
doll-like face, toddler proportions, baby face, child, children,
little kid, preteen, preschool child

The installed undistilled pipeline takes negative_prompt_embeds, rather than a raw negative_prompt string. Encode the negative text and pass its tensor. This uses real CFG, so guidance must be greater than 1:

negative = "childish face, chibi, oversized anime eyes, oversized head, baby face, preteen"
negative_embeds, _ = pipe.encode_prompt(
    negative, device=torch.device("cuda"), max_sequence_length=256
)
image = pipe(
    prompt="vvar_char, a mature-looking 17-year-old young man, near-adult facial proportions, modest almond-shaped eyes, red hoodie and navy jeans, warm painted animation, solid bright green background",
    negative_prompt_embeds=negative_embeds,
    width=832, height=1248, num_inference_steps=50,
    guidance_scale=4.0, max_sequence_length=256,
    generator=torch.Generator("cuda").manual_seed(28008),
).images[0]

For actual school-age children, keep the age unchanged and describe natural child proportions. Use negatives such as chibi, oversized eyes, oversized head, toddler proportions; excluding "child" would oppose the intended age. The effect on school-age faces can be smaller than on teenagers.

Consistency with a reference image

The base model supports image conditioning. First generate and select a canonical character, then use that image for individual emotion or outfit changes:

from PIL import Image
reference = Image.open("character.png").convert("RGB")
image = pipe(
    image=reference,
    prompt="vvar_char, use the same person as the reference. Preserve facial identity, age, hairstyle, skin tone, body proportions, illustration style and framing. Change only the facial expression to a delighted smile. Keep the original outfit and uniform green background.",
    width=832, height=1248, num_inference_steps=50,
    guidance_scale=4.0, max_sequence_length=256,
    generator=torch.Generator("cuda").manual_seed(29006),
).images[0]

Our engineer examples show the same source character with a smile and a blue jacket. Reference conditioning preserved identity and framing much better than text alone in the tested cases, but identity is not guaranteed perfectly. These adapters learn a visual style; they are not individual-person identity LoRAs. Reference images increase generation time because the transformer processes their image tokens.

python infer.py characters --reference character.png --prompt "vvar_char, keep the same person and face; change only the outfit to a blue jacket over an ivory shirt; preserve the green background" --output new-outfit.png

40 or 50 steps?

Four paired ghibli-warm-painted subjects (two characters,two locations) were generated with identical prompts,seeds,dimensions,CFG4 and LoRA1.0. In our visual review 40 and50 steps looked very close, with no clear overall quality loss at 40. Fine detail, lighting and some background object shapes differed. This small sample does not establish equal quality for every prompt. Keep 50 as the default for chosen final images; use 40 for faster iteration. The warm-style pairs averaged 25.64s at 40 and 32.00s at 50 on one A100; exact timings and prompts are in ghibli-step-comparison.json.

Higher native throughput

The optional scripts/native_flux_inference.py fuses the LoRA and QKV projections, compiles repeated transformer blocks and batches the two real CFG branches. It retains the undistilled model, BF16, CFG4 and all40/50 denoising steps. It also supports negative text and reference images. These changes can cause floating-point differences; they do not guarantee bit-identical pixels.

import sys
sys.path.insert(0, "scripts")
from native_flux_inference import create_native_pipeline, generate_native_image

pipe = create_native_pipeline("characters")
prompt = "vvar_char, a mature-looking 17-year-old young man, proportionate face and restrained almond-shaped eyes, red hoodie and navy jeans, Studio Ghibli-inspired warm painted animation, uniform bright green background"
# Keep this pipeline loaded. The first call includes compilation and kernel warmup.
generate_native_image(pipe, prompt, steps=50)[0].save("warmup.png")
generate_native_image(pipe, prompt, seed=28008, steps=50)[0].save("character.png")

Four-case warm measurements on one A100-SXM4-80GB averaged 25.62s/image at 40 steps and 31.88s/image at 50 steps, including positive text encoding and VAE decoding, with the empty negative embedding cached. Loading, compilation, warmup and PNG saving are excluded. Cases included two832×1248 characters and two1376×768 locations. A two-A100 CFG-parallel configuration averaged16.43s/image at 40 steps; it uses two GPUs per image and has lower throughput per GPU than independent single-GPU workers.

Reference inputs add image tokens. In two tested cases at 40 steps, reducing only the reference from832×1248 to416×624 reduced average generation time from55.42s to31.99s while output stayed832×1248. Faces remained recognizable with small detail changes. Measure your own prompts and inspect identity before choosing reduced references. The helper's CLI accepts --reference, --negative-prompt, --steps40/50, and --cfg-modeparallel when two visible GPUs are available; write flags with spaces, e.g. --steps 40 --cfg-mode parallel.

The direct public-adapter loading path was verified by reproducing the original50-step gardener example pixel for pixel. The optimized helper has also been exercised on actual40/50-step generations. Full benchmark settings and pinned software revisions accompany the release in native-benchmarks.json.

Training and provenance

Trained on TTS-AGI/vivarium-living-world-assets, with separate character/background caption sets. Character split:81 train and9 held-out; background split:59 train and8 held-out. Rank64,alpha64; adapters were selected after evaluating intermediate checkpoints. Full category training settings and selection reports are included alongside each adapter.

See provenance.json and adapter-manifest.json for pinned source revisions,selected training steps and SHA256 checksums.

The base model is distributed under the FLUX Non-Commercial License. This adapter release retains that licensing designation; consult the base-model license for the applicable terms. The base checkpoint itself is not included in this repository.

Child and teen example recheck

All six child/teen example PNGs were checked with both official LAION classifiers on original and foreground views. Two examples were regenerated in the same warm-painted direction and checked again; their exact prompts, seeds and parameters are in the updated example manifest. See scores, thresholds and replacement details. Adapter weights remain unchanged.