--- license: apache-2.0 library_name: pytorch pipeline_tag: text-to-image language: - en datasets: - pixparse/cc12m-wds tags: - text-to-image - flow-matching - rectified-flow - latent-diffusion - tiny-model - from-scratch - muon model-index: - name: AuroraIMG-6M results: - task: type: text-to-image name: Text-to-image generation dataset: name: GenEval type: geneval metrics: - type: geneval_overall value: 0.1387 name: GenEval overall (avg. over tasks) - task: type: text-to-image name: Text-to-image generation dataset: name: GenEval 2 type: geneval2 metrics: - type: soft_tifa_gm value: 2.21 name: Soft-TIFA GM (prompt-level, official) - type: soft_tifa_am value: 21.84 name: Soft-TIFA AM (atom-level) --- # AuroraIMG-6M AuroraIMG-6M is a text-to-image model that's just **6.82M parameters**, and that number includes everything: the text encoder, the denoiser and the VAE. It generates 64×64 images from short English captions, and every part of it was trained from scratch on CC12M. No pretrained text encoder or VAE was used. It reaches an **FID of 33** and a **CLIP of 24.8** on a 10k-image CC12M holdout set. ## At a glance | | | |---|---| | **Resolution** | 64×64 RGB | | **Total parameters** | 6.82M (Text encoder 1.32M / Denoiser 4.42M / VAE 1.08M) | | **Method** | Latent flow matching (rectified flow) with classifier-free guidance | | **Latent** | 8 × 16 × 16 | | **Text encoder** | 4-layer transformer, 4096-token BPE, up to 64 tokens | | **Sampler** | Euler, 50 steps, CFG 6.0 (best in the CFG sweep) | | **Data** | CC12M, about 9.88M training pairs | ## Quick start ```python from diffusers import DiffusionPipeline import torch device = "cuda" if torch.cuda.is_available() else "cpu" dtype = torch.bfloat16 if device == "cuda" else torch.float32 pipe = DiffusionPipeline.from_pretrained( "AuroraAI-Research/AuroraIMG-6M", custom_pipeline="AuroraAI-Research/AuroraIMG-6M", trust_remote_code=True, torch_dtype=dtype ).to(device) prompt = "a sunset over the ocean" print(f"Generating: '{prompt}'...") generator = torch.Generator(device=device).manual_seed(42) image = pipe( prompt=prompt, num_inference_steps=50, guidance_scale=6.0, generator=generator ).images[0] filename = "Aurora-Image.png" image.save(filename) print(f"\n Saved to {filename}!") ``` The model is small enough to run on CPU. ## Samples Same eight prompts, same seed, increasing guidance (CFG 1, 2, 3 and 5, top to bottom): ![CFG sweep](guidance_sweep.png) Final checkpoint at CFG 3 for the fixed prompt set ("a red car on a street", "a dog running on green grass", "a sunset over the ocean", "a snowy mountain with blue sky", "a plate of food on a table", "a man standing in front of a building", "a small boat on a calm lake", "a vase of flowers on a wooden table"): ![Final samples](samples_0300000.png) Real COCO val2014 images (top row) next to generated images from their captions (bottom row): ![COCO real vs generated](coco_real_vs_gen.png) And the VAE on its own: originals on top, reconstructions below. ![VAE reconstructions](vae_recon.png) ## How it works **VAE (1.08M).** A small convolutional VAE with residual blocks and GroupNorm that squeezes a 64×64×3 image into an 8-channel 16×16 latent. It was trained with an L1 loss, a gradient-based sharpness loss, a Gram-matrix loss and a tiny KL term (5e-5). There's no adversarial or LPIPS loss. On holdout images it reconstructs at 31.9 dB PSNR. **Text encoder (1.32M).** A 4-layer pre-LN transformer (width 128, 4 heads) with rotary position embeddings, sitting on a byte-level BPE tokenizer trained on the CC12M captions. It's trained jointly with the denoiser, so it learns whatever representation the image model finds most useful. **Denoiser (4.42M).** A compact U-Net that works on the latent at 16×16, 8×8 and 4×4. Residual blocks use AdaLN-Zero conditioning (scale, shift and gate from the time embedding, zero-initialised so each block starts as identity). The time embedding uses random Fourier features, with a mean-pooled text embedding added on top. Each level has an attention block with self-attention, cross-attention to the text tokens and a feed-forward layer. ## Training **Data:** The whole CC12M set (`pixparse/cc12m-wds`) was streamed and filtered. Captions were lowercased and cleaned of HTML and URLs, and generic ones ("image", "stock photo", …) or anything under 3 words were dropped. Images under 128px on the short side, or not RGB, were dropped too. The rest were resized to 64px on the short side (bicubic), center-cropped, and de-duplicated by pixel hash. That left 9,882,283 training pairs plus a 50k holdout. Random horizontal flips are the only augmentation. **Stage 1: VAE.** 25k steps at batch size 2048 with AdamW (lr 5e-4, cosine decay). Final loss 0.0880 (L1 0.0343, KL 5.91). **Stage 2: flow matching.** 300k steps at batch size 2048, which is roughly 614M samples or ~62 epochs. The model predicts velocity (`ε − z`) with an MSE loss, and timesteps come from a logit-normal distribution mixed with 15% uniform. Cached latents are re-noised with the VAE's own posterior variance during training. Weight matrices and conv kernels are optimised with Muon (lr 0.02), and everything else (embeddings, input/output layers, norms, biases, time and pool projections) with AdamW (lr 5e-4). On top of that: 10k warmup steps, cosine decay to 1%, gradient clipping at 1.0, EMA 0.9998, 10% caption dropout for CFG, bf16 autocast and `torch.compile`. Final training loss: 0.6208. All samples and metrics below use the EMA weights. ## Results Everything for AuroraIMG-6M is measured at the native 64×64 resolution. FID uses `torchmetrics` (2048-d Inception features) and CLIP score uses `openai/clip-vit-base-patch32`. Both upsample the images internally, so treat them as numbers for tracking this model rather than for lining up against 256px+ models. ### CC12M holdout (10k images) I swept CFG from 1.0 to 9.0, and 6.0 gave the best results, achieving nearly the maximum CLIP it can, without sacrificing FID, so that's the recommended setting. | | FID ↓ | CLIP score ↑ | |---|---|---| | **AuroraIMG-6M (CFG 6.0)** | **33.01** | **24.80** | | Real images | n/a | 28.25 | The generated images land at about 88% of the CLIP score of the real photos (24.80 vs 28.25) when compared against their captions. ### VAE only | FID ↓ | PSNR ↑ | CLIP score ↑ | |---|---|---| | 8.97 | 31.92 dB | 26.95 | ### GenEval Overall score (averaged over tasks): **0.1387**. 12.88% of images and 20.61% of prompts were fully correct (2212 images, 553 prompts). | Task | Accuracy | |---|---| | single_object | 50.00% (160 / 320) | | colors | 26.86% (101 / 376) | | two_object | 3.79% (15 / 396) | | counting | 1.56% (5 / 320) | | position | 0.75% (3 / 400) | | color_attr | 0.25% (1 / 400) | #### Comparison with Stable Diffusion 1.5 | Model | Params | Overall ↑ | Single object | Two object | Counting | Colors | Position | Color attr. | |---|---|---|---|---|---|---|---|---| | **AuroraIMG-6M** | **6.82M** (total) | **0.14** | 0.50 | 0.04 | 0.02 | 0.27 | 0.01 | 0.00 | | Stable Diffusion v1.5 | 0.86B (U-Net only) | 0.43 | 0.97 | 0.38 | 0.35 | 0.76 | 0.04 | 0.06 | ### GenEval 2 AuroraIMG-6M was run at CFG 6.0, 50 Euler steps, 4 samples per prompt. | Model | Params | Soft-TIFA GM ↑ | Soft-TIFA AM ↑ | |---|---|---|---| | **AuroraIMG-6M** | **6.82M** | **2.21** | **21.84** | | Stable Diffusion 2.1 | 0.87B (U-Net only) | 5.2 | 40.8 | | Supra2-IMG | 104.1M (denoiser only) | 6.80 | 36.38 | | Surjo-Image-Preview | 108.3M (denoiser only) | 8.07 | 42.30 | Per-skill breakdown (Soft-TIFA AM, atom level): | Skill | AuroraIMG-6M | Supra2-IMG | Surjo-Image-Preview | |---|---|---|---| | object | 29.64 | 46.77 | 61.03 | | attribute | 26.01 | 43.56 | 45.80 | | position | 15.33 | 31.07 | 28.91 | | count | 11.90 | 22.14 | 22.30 | | verb | 0.29 | 1.51 | 2.03 | **Prompting tips.** It was trained on CC12M alt-text, so short, lowercase, caption-style prompts work best ("a dog running on green grass", "a snowy mountain with blue sky"). ## Good for - Experimenting with tiny generative models: Muon, flow matching, AdaLN-Zero, small VAEs, joint text-encoder training. - Running text-to-image on very limited hardware. ## Things to keep in mind - **Resolution.** Output is 64×64, so small details like faces, hands and text aren't going to be visible. - **Prompt complexity.** It does best with a single main subject and a simple scene. Counting, positions and multi-attribute prompts (see GenEval and GenEval 2) are where it falls short. - **Language.** English only, lowercased, up to 62 caption tokens. - **Data bias.** CC12M is web-scraped alt-text, so the model picks up its biases and its stock-photo look. Only the quality filters described above were applied, with no extra safety filtering of data or outputs. - **Not for** generating realistic images of real people or events, or anywhere accuracy and fairness matter. ## License The model weights are released under the Apache 2.0 license. The model was trained on [CC12M](https://huggingface.co/datasets/pixparse/cc12m-wds), which is distributed as image URLs and captions; the underlying images carry their own licenses, so check the dataset terms for your use case. ## Citation ```bibtex @misc{auroraimg6m, title = {AuroraIMG-6M: a 6.8M-parameter text-to-image model}, author = {AuroraAI-Research}, year = {2026}, howpublished = {\url{https://huggingface.co/AuroraAI-Research/AuroraIMG-6M}} } ```