drama-3-TTS-GGUF

Fish Audio Drama 3 is Fish Audio's new-generation text-to-speech model, built for multi-character scenes and long-form content. It supports multi-speaker generation (up to 16 characters via <|speaker:0|> … <|speaker:15|> tags), voice cloning from a short reference clip, and natural-language control, including inline point tags such as [sighs] and [laughs]. It also supports editing by continuation: you pass earlier takes as context and change only a few words. The model has 35B parameters with about 3B active per token. It uses a mixture-of-experts text backbone with hybrid full and linear attention, FP8 experts, and a codebook-MTP audio decoder over 10 codebooks. It covers roughly 80 languages, and long text is automatically chunked and can be streamed. It runs on SGLang-Omni and vLLM-Omni. On a single H200, a single stream gets first audio in under 100 ms with a real-time factor of about 0.13, and throughput reaches about 1,687 frames/s at 64 concurrent streams. Placing the vocoder on a second GPU raises throughput by about 72% and lowers median time to first audio by about 37% at 256 streams. The model is released under the Fish Audio research/community license.

Drama-3 TTS (Fish Audio) is now supported in the mainline llama.cpp release, v1.0.0.

Model Files

File Name Quant Type File Size File Link Description
drama-3.MXFP4_MOE.gguf MXFP4_MOE 19.8 GB Link MXFP4_MOE: Highly optimized for NVIDIA Blackwell hardware accelerators (e.g., RTX Pro 6000, RTX 50-series, B200, B300), offering native hardware-accelerated FP4 Tensor Core speeds and a strong balance of size and performance.
drama-3.Q3_K_L.gguf Q3_K_L 18.1 GB Link Lower quality but usable, good for low RAM availability.
drama-3.Q3_K_M.gguf Q3_K_M 16.8 GB Link Low quality, smallest file size.
drama-3.Q4_K_M.gguf Q4_K_M 21.2 GB Link Good quality, default size for most use cases, recommended.
drama-3.Q5_K_M.gguf Q5_K_M 24.7 GB Link High quality, recommended.
drama-3.Q5_K_S.gguf Q5_K_S 24 GB Link High quality with slight space savings, recommended.
drama-3.Q6_K.gguf Q6_K 28.5 GB Link Very high quality, near perfect, recommended.
drama-3.mmproj-bf16.gguf mmproj-bf16 2.94 GB Link Multimodal projection file in BF16 format. Used for audio/TTS capabilities.

Run with llama.cpp (MXFP4 MoE)

Install dependencies and clone llama.cpp

!pip install huggingface-hub

!git clone https://github.com/ggml-org/llama.cpp.git
%cd /data/llama.cpp

Download GGUF model files

Download the MXFP4 MoE model and BF16 multimodal projector:

from huggingface_hub import snapshot_download

# Target local directory
local_dir = "./model/drama-3-TTS-GGUF"

# Download only the specified files
snapshot_download(
    repo_id="prithivMLmods/drama-3-TTS-GGUF",
    local_dir=local_dir,
    allow_patterns=[
        "drama-3.MXFP4_MOE.gguf",
        "drama-3.mmproj-bf16.gguf",
    ],
    resume_download=True,
)

Build llama-tts

Build the TTS CLI with the CUDA backend enabled:

!cmake --build . --config Release -j$(nproc) --target llama-tts

Text-to-speech generation (GPU offloading)

Run inference using llama-tts with GPU offloading enabled:

!/data/llama.cpp/build/bin/llama-tts \
    -m /data/llama.cpp/model/drama-3-TTS-GGUF/drama-3.MXFP4_MOE.gguf \
    -mm /data/llama.cpp/model/drama-3-TTS-GGUF/drama-3.mmproj-bf16.gguf \
    -p "<|speaker:0|> Did you remember to buy the bread? <|speaker:1|> Of course I did, it is on the kitchen table." \
    -ngl 99 \
    -n 1000 \
    --seed 42 \
    --output /data/llama.cpp/model/drama-3-TTS-GGUF/out.wav

Output

The generated audio file will be saved at:

/data/llama.cpp/model/drama-3-TTS-GGUF/out.wav

You can play the generated WAV file using an audio player or notebook audio widget.


llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Quantized using llama.cpp(v1.0.0) by prithivMLmods.

Downloads last month
874
GGUF
Model size
35B params
Architecture
qwen35moe
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

5-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for prithivMLmods/drama-3-TTS-GGUF

Quantized
(59)
this model

Collections including prithivMLmods/drama-3-TTS-GGUF