kasparovabi/drama-3-tr-speaker

Drama 3 Turkish speaker (32 experts per layer)

Built with Fish Audio. This is a Derivative Work of Fish Audio Drama 3. It is not an official Fish Audio release, and Fish Audio does not sponsor or endorse it.

A small Turkish "speaker" for Drama 3. The original 35B MoE reads the text prompt once. A distilled speaker that keeps 32 of the 256 routed experts in every layer (12.5%) then generates every audio frame. On one RTX 5070 Ti (16 GB) the pair runs about 2.7 times faster than real time on short sentences and about 3.3 times on paragraphs, and keeps Drama 3's inline directions and emotion. Experts are streamed straight from the NVMe drive, with the 6,000 most used ones kept in pinned RAM.

Try it in your browser in the Hugging Face Space. It runs on ZeroGPU, so the first request may wait a little for a GPU.

Update, 22 October 2026. The engine got 3 times faster on short sentences (0.86x to 2.7x real time in disk mode) with the same weights, and Freya-TR-Eval WER went from 2.78% to 2.27% in the same configuration. What changed is under Speed and memory. There is also a new experiment, emotion intensity by activation steering.

Türkçe özet aşağıda. How it was built, including a bug I found after release, is written up in blog.md, also on dev.to and in Turkish on Medium.

Results

All numbers come from one machine (RTX 5070 Ti 16 GB, Samsung 990 EVO Plus NVMe, Windows 11). Sampling uses temperature 1.0 and top-p 0.9 as recommended by the Drama 3 model card.

Freya-TR-Eval. The public Turkish benchmark by Freya (dataset, paper), 495 everyday sentences, scored with Freya's own eval/benchmark.py protocol (8 kHz band matching, faster-whisper large-v3, beam 5, Turkish normalization, corpus WER and CER with jiwer).

System	WER	CER
Drama 3 + this speaker, no guidance, experts in RAM	2.68%	0.70%
Drama 3 + this speaker, guidance strength 4, experts read from disk	2.78%	0.67%
Same, with the October 22 engine (compiled, hot expert tier)	2.27%	0.62%
Same, with the dense layers also in block FP8 (TTS_FP8=1)	3.29%	0.99%

In the first run one sentence of 495 ran on for 13 s and was transcribed as garbage, in the second and third runs none did, and the FP8 run had one sentence run on to 25 s. Each row is a single run at temperature 1.0, so differences of a few tenths of a point are within sampling noise. The FP8 gap of one point is larger than that, which is why FP8 stays off by default. The other errors are mostly proper nouns and suffix spacing. For context, Freya reports 8.0% WER for FreyaTTS-small (183M) and 11.1% for XTTS-v2 on the same set. Those are much smaller and much faster models built for a different job, so the numbers are not a like-for-like comparison.

Closeness to the teacher. Teacher-forced on 200 held-out ISSAI TSC utterances, the speaker's main-code distribution is close to unmodified Drama 3.

	KL to teacher	NLL
Unmodified Drama 3	0	4.973
This speaker	0.066	4.984

On our own 16-sentence set (10 neutral, 6 with directions) the Whisper WER is 0.6% for both the teacher and the speaker.

Speed and memory

Guidance strength 4, experts read from disk, October 22 engine unless noted.

	Value
Short sentences (495 Freya-TR-Eval items, median 3.5 s), hot expert tier, compiled	2.7x real time (median)
Paragraphs of 16 to 20 s, same setup	3.2 to 3.4x real time in five of six runs, 1.9x in one
Short sentences without the hot tier, compiled	1.2x real time (median)
Short sentences, first release, disk mode	0.86x real time
Peak VRAM	14.0 GiB on short sentences, 15.0 GiB on paragraphs
System RAM	25 GB with the hot tier, 4.9 GB without it
Windows commit charge	about 20 GB on top of that, most of it mirrors the GPU allocations, so keep the page file enabled

What made it faster, in order of payoff.

1. torch.compile on the per-frame step, then CUDA graph capture of the compiled code. Profiling showed more than 2,000 small elementwise and dtype cast kernels per frame around the matrix multiplications. A frame went from 22.7 ms to 13.6 ms.
2. Logits only for the 4,097 tokens the audio head can sample (end of speech and the semantic codes) instead of the full 252k vocabulary. About 1 ms per frame, same output.
3. A pinned RAM tier for the 6,000 experts that 300 traced Turkish sentences used most. Before, reading one short sentence pulled about 3,100 experts (9 GB) from the drive. Now about 40 still come from the drive.
4. Prompt reading turned out to be bound by Python rather than PCIe, since every expert load issued five separate copies. An expert's gate, up and down weights now share one contiguous GPU slot and the scales move with one gather per layer, so reading takes about 0.42 s instead of 0.65 s.

Prefetching the next layer's experts with its router applied to the current hidden state found 80 to 90% of them, but the wrong guesses evicted experts that were still needed and reading got slower. PyTorch's pin_memory=True rounds allocations up to a power of two, so the 18.6 GB hot tier took 32 GB of RAM until it was registered with cudaHostRegister instead. Block FP8 for the dense layers needed its own Triton kernel, because bf16 already reaches 85 to 90% of memory bandwidth on these batch-2 products and row-wise scaled FP8 matmul is not available on sm120 in torch 2.14. It saves 2 ms per frame and is optional because of the WER result above.

Samples

Generated with the weights in this repository, guidance strength 4, seed 0 (--seed 0), experts read from disk. Sampling runs at temperature 1.0, so other seeds give other takes, and some takes drift further from the reference voice than these. The voice is a synthetic robot voice created by the author.

File	Text
samples/00.mp3	Merhaba, ben Tıkır. Bugün sana küçük bir şey göstermek istiyorum.
samples/01.mp3	[heyecanla] İnanamıyorum, sonunda başardık!
samples/02.mp3	[fısıldayarak] Sessiz ol, herkes uyuyor.
samples/03.mp3	[sinirli] Bunu kaç kere söyleyeceğim, kapıyı kapatır mısın?
samples/04.mp3	[sıcak ve gizemli bir masalcı gibi] Bir varmış bir yokmuş, uzak bir dağın eteğinde küçük bir robot yaşarmış...

How it works

Drama 3 is a 40-layer hybrid model (30 Gated DeltaNet layers, 10 full-attention layers) with 256 routed FP8 experts per layer, 8 active per token, plus a shared expert. Generation has two phases. In prefill the model reads the system prompt, the reference audio and the text. In decoding it produces one audio frame (about 21.5 per second) at a time.

1. Reader and speaker. The unmodified model handles prefill. The speaker, with 32 experts per layer, handles every audio frame. The two meet through the KV cache and the DeltaNet states. Separating prefill from decode experts was published for language models by SlimWise (arXiv 2609.34117). We arrived at the same split independently for TTS and push the decode side much further (12.5% of experts instead of 50%).
2. Trajectory anchoring. Each layer of the speaker is trained on its own drifted inputs, that is, on the hidden states produced by the already distilled lower layers, while the target is the teacher's next hidden state on the clean trajectory. This is the asymmetric calibration objective of GPTAQ (ICML 2025), used here to fine-tune a pruned expert set rather than to quantize. In our ablation it beats plain function matching at every scale.
3. Streaming from disk. The reader needs many different experts per sentence. Instead of keeping the 41.5 GB checkpoint in RAM, each 3 MB expert is read with unbuffered I/O directly from the NVMe drive into pinned memory. This is a known offloading technique. On this machine it is as fast as reading from the page cache.
4. Direction guidance. The prompt is also run with its bracketed directions blanked out, and the main-code logits are pushed away from that unguided run (classifier-free guidance, strength 4). This made directions such as whispering noticeably more reliable. Guidance for emotion in autoregressive TTS was studied before, for example SMG-CFG on CosyVoice2.
5. Streaming codec decode. The codec is causal. Long-range context lives only in the quantizer's windowed attention (8 layers with a 128-frame window), so the quantizer runs on 1024 frames of history and the convolutional decoder only on new frames plus 8 frames of overlap. The streamed audio matches a full decode within bf16 noise.

Findings we think are worth sharing

1. The speaker can be very small. With the reader intact, 32 experts per layer are enough for the audio phase. Uniform pruning of both phases at the same budget is 3 to 7 times worse in KL (at 128 experts, 0.361 against 0.049).
2. Emotion is formed while reading. A version whose reader was also distilled (16 experts per layer on text positions) kept intelligibility but sounded flat and lost the directions in listening tests, while shrinking only the speaker did not. Style and emotion appear to be written into the cache during prefill.
3. Text routing is local. On Turkish text positions the expert choice is largely fixed by the current and previous token (5 to 7 of 8 experts shared across different sentences, in all 40 layers). This matches the context-independent specialization reported by OpenMoE (ICML 2024).

Emotion intensity by activation steering

Guidance already runs the prompt twice, once with its directions and once with them blanked out. The mean difference between the two runs' hidden states on identical text rows gives one vector per direction and layer (30 sentences each). Adding a multiple of it to one layer's output while the prompt is read, with no direction in the sentence, moves the voice toward that direction. Layers 8 and 16 had no effect. At layers 24 to 28 with strength 2 to 3 it shifts pitch and energy about as far as the real direction for whispering (pitch down 47.6 Hz against 41.6 for the direction) and excitement (up 14 Hz against 10.6), with Whisper WER at 0 to 2% and speaker similarity equal to or above the real direction. For sadness it moves less than the direction (pitch down 21 Hz against 29) but keeps the voice closer to the reference (0.80 against 0.68). Activation steering for emotion in TTS has been published before (EmoSteer-TTS, SteerSpeech), so the contribution here is only that it works on this MoE through the guidance pair and gives a continuous intensity knob. Biasing the 256-way router toward the experts a direction prefers, in the spirit of SteerMoE, did not move prosody. Code is steer_act.py on GitHub.

Ablation (teacher-forced, 300 training and 100 test utterances)

Speaker training	KL at 32 experts	KL at 64 experts
Expert selection only	0.747	0.266
Function matching	0.153	0.048
Trajectory anchoring	0.109	0.035

Usage

The inference engine and training code are on GitHub, at kasparovabi/drama-3-tr. You need the original fishaudio/drama-3 checkpoint and its codec, a CUDA GPU with 16 GB, and an NVMe drive.

hf download kasparovabi/drama-3-tr-speaker speaker.safetensors config.json --local-dir .
hf download kasparovabi/drama-3-tr-speaker trace_experts.pt --local-dir results
TTS_COMPILE=max-autotune-no-cudagraphs python engine.py --speaker speaker.safetensors --graph --slots 300 --disk --hot 6000 \
  --ref my_voice.wav --ref_text my_voice.txt --sentences sentences.json --out out

trace_experts.pt holds how often each expert was used while reading 300 Turkish sentences with directions, and --hot 6000 pins the 6,000 most used ones (18.6 GB of RAM). Leave --hot out on a machine with less than 32 GB of RAM. TTS_COMPILE adds a few minutes to start-up.

Directions go in square brackets, in Turkish, and Turkish directions worked better than English ones in our listening tests.

[heyecanla] İnanamıyorum, sonunda başardık!
[fısıldayarak] Sessiz ol, herkes uyuyor.
[sıcak ve gizemli bir masalcı gibi] Bugün sana küçük bir hikâye anlatacağım.

Limitations

1. Tested on a single machine and a single reference voice in listening tests. The speaker was trained on many ISSAI TSC speakers, but other voices were not listened to systematically.
2. Turkish only. The speaker was distilled on Turkish speech, other languages were not evaluated.
3. Like the original model, generation sometimes runs on (long laughter, silence after a direction). We saw it once in 990 generated Freya-TR-Eval sentences.
4. Guidance acts on the main code only. Whispering was realized in about half of the attempts at strength 3 to 4.
5. Start-up takes a few minutes with TTS_COMPILE, and the first request after start-up takes about 30 s more because the reference prompt is read once and cached. Switching to another reference voice costs that read again.
6. On Windows the driver backs GPU memory with commit charge, so the process commits about 20 GB more than its working set. A machine with 16 GB of RAM works only without --hot and with the page file enabled.
7. engine.py does not cap the length of a take. server.py does, from the length of the text, and generates another take when one runs on.

License and use restrictions

The weights are a Derivative Work of Fish Audio Drama 3 and are distributed under the Fish Audio Community License v1.0. See NOTICE for what was changed. Free for research and non-commercial use. Commercial use is allowed only below the revenue threshold in the license, and hosted TTS services need a separate license from Fish Audio. You may not use this model to imitate an identifiable person's voice without consent, to impersonate or deceive, or to remove watermarks. The full terms in the license apply to anyone who receives these weights.

The sample voice is a synthetic robot voice created by the author.

Türkçe özet

Bu depo Fish Audio Drama 3 için Türkçe bir "konuşmacı" içeriyor. Orijinal 35B model metni bir kez okuyor, ses karelerini ise her katmanda 256 uzmandan yalnız 32'sini kullanan damıtılmış konuşmacı üretiyor. Tek bir RTX 5070 Ti (16 GB) üzerinde kısa cümlede gerçek zamanın yaklaşık 2,7 katı, paragrafta yaklaşık 3,3 katı hızla çalışıyor, köşeli parantezli Türkçe yönergeleri ve duyguyu koruyor. Uzmanlar NVMe diskten okunuyor, en sık kullanılan 6.000 uzman sabitlenmiş RAM'de tutuluyor (18,6 GB). Bu katman kapatılınca üretim 4,9 GB RAM ile yetiniyor ama kısa cümlede hız 1,2 kata iniyor.

22 Ekim 2026 güncellemesi. Aynı ağırlıklarla motor kısa cümlede üç kat hızlandı. Kare adımı torch.compile ile derlenip CUDA grafiğine alındı, çıktı katmanı yalnız ses tokenleriyle hesaplanıyor, uzman kopyaları uzman başına tek komuta indi. Yönerge vektörünü bir katmana ekleyerek duygu şiddetini sürekli bir düğmeyle ayarlama deneyi de yukarıda anlatılıyor.

Freya-TR-Eval'de (495 cümle, Freya'nın kendi ölçüm protokolü) yeni motorla WER yüzde 2,27, CER yüzde 0,62. Okuma ve konuşma evresini ayırma fikri dil modelleri için SlimWise tarafından, eğitim hedefi ise GPTAQ tarafından daha önce yayımlandı. Bu çalışmanın katkısı bunları bir TTS modelinde Türkçe için uygulamak ve iki bulgu. Konuşmacı uzmanların yüzde 12,5'iyle yetinebiliyor. Duygu ise okuma sırasında oluşuyor, okuyucuyu küçültünce ses düzleşiyor.

Nasıl yapıldığını Medium'daki Türkçe yazıda anlattım, İngilizcesi blog.md dosyasında. Lisans Fish Audio Community License v1.0. Kimsenin sesini izinsiz taklit etmek için kullanılamaz.

Citation

@misc{kazankaya2026drama3tr,
  title  = {Drama 3 Turkish speaker: reading with the full model, speaking with 32 experts},
  author = {Ahmet Kazankaya},
  year   = {2026},
  url    = {https://huggingface.co/kasparovabi/drama-3-tr-speaker}
}