parakeet-ctc-0.6b-sapc1

nvidia/parakeet-ctc-0.6b fine-tuned on the Speech Accessibility Project corpus 1 (SAPC1) train split for recognition of dysarthric and otherwise disordered speech.

On the SAPC1 dev split it roughly halves the error rate of the generic model:

	WER	CER
nvidia/parakeet-ctc-0.6b	20.08%	12.62%
dys-asr/parakeet-ctc-0.6b-sapc1	10.32%	6.09%

Output text convention

This model writes numbers as words and emits upper-case, unpunctuated text.

audio:  "lower the temperature three degrees"
output: LOWER THE TEMPERATURE THREE DEGREES     # not "... 3 DEGREES"

The tokeniser has 1,025 tokens and no digit characters, so numerals cannot be produced. Training transcripts are therefore verbalised before tokenisation: "2" becomes TWO, "$45" becomes FORTY FIVE DOLLARS, "1949" becomes NINETEEN FORTY NINE, and dash-joined digit groups are read digit by digit (053-621 becomes OH FIVE THREE DASH SIX TWO ONE).

This matters for scoring. An earlier version of this model was trained on labels that kept numerals, which the tokeniser mapped to <unk>; it emitted <unk> wherever a number belonged and could never produce one. Fixing that was worth 3.29 WER points. If you score this model, apply the same verbalisation to your references, or numerals will dominate your error count.

Transcript markup

Transcripts are stripped of square-bracketed spans and #ts / #dis provenance tags before normalisation, because neither is spoken in the audio. In SAPC1 this is almost nothing -- 15 of 218,898 train records carry a bracket, and the dev references move by 53 words out of 337,893 -- so it does not meaningfully change this model. It is recorded because it matters enormously for the SAPC2 sibling, where brackets hold the canonical interview prompt on 12.4% of records; leaving them in trains the model to transcribe a question nobody asked. This model was retrained with the stripping in place so that both releases share one convention.

Training data

SAPC1 train split. Utterances outside 0.5-30 s are excluded, and 2 transcripts that normalise to empty are dropped.

	train	dev (evaluation)
speakers	580	83
utterances, total	218,900	31,114
utterances, used	212,195	30,139
audio used	408.1 h	58.3 h

Speakers by aetiology:

	train	dev
Parkinson's Disease	277	40
ALS	129	18
Cerebral Palsy	98	15
Down Syndrome	49	7
Stroke	27	3

No speaker appears in both splits (verified: zero speaker-id overlap). The corpus is heavily weighted toward Parkinson's Disease, at 48% of dev speakers, so an aggregate figure is substantially a statement about Parkinson's speech.

Training hyperparameters

Chosen by a 20-cell grid over learning rate and effective batch size, evaluated on the dev split. The grid spread 7.2 WER points, so these settings matter more than they might appear.

base model	nvidia/parakeet-ctc-0.6b
optimizer	AdamW, weight decay 0.01
learning rate	1e-4
schedule	tri-stage, 10% warmup, 40% hold
per-device batch	16 x 2 GPUs, SyncBatchNorm over all 32
gradient accumulation	1
effective batch	32
epochs	10 (66,320 optimizer updates, 6,632 per epoch)
precision	bf16 mixed
layerdrop	0.05
gradient clipping	1.0
feature encoder	trainable (not frozen)
gradient checkpointing	off
batch sampling	length-grouped
seed	42
hardware	2x NVIDIA A100-SXM4-80GB, ~4.9 h

Per-device batch size is not a free choice. Parakeet's Conformer normalises over the per-device batch in 24 BatchNorm1d layers, so it behaves as a hyperparameter in its own right: holding the learning rate and effective batch fixed and changing only per-device batch from 4 to 32 was worth about a WER point.

An earlier release of this model was trained on one GPU holding all 32 examples, on the reasoning that splitting across two would force per-device 16 and change those statistics. SyncBatchNorm removes the objection: it all-reduces the statistics across both processes, so 2 x 16 normalises over all 32 exactly as a single device would. The two-GPU run reported here scores 0.32 WER points better than the single-GPU one it replaces, which is within the range that seed variance alone could explain, so read it as no worse, not as an improvement from parallelism.

Dev WER was still falling on the final epoch (10.08% to 10.01% on the trainer's own metric), so this recipe has not saturated; more epochs would likely gain a further 0.2-0.3 points.

Usage

import torch
from transformers import AutoModelForCTC, AutoProcessor

model_id = "dys-asr/parakeet-ctc-0.6b-sapc1"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id).eval()

# audio: 16 kHz mono float32
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt")
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.batch_decode(logits.argmax(dim=-1)))

Greedy CTC decoding, no language model. Evaluation WER is identical at batch sizes 1 through 32, so padding does not affect results.

Limitations

• Numerals are written as words, not digits. See the convention above.
• No punctuation or casing. Output is upper-case and unpunctuated.
• Accuracy is very uneven across speakers. Over the 83 dev speakers the per-speaker WER runs from 0.0% to 69.0%, median 7.3%, with 3 speakers above 40% and 1 above 60%. The aggregate is not a useful prediction for an individual, and the spread correlates with severity.
• Mild speakers gain least. For speakers the nvidia/parakeet-ctc-0.6b already handles well, fine-tuning on disordered speech yields little, and on a numeral-free comparison across 49 held-out speakers it never hurt but sometimes barely helped.
• English only, 16 kHz mono.
• Single training seed. No variance estimate on the reported figures.