Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
Extracting clinical information from medical transcripts in low-resource languages remains a significant challenge in healthcare natural language processing (NLP). This study evaluates a two-step pipeline combining Aya-expanse-8B as a Persian-to-English translation model with five open-source small language models (SLMs) -- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-1.5B-Instruct, and Gemma-3-1B-it -- for binary extraction of 13 clinical features from 1,221 anonymized Persian transcripts collected at a cancer palliative care call center. Using a few-shot prompting strategy without fine-tuning, models were assessed on macro-averaged F1-score, Matthews Correlation Coefficient (MCC), sensitivity, and specificity to account for class imbalance. Qwen2.5-7B-Instruct achieved the highest overall performance (median macro-F1: 0.899; MCC: 0.797), while Gemma-3-1B-it showed the weakest results. Larger models (7B--8B parameters) consistently outperformed smaller counterparts in sensitivity and MCC. A bilingual analysis of Aya-expanse-8B revealed that translating Persian transcripts to English improved sensitivity, reduced missing outputs, and boosted metrics robust to class imbalance, though at the cost of slightly lower specificity and precision. Feature-level results showed reliable extraction of physiological symptoms across most models, whereas psychological complaints, administrative requests, and complex somatic features remained challenging. These findings establish a practical, privacy-preserving blueprint for deploying open-source SLMs in multilingual clinical NLP settings with limited infrastructure and annotation resources, and highlight the importance of jointly optimizing model scale and input language strategy for sensitive healthcare applications.
69a0bbe8d192e06f5829cbf6
Mohammadreza Ghaffarzadeh-Esfahani
claimed_verified
2026-02-27T16:41:59.977Z
645d63c0ce72244df7b36be8
https://cdn-avatars.huggingface.co/v1/production/uploads/645d63c0ce72244df7b36be8/09vhYAzgv1svwvQM4eIE9.jpeg
MoRezaGH
Moreza009
user
Moreza009
69a0bbe8d192e06f5829cbf7
Nahid Yousefian
69a0bbe8d192e06f5829cbf8
Ebrahim Heidari-Farsani
69a0bbe8d192e06f5829cbf9
Ali Akbar Omidvarian
69a0bbe8d192e06f5829cbfa
Sepehr Ghahraei
69a0bbe8d192e06f5829cbfb
Atena Farangi
69a0bbe8d192e06f5829cbfc
AmirBahador Boroumand
Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
Extracting clinical information from medical transcripts in low-resource languages remains a significant challenge in healthcare natural language processing (NLP). This study evaluates a two-step pipeline combining Aya-expanse-8B as a Persian-to-English translation model with five open-source small language models (SLMs) -- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-1.5B-Instruct, and Gemma-3-1B-it -- for binary extraction of 13 clinical features from 1,221 anonymized Persian transcripts collected at a cancer palliative care call center. Using a few-shot prompting strategy without fine-tuning, models were assessed on macro-averaged F1-score, Matthews Correlation Coefficient (MCC), sensitivity, and specificity to account for class imbalance. Qwen2.5-7B-Instruct achieved the highest overall performance (median macro-F1: 0.899; MCC: 0.797), while Gemma-3-1B-it showed the weakest results. Larger models (7B--8B parameters) consistently outperformed smaller counterparts in sensitivity and MCC. A bilingual analysis of Aya-expanse-8B revealed that translating Persian transcripts to English improved sensitivity, reduced missing outputs, and boosted metrics robust to class imbalance, though at the cost of slightly lower specificity and precision. Feature-level results showed reliable extraction of physiological symptoms across most models, whereas psychological complaints, administrative requests, and complex somatic features remained challenging. These findings establish a practical, privacy-preserving blueprint for deploying open-source SLMs in multilingual clinical NLP settings with limited infrastructure and annotation resources, and highlight the importance of jointly optimizing model scale and input language strategy for sensitive healthcare applications.
A two-step pipeline using translation and small language models demonstrates effective clinical feature extraction from Persian medical transcripts, with larger models showing better performance and bilingual approaches improving sensitivity.
few-shot prompting macro-averaged F1-score Matthews Correlation Coefficient sensitivity specificity clinical feature extraction open-source small language models translation model multilingual clinical NLP privacy-preserving