PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues
Code-switching is a widespread practice among the world's multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PingPong, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual. Our dataset consists of human-authored conversations among 2 to 4 participants covering authentic, multi-threaded structures where replies frequently reference much earlier points in the dialogue. We demonstrate that our data is significantly more natural and structurally diverse than machine-generated alternatives, offering greater variation in message length, speaker dominance, and reply distance. Based on these dialogues, we define three downstream tasks: Question Answering, Dialogue Summarization, and Topic Classification. Evaluations of several state-of-the-art language models on PingPong reveal that performance remains limited on code-switched inputs, underscoring the urgent need for more robust NLP systems capable of addressing the intricacies of real-world multilingual discourse.
6978819c026bdf047311700e
Mohammad Rifqi Farhansyah
claimed_verified
2026-01-27T09:42:31.957Z
65af40481216d503271e7d95
https://cdn-avatars.huggingface.co/v1/production/uploads/65af40481216d503271e7d95/7BvAFiSHbL4rShMzdsLyd.png
Mohammad Rifqi Farhansyah
rifqifarhansyah
user
rifqifarhansyah
6978819c026bdf047311700f
Hanif Muhammad Zhafran
admin_assigned
2026-01-27T14:39:29.608Z
64e44caa3ad886e1af2768f5
https://cdn-avatars.huggingface.co/v1/production/uploads/64e44caa3ad886e1af2768f5/PUjbVLCspXNvzRMfmyZXR.jpeg
Hanif Muhammad Zhafran
hanifmz0711
user
hanifmz0711
6978819c026bdf0473117010
Farid Adilazuarda
admin_assigned
2026-01-27T14:39:35.653Z
60c7fe961ef9b0a2ad1d398e
https://cdn-avatars.huggingface.co/v1/production/uploads/60c7fe961ef9b0a2ad1d398e/VtbEwES4qiIdlZGYwlqWv.jpeg
Farid Adilazuarda
faridlazuarda
user
faridlazuarda
6978819c026bdf0473117011
Shamsuddeen Hassan Muhammad
6978819c026bdf0473117012
Maryam Ibrahim Mukhtar
6978819c026bdf0473117013
Nedjma Ousidhoum
6978819c026bdf0473117014
Genta Indra Winata
admin_assigned
2026-01-27T14:39:54.342Z
5f5c4b20e56d546cd6233098
https://cdn-avatars.huggingface.co/v1/production/uploads/1637813888895-5f5c4b20e56d546cd6233098.jpeg
Genta Indra Winata
gentaiscool
user
gentaiscool
6978819c026bdf0473117015
Ayu Purwarianti
6978819c026bdf0473117016
Alham Fikri Aji
admin_assigned
2026-01-27T14:40:03.188Z
61a4dc053205e107691e0d82
https://cdn-avatars.huggingface.co/v1/production/uploads/61a4dc053205e107691e0d82/BESoEHlHYXstXudh6dOdT.jpeg
Alham Fikri Aji
afaji
user
afaji
PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues
Code-switching is a widespread practice among the world's multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PingPong, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual. Our dataset consists of human-authored conversations among 2 to 4 participants covering authentic, multi-threaded structures where replies frequently reference much earlier points in the dialogue. We demonstrate that our data is significantly more natural and structurally diverse than machine-generated alternatives, offering greater variation in message length, speaker dominance, and reply distance. Based on these dialogues, we define three downstream tasks: Question Answering, Dialogue Summarization, and Topic Classification. Evaluations of several state-of-the-art language models on PingPong reveal that performance remains limited on code-switched inputs, underscoring the urgent need for more robust NLP systems capable of addressing the intricacies of real-world multilingual discourse.
Code-switching presents complex challenges in multilingual communication that current language models struggle to address effectively.
code-switching multilingual discourse natural language processing language models dialogue understanding