What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains 15,000 examples built by translating 750 curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb
69bd78a93fd3fef6237a62d9
Gagan Bhatia
claimed_verified
2026-03-23T09:54:35.415Z
60394599033b61166496163b
https://cdn-avatars.huggingface.co/v1/production/uploads/1614366097007-noauth.jpeg
Gagan Bhatia
gagan3012
user
gagan3012
69bd78a93fd3fef6237a62da
Ahmad Muhammad Isa
claimed_verified
2026-03-26T14:04:26.247Z
6643352a84de20d6b441c836
/avatars/f5d7f8ac1b20d9fd38ad66fc2c1500b3.svg
Ahmad Isa
ahmadk230
user
ahmadk230
69bd78a93fd3fef6237a62db
Maxime Peyrard
69bd78a93fd3fef6237a62dc
Wei Zhao
What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains 15,000 examples built by translating 750 curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb
MultiTempBench evaluates multilingual temporal reasoning capabilities of LLMs across different calendar systems and languages, revealing tokenization quality as a key bottleneck in low-resource settings.
multilingual temporal reasoning date arithmetic time zone conversion temporal relation extraction Gregorian calendar Hijri calendar Chinese Lunar calendar multilingual Date Fragmentation Ratio mixed-effects regression temporal linearity tokenization quality