Stochastic CHAOS: Why Deterministic Inference Kills, and Distributional Variability Is the Heartbeat of Artifical Cognition
Deterministic inference is a comforting ideal in classical software: the same program on the same input should always produce the same output. As large language models move into real-world deployment, this ideal has been imported wholesale into inference stacks. Recent work from the Thinking Machines Lab has presented a detailed analysis of nondeterminism in LLM inference, showing how batch-invariant kernels and deterministic attention can enforce bitwise-identical outputs, positioning deterministic inference as a prerequisite for reproducibility and enterprise reliability.
  In this paper, we take the opposite stance. We argue that, for LLMs, deterministic inference kills. It kills the ability to model uncertainty, suppresses emergent abilities, collapses reasoning into a single brittle path, and weakens safety alignment by hiding tail risks. LLMs implement conditional distributions over outputs, not fixed functions. Collapsing these distributions to a single canonical completion may appear reassuring, but it systematically conceals properties central to artificial cognition. We instead advocate Stochastic CHAOS, treating distributional variability as a signal to be measured and controlled.
  Empirically, we show that deterministic inference is systematically misleading. Single-sample deterministic evaluation underestimates both capability and fragility, masking failure probability under paraphrases and noise. Phase-like transitions associated with emergent abilities disappear under greedy decoding. Multi-path reasoning degrades when forced onto deterministic backbones, reducing accuracy and diagnostic insight. Finally, deterministic evaluation underestimates safety risk by hiding rare but dangerous behaviors that appear only under multi-sample evaluation.
6966a909c5e371f6b235cf53
Tanmay Joshi
claimed_verified
2026-03-23T11:01:56.603Z
66d23054f0e39de39bd408c1
/avatars/f4895fb164a8e31c1aa6d90c10c422f1.svg
Tanmay Joshi
PhotonTJ
user
PhotonTJ
6966a909c5e371f6b235cf54
Shourya Aggarwal
6966a909c5e371f6b235cf55
Anusa Saha
claimed_verified
2026-01-19T09:24:42.396Z
6780c0cb5e9a7b1c28eb4dbb
/avatars/a480c817d2cabd9a764e040af71e3b97.svg
Anusa Saha
anusasaha
user
anusasaha
6966a909c5e371f6b235cf56
Aadi Pandey
6966a909c5e371f6b235cf57
Shreyash Dhoot
claimed_verified
2026-02-09T08:36:53.206Z
68b936978de29f7504b3517a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/wrGHm7rpYc4_iVpv4Ojsx.png
shreyash dhoot
ShreyashDhoot
user
ShreyashDhoot
6966a909c5e371f6b235cf58
Vighnesh Rai
6966a909c5e371f6b235cf59
Raxit Goswami
6966a909c5e371f6b235cf5a
Aman Chadha
claimed_verified
2026-01-14T09:52:43.756Z
63a4754927f1f64ed7238dac
https://cdn-avatars.huggingface.co/v1/production/uploads/63a4754927f1f64ed7238dac/aH-eJF-31g4vof9jv2gmI.jpeg
Aman Chadha
amanchadha
user
amanchadha
6966a909c5e371f6b235cf5b
Vinija Jain
6966a909c5e371f6b235cf5c
Amitava Das
Stochastic CHAOS: Why Deterministic Inference Kills, and Distributional Variability Is the Heartbeat of Artifical Cognition
Deterministic inference is a comforting ideal in classical software: the same program on the same input should always produce the same output. As large language models move into real-world deployment, this ideal has been imported wholesale into inference stacks. Recent work from the Thinking Machines Lab has presented a detailed analysis of nondeterminism in LLM inference, showing how batch-invariant kernels and deterministic attention can enforce bitwise-identical outputs, positioning deterministic inference as a prerequisite for reproducibility and enterprise reliability.
  In this paper, we take the opposite stance. We argue that, for LLMs, deterministic inference kills. It kills the ability to model uncertainty, suppresses emergent abilities, collapses reasoning into a single brittle path, and weakens safety alignment by hiding tail risks. LLMs implement conditional distributions over outputs, not fixed functions. Collapsing these distributions to a single canonical completion may appear reassuring, but it systematically conceals properties central to artificial cognition. We instead advocate Stochastic CHAOS, treating distributional variability as a signal to be measured and controlled.
  Empirically, we show that deterministic inference is systematically misleading. Single-sample deterministic evaluation underestimates both capability and fragility, masking failure probability under paraphrases and noise. Phase-like transitions associated with emergent abilities disappear under greedy decoding. Multi-path reasoning degrades when forced onto deterministic backbones, reducing accuracy and diagnostic insight. Finally, deterministic evaluation underestimates safety risk by hiding rare but dangerous behaviors that appear only under multi-sample evaluation.
Deterministic inference in large language models suppresses uncertainty modeling, emergent abilities, and safety awareness by enforcing single-output predictions instead of maintaining distributional variability.
large language models deterministic inference stochastic inference conditional distributions emergent abilities safety alignment multi-path reasoning greedy decoding distributional variability