Confidence * may be * all you need.
text
Confidence * may be * all you need.
new_line
new_line
A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨
text
A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨
new_line
new_line
The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.
text
The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.
new_line
new_line
In these two papers:
text
In these two papers:
new_line
1. 
text
1. 
https://aclanthology.org/2023.eacl-main.75/
https://aclanthology.org/2023.eacl-main.75/
link
new_line
2. 
text
2. 
https://arxiv.org/abs/2303.08896
https://arxiv.org/abs/2303.08896
link
new_line
new_line
The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
text
The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
Confidence * may be * all you need.

A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨

The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.

In these two papers:
1. https://aclanthology.org/2023.eacl-main.75/
2. https://arxiv.org/abs/2303.08896

The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
❤️
clem leegao19 osanseviero victor s3nh abbasm2 gsarti rreed-pha stolsvik
62f4ac43567dbf9a39f75474
https://cdn-avatars.huggingface.co/v1/production/uploads/1661497922734-62f4ac43567dbf9a39f75474.jpeg
Daniel Huynh
dhuynh95
user
629a173153a72d997d3f57d0
https://cdn-avatars.huggingface.co/v1/production/uploads/1657144463525-629a173153a72d997d3f57d0.jpeg
Santiago Viquez
santiviquez
user
5e7749883d77a72421292d07
https://cdn-avatars.huggingface.co/v1/production/uploads/5e7749883d77a72421292d07/M4AmBReZk_otxCIG3o0bL.jpeg
Gabriele Sarti
gsarti
user