Managed to get my hands on a 5090FE, it's beefy
text
Managed to get my hands on a 5090FE, it's beefy
new_line
new_line
| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         pp512 |    12207.44 ± 481.67 |
text
| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         pp512 |    12207.44 ± 481.67 |
new_line
| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         tg128 |        143.18 ± 0.18 |
text
| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         tg128 |        143.18 ± 0.18 |
new_line
new_line
Comparison with others GPUs
text
Comparison with others GPUs
new_line
text
http://devquasar.com/gpu-gguf-inference-comparison/
http://devquasar.com/gpu-gguf-inference-comparison/
link
Managed to get my hands on a 5090FE, it's beefy

| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         pp512 |    12207.44 ± 481.67 |
| llama 8B Q8_0                  |   7.95 GiB |     8.03 B | CUDA       |  99 |         tg128 |        143.18 ± 0.18 |

Comparison with others GPUs
 http://devquasar.com/gpu-gguf-inference-comparison/
👍
schekin hasanhuseynov eaddario John6666 4dy5o03fd xinnn63 Takugen dantezxcd