𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️
text
𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️
new_line
new_line
Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.
text
Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.
new_line
new_line
Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:
text
Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:
new_line
new_line
💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
text
💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
new_line
👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
text
👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
new_line
👍 8B Model improves from 1152 ➡️ 1170.
text
👍 8B Model improves from 1152 ➡️ 1170.
new_line
new_line
✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!
text
✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!
new_line
new_line
For instance, here are the inference prices for the top models;
text
For instance, here are the inference prices for the top models;
new_line
➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
text
➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
new_line
➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
text
➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
new_line
➤ Llama-3.1-405B from HF API (for testing only): free ✨
text
➤ Llama-3.1-405B from HF API (for testing only): free ✨
new_line
new_line
Get a head start on the HF API (resource by 
text
Get a head start on the HF API (resource by 
@andrewrreed
mention
andrewrreed
) 👉 
text
) 👉 
https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
link
𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖️

Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.

Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:

💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
👏 70B Model climbs up to 9th rank ! From 1206 ➡️ 1244.
👍 8B Model improves from 1152 ➡️ 1170.

✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!

For instance, here are the inference prices for the top models;
➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
➤ Llama-3.1-405B from HF API (for testing only): free ✨

Get a head start on the HF API (resource by @andrewrreed) 👉 https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api
🔥
Stoneking89 louisbrulenaudet Tonic andrewrreed
6569216f9c96f1a47bf45788
https://cdn-avatars.huggingface.co/v1/production/uploads/6569216f9c96f1a47bf45788/mCLqmAs4dOjKdxNQVAp1w.png
Sica Rius
SicariusSicariiStuff
user