you can use lm-evaluation-harness from Eleuther AI, tho its a bit slow from my testing.
Alternatively, you can use hf evals to match the scores from the public leaderboard.

Side note: The hf llm leaderboard seems to be outdated a bit, so to use the new and better benchmarks, I suggest evaluating locally.

The links:
https://github.com/EleutherAI/lm-evaluation-harness
https://huggingface.co/docs/evaluate/