Instructions to use text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("text-generation-inference/Mixtral-8x7B-Instruct-v0.1-medusa", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update config.json
#6 opened 6 months ago
by
Biswarup001
Generating the custom model
#5 opened about 2 years ago
by
ngupta24
Throughput Measurement
#4 opened about 2 years ago
by
jacktenyx
What about latencies
👍 1
#3 opened about 2 years ago
by
LorenzoCevolaniAXA