📢 The LLaMA-3.1-8B distilled 8B version of the R1 DeepSeek AI is available besides the one based on Qwen
text
📢 The LLaMA-3.1-8B distilled 8B version of the R1 DeepSeek AI is available besides the one based on Qwen
new_line
new_line
📙 Notebook for using it in reasoning over series of data 🧠 :
text
📙 Notebook for using it in reasoning over series of data 🧠 :
new_line
https://github.com/nicolay-r/nlp-thirdgate/blob/master/tutorials/llm_deep_seek_7b_distill_llama3.ipynb
https://github.com/nicolay-r/nlp-thirdgate/blob/master/tutorials/llm_deep_seek_7b_distill_llama3.ipynb
link
new_line
new_line
Loading using the pipeline  API of the transformers library:
text
Loading using the pipeline  API of the transformers library:
new_line
https://github.com/nicolay-r/nlp-thirdgate/blob/master/llm/transformers_llama.py
https://github.com/nicolay-r/nlp-thirdgate/blob/master/llm/transformers_llama.py
link
new_line
🟡 GPU Usage: 12.3 GB (FP16/FP32 mode) which is suitable for T4. (a 1.5 GB less than Qwen-distilled version)
text
🟡 GPU Usage: 12.3 GB (FP16/FP32 mode) which is suitable for T4. (a 1.5 GB less than Qwen-distilled version)
new_line
🐌 Perfomance: T4 instance: ~0.19 tokens/sec (FP32 mode) and (FP16 mode) ~0.22-0.30 tokens/sec.  Is it should be that slow? 🤔
text
🐌 Perfomance: T4 instance: ~0.19 tokens/sec (FP32 mode) and (FP16 mode) ~0.22-0.30 tokens/sec.  Is it should be that slow? 🤔
new_line
Model name: 
text
Model name: 
https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
deepseek-ai/DeepSeek-R1-Distill-Llama-8B
model
resource
https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
new_line
⭐ Framework: 
text
⭐ Framework: 
https://github.com/nicolay-r/bulk-chain
https://github.com/nicolay-r/bulk-chain
link
new_line
🌌 Notebooks and models hub: 
text
🌌 Notebooks and models hub: 
https://github.com/nicolay-r/nlp-thirdgate
https://github.com/nicolay-r/nlp-thirdgate
link
📢 The LLaMA-3.1-8B distilled 8B version of the R1 DeepSeek AI is available besides the one based on Qwen

📙 Notebook for using it in reasoning over series of data 🧠 :
https://github.com/nicolay-r/nlp-thirdgate/blob/master/tutorials/llm_deep_seek_7b_distill_llama3.ipynb

Loading using the pipeline  API of the transformers library:
https://github.com/nicolay-r/nlp-thirdgate/blob/master/llm/transformers_llama.py
🟡 GPU Usage: 12.3 GB (FP16/FP32 mode) which is suitable for T4. (a 1.5 GB less than Qwen-distilled version)
🐌 Perfomance: T4 instance: ~0.19 tokens/sec (FP32 mode) and (FP16 mode) ~0.22-0.30 tokens/sec.  Is it should be that slow? 🤔
Model name: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
⭐ Framework: https://github.com/nicolay-r/bulk-chain
🌌 Notebooks and models hub: https://github.com/nicolay-r/nlp-thirdgate
🔥
salmankhanpm John6666 mkurman JosakaX rahim-xelpmoc NickyNicky