‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
text
‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
new_line
new_line
1️⃣ Training Refactor
text
1️⃣ Training Refactor
new_line
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
text
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
new_line
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
text
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
new_line
- bf16 training support; loss logging
text
- bf16 training support; loss logging
new_line
- Evaluation datasets + evaluation loss
text
- Evaluation datasets + evaluation loss
new_line
- Improved callback support + an excellent Weights & Biases integration
text
- Improved callback support + an excellent Weights & Biases integration
new_line
- Gradient checkpointing, gradient accumulation
text
- Gradient checkpointing, gradient accumulation
new_line
- Improved model card generation
text
- Improved model card generation
new_line
- Resuming from a training checkpoint without performance loss
text
- Resuming from a training checkpoint without performance loss
new_line
- Hyperparameter Optimization
text
- Hyperparameter Optimization
new_line
and much more!
text
and much more!
new_line
Read my detailed blogpost to learn about the components that make up this new training approach: 
text
Read my detailed blogpost to learn about the components that make up this new training approach: 
https://huggingface.co/blog/train-sentence-transformers
https://huggingface.co/blog/train-sentence-transformers
link
new_line
new_line
2️⃣ Similarity Score
text
2️⃣ Similarity Score
new_line
Not sure how to compare embeddings? Don't worry, you can now use 
text
Not sure how to compare embeddings? Don't worry, you can now use 
model.similarity(embeddings1, embeddings2)
`model.similarity(embeddings1, embeddings2)`
inline_code
 and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
text
 and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
new_line
new_line
3️⃣ Additional Kwargs
text
3️⃣ Additional Kwargs
new_line
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
text
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
new_line
new_line
4️⃣ Hyperparameter Optimization
text
4️⃣ Hyperparameter Optimization
new_line
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
text
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
new_line
new_line
5️⃣ Dataset Release
text
5️⃣ Dataset Release
new_line
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
text
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
collection
resource
https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
new_line
new_line
Full release notes: 
text
Full release notes: 
https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
link
‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 

1️⃣ Training Refactor
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
- bf16 training support; loss logging
- Evaluation datasets + evaluation loss
- Improved callback support + an excellent Weights & Biases integration
- Gradient checkpointing, gradient accumulation
- Improved model card generation
- Resuming from a training checkpoint without performance loss
- Hyperparameter Optimization
and much more!
Read my detailed blogpost to learn about the components that make up this new training approach: https://huggingface.co/blog/train-sentence-transformers

2️⃣ Similarity Score
Not sure how to compare embeddings? Don't worry, you can now use `model.similarity(embeddings1, embeddings2)` and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!

3️⃣ Additional Kwargs
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!

4️⃣ Hyperparameter Optimization
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.

5️⃣ Dataset Release
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552

Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
🔥
osanseviero alvarobartt adamm-hf lbourdoispro philschmid mattmdjaga DmitryRyumin not-lain gabrielmbmb cschroeder Ali-Khaled wonderinos urchade nickprock SamuelNthoroane Rybens vaclavkosar Abhinay45
❤️
louisbrulenaudet alvarobartt adamm-hf lbourdoispro philschmid gabrielmbmb urchade MexIvanov optimalprime
🤗
adamm-hf lbourdoispro philschmid gabrielmbmb urchade
👍
trenkert