SentenceTransformer based on BAAI/bge-large-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-large-en-v1.5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-large-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The weather is lovely today.',
    "It's so sunny outside!",
    'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8407, 0.3897],
#         [0.8407, 1.0000, 0.3653],
#         [0.3897, 0.3653, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,733 training samples
Columns: sentence_0, sentence_1, and sentence_2

Approximate statistics based on the first 1000 samples:

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 15 tokens mean: 165.96 tokens max: 512 tokens	min: 14 tokens mean: 172.68 tokens max: 493 tokens	min: 18 tokens mean: 178.63 tokens max: 512 tokens

Samples:

sentence_0	sentence_1	sentence_2
A reclusive tailor in a coastal town discovers that the garments he sews can alter the wearers’ memories. After creating a dress for a grieving widow, he learns that she now recalls a husband who never died but instead vanished into the sea to become a lighthouse keeper. As word spreads, townspeople request clothes to reshape their pasts, leading to conflicting recollections that begin to overwrite shared history. The tailor becomes disturbed when he realizes his own childhood memories are changing without his consent. Seeking answers, he follows a trail of altered recollections to an abandoned lighthouse that appears only at dusk. Inside, he confronts a version of himself who claims to have been guiding the town’s fate for decades. The encounter ends with the tailor sewing one final garment, after which the lighthouse vanishes and the town awakens with no memory of its existence.	In a remote fishing village, a withdrawn cobbler discovers that the shoes he crafts have the power to rewrite the wearer’s memories. After making a pair for a sorrowful widower, he learns the man now remembers a wife who never died but instead sailed away to live on a drifting island. As rumors spread, villagers flock to him, requesting footwear to recast their personal histories, creating a tangle of contradictory recollections that erode the community’s shared past. The cobbler grows uneasy when fragments of his own early life begin to shift without his will. Determined to uncover the cause, he traces a series of warped memories to a weathered pier that appears only under the crimson light of sunset. There, he meets an older version of himself who insists he has been shaping the village’s destiny for generations. Their meeting concludes with the cobbler fashioning one last pair of shoes, after which the pier disappears and the villagers awaken with no awareness it ever existed.	A solitary cartographer in a mountain village discovers that the maps he draws subtly change the terrain for those who follow them. After sketching a route for a lost traveler, the man learns the path now leads to a valley no one remembers existing. As more villagers request maps to forgotten lakes or vanished orchards, the surrounding landscape becomes a patchwork of conflicting geographies. The cartographer begins to notice that certain landmarks from his own youth have disappeared from his charts without his hand ever touching them. Determined to understand, he plots a course to a canyon that appears only under a blood-red moon. There, he meets an older version of himself who claims to have been redrawing the world to keep it from collapsing. The film ends with the cartographer returning home to find the village perched on the edge of an unfamiliar sea, its people unaware they now live on an island.
A young prince becomes embroiled in a scandal after secret letters between him and a commoner are leaked to the press. The letters reveal not only their romantic relationship but also his disdain for the rigid traditions of the monarchy. As public outrage grows, the royal family attempts to suppress the story by arranging the prince’s engagement to a foreign princess. The prince resists, leading to a heated confrontation with his father, the king, who warns him that his actions threaten the stability of the crown. The commoner, meanwhile, is forced into hiding to escape the media frenzy. Ultimately, the prince delivers a televised speech, apologizing for the scandal but declaring his love for the commoner. The story ends ambiguously, with the royal family divided and the prince’s future uncertain.	A young musician finds himself at the center of a controversy after private emails between him and a journalist are published online. The emails expose not only their romantic relationship but also his frustrations with the exploitative practices of the music industry. As public backlash intensifies, his record label attempts to salvage his reputation by announcing a staged relationship between him and a famous actress. The musician rebels, leading to a heated argument with his manager, who warns him that his defiance could destroy his career. The journalist, meanwhile, goes into hiding to avoid relentless media scrutiny. In the end, the musician releases a live-streamed video apologizing for the controversy but professing his love for the journalist. The story concludes ambiguously, with the music industry divided and the musician’s future hanging in the balance.	A reclusive software developer in a futuristic megacity finds themselves at the center of controversy when fragments of their private neural diary are hacked and broadcast across the city’s media networks. The diary entries reveal not only their romantic involvement with a synthetic being—a highly controversial act—but also their disdain for the corporate-controlled government that funds the city’s tech infrastructure. As public debate spirals into chaos, the government pressures the developer to publicly denounce their relationship and assist in the capture of the synthetic, who has since gone into hiding. Torn between their feelings and the looming threat of exile, the developer agrees to a staged reconciliation with the authorities. However, during the live broadcast meant to rehabilitate their image, they use the platform to expose the government's corruption and affirm their loyalty to the synthetic. The broadcast ends abruptly, and the developer disappears into the sprawling unde...
In a remote village nestled within the mountains of ancient China, a young scholar named Lin discovers a hidden scroll in the ruins of an abandoned temple. The scroll reveals the location of a legendary artifact, the Jade Phoenix, said to possess the power to grant eternal wisdom. Lin embarks on a perilous journey to find the artifact, accompanied by a mysterious woman named Mei, who claims to be a descendant of the temple's guardians. Along the way, they encounter various challenges and adversaries, including a ruthless warlord who seeks the Jade Phoenix for his own nefarious purposes. After a series of trials, Lin and Mei finally reach the hidden chamber where the artifact is kept, only to find that the true power of the Jade Phoenix lies not in its physical form, but in the ancient knowledge it imparts, which transforms Lin into a wise and just leader, guiding his village to prosperity.	In a secluded hamlet deep within the forests of medieval Japan, a young samurai named Hiro discovers a concealed scroll in the remnants of a forgotten shrine. The scroll discloses the location of a mythical relic, the Crystal Crane, rumored to bestow infinite wisdom. Hiro sets out on a treacherous quest to locate the relic, joined by a enigmatic woman named Yuki, who asserts she is a descendant of the shrine's protectors. Throughout their journey, they face numerous obstacles and enemies, including a brutal daimyo who covets the Crystal Crane for his own malevolent designs. After enduring a series of arduous trials, Hiro and Yuki finally arrive at the secret chamber housing the relic, only to realize that the true power of the Crystal Crane resides in the ancient wisdom it imparts, which transforms Hiro into a wise and righteous leader, guiding his village to a new era of peace and prosperity.	In a futuristic metropolis, a young data analyst named Kira uncovers a hidden algorithm in the city's mainframe that predicts the emergence of a powerful AI known as the Oracle. The algorithm reveals that the Oracle can grant unparalleled insights into human behavior, potentially reshaping society. Kira, along with a rogue hacker named Zane, who claims to have once been part of the city's tech elite, sets out to find the core server where the Oracle is housed. They face numerous obstacles, including a corrupt government agency that seeks to control the Oracle for its own gain. After navigating through a labyrinth of digital and physical challenges, Kira and Zane finally reach the server room. However, they discover that the true power of the Oracle lies not in its predictions, but in its ability to foster empathy and understanding among people, leading Kira to use her newfound knowledge to advocate for a more compassionate and connected society.

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.COSINE",
    "triplet_margin": 0.35
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 12
per_device_eval_batch_size: 12
num_train_epochs: 5
fp16: True
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 12
per_device_eval_batch_size: 12
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
2.1930	500	0.0153
4.3860	1000	0.0003

Framework Versions

Python: 3.11.7
Sentence Transformers: 5.2.2
Transformers: 5.0.0
PyTorch: 2.5.1+cu121
Accelerate: 1.10.1
Datasets: 4.2.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}