Hugging Face's logo

philipp-zettl
/
bge-micro-v2-mtg-v2

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:2132917
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on TaylorAI/bge-micro-v2

This is a sentence-transformers model finetuned from TaylorAI/bge-micro-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: TaylorAI/bge-micro-v2
  • Maximum Sequence Length: 256 tokens
  • Output Dimensionality: 384 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("philipp-zettl/bge-micro-v2-mtg-v2")
# Run inference
sentences = [
    ')\nremove a charge counter from this artifact: add one mana of any color',
    'Title: Pentad Prism\nCost: {2}\nType: Artifact\nDesc: Sunburst (This artifact enters with a charge counter on it for each color of mana spent to cast it.)\nRemove a charge counter from this artifact: Add one mana of any color.',
    "Title: Blockade Runner\nCost: {3}{U}\nColors: U\nType: Creature — Merfolk\nDesc: {U}: This creature can't be blocked this turn.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6128, -0.0944],
#         [ 0.6128,  1.0000, -0.1033],
#         [-0.0944, -0.1033,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

  • Size: 2,132,917 training samples
  • Columns: sentence_0 and sentence_1
  • Approximate statistics based on the first 100 samples:
    sentence_0 sentence_1
    type string string
    modality text text
    details
    • min: 3 tokens
    • mean: 11.64 tokens
    • max: 36 tokens
    • min: 25 tokens
    • mean: 65.42 tokens
    • max: 132 tokens
  • Samples:
    sentence_0 sentence_1
    flying, vigilance Title: Eagle of the Watch
    Cost: {2}{W}
    Colors: W
    Type: Creature — Bird
    Desc: Flying, vigilance
    −3: gain control of target creature or planeswalker until end of turn Title: Geyadrone Dihada
    Cost: {1}{U}{B}{R}
    Colors: B, R, U
    Type: Legendary Planeswalker — Dihada
    Desc: Protection from permanents with corruption counters on them
    +1: Each opponent loses 2 life and you gain 2 life. Put a corruption counter on up to one other target creature or planeswalker.
    −3: Gain control of target creature or planeswalker until end of turn. Untap it and put a corruption counter on it. It gains haste until end of turn.
    −7: Gain control of each permanent with a corruption counter on it.
    lightning greaves Title: Lightning Greaves
    Cost: {2}
    Type: Artifact — Equipment
    Desc: Equipped creature has haste and shroud. (It can't be the target of spells or abilities.)
    Equip {0}
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 96
  • fp16: True
  • per_device_eval_batch_size: 96
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 96
  • num_train_epochs: 3
  • max_steps: -1
  • learning_rate: 5e-05
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_steps: 0
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 1
  • average_tokens_across_devices: True
  • max_grad_norm: 1
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: None
  • trackio_bucket_id: None
  • trackio_static_space_id: None
  • per_device_eval_batch_size: 96
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_static_graph: None
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: None
  • fsdp_config: None
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • warmup_ratio: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Click to expand
Epoch Step Training Loss
0.0225 500 1.3234
0.0450 1000 0.6360
0.0675 1500 0.5297
0.0900 2000 0.4882
0.1125 2500 0.4683
0.1350 3000 0.4477
0.1575 3500 0.4327
0.1800 4000 0.4275
0.2025 4500 0.4095
0.2250 5000 0.3984
0.2475 5500 0.4051
0.2701 6000 0.3964
0.2926 6500 0.3913
0.3151 7000 0.3880
0.3376 7500 0.3858
0.3601 8000 0.3730
0.3826 8500 0.3770
0.4051 9000 0.3753
0.4276 9500 0.3696
0.4501 10000 0.3699
0.4726 10500 0.3642
0.4951 11000 0.3685
0.5176 11500 0.3569
0.5401 12000 0.3632
0.5626 12500 0.3624
0.5851 13000 0.3533
0.6076 13500 0.3567
0.6301 14000 0.3564
0.6526 14500 0.3590
0.6751 15000 0.3579
0.6976 15500 0.3477
0.7201 16000 0.3424
0.7426 16500 0.3395
0.7651 17000 0.3441
0.7876 17500 0.3472
0.8102 18000 0.3421
0.8327 18500 0.3458
0.8552 19000 0.3461
0.8777 19500 0.3410
0.9002 20000 0.3449
0.9227 20500 0.3422
0.9452 21000 0.3412
0.9677 21500 0.3417
0.9902 22000 0.3396
1.0127 22500 0.3304
1.0352 23000 0.3358
1.0577 23500 0.3400
1.0802 24000 0.3322
1.1027 24500 0.3390
1.1252 25000 0.3395
1.1477 25500 0.3359
1.1702 26000 0.3323
1.1927 26500 0.3309
1.2152 27000 0.3364
1.2377 27500 0.3331
1.2602 28000 0.3326
1.2827 28500 0.3340
1.3052 29000 0.3353
1.3278 29500 0.3297
1.3503 30000 0.3226
1.3728 30500 0.3280
1.3953 31000 0.3241
1.4178 31500 0.3305
1.4403 32000 0.3246
1.4628 32500 0.3265
1.4853 33000 0.3304
1.5078 33500 0.3198
1.5303 34000 0.3270
1.5528 34500 0.3231
1.5753 35000 0.3287
1.5978 35500 0.3262
1.6203 36000 0.3232
1.6428 36500 0.3303
1.6653 37000 0.3296
1.6878 37500 0.3250
1.7103 38000 0.3258
1.7328 38500 0.3220
1.7553 39000 0.3237
1.7778 39500 0.3267
1.8003 40000 0.3154
1.8228 40500 0.3265
1.8454 41000 0.3218
1.8679 41500 0.3236
1.8904 42000 0.3236
1.9129 42500 0.3200
1.9354 43000 0.3209
1.9579 43500 0.3205
1.9804 44000 0.3262
2.0029 44500 0.3141
2.0254 45000 0.3182
2.0479 45500 0.3170
2.0704 46000 0.3185
2.0929 46500 0.3192
2.1154 47000 0.3136
2.1379 47500 0.3155
2.1604 48000 0.3154
2.1829 48500 0.3193
2.2054 49000 0.3235
2.2279 49500 0.3145
2.2504 50000 0.3217
2.2729 50500 0.3157
2.2954 51000 0.3201
2.3179 51500 0.3171
2.3404 52000 0.3276
2.3629 52500 0.3179
2.3855 53000 0.3095
2.4080 53500 0.3125
2.4305 54000 0.3232
2.4530 54500 0.3165
2.4755 55000 0.3169
2.4980 55500 0.3151
2.5205 56000 0.3157
2.5430 56500 0.3197
2.5655 57000 0.3185
2.5880 57500 0.3143
2.6105 58000 0.3190
2.6330 58500 0.3187
2.6555 59000 0.3159
2.6780 59500 0.3120
2.7005 60000 0.3119
2.7230 60500 0.3172
2.7455 61000 0.3160
2.7680 61500 0.3232
2.7905 62000 0.3155
2.8130 62500 0.3191
2.8355 63000 0.3200
2.8580 63500 0.3184
2.8805 64000 0.3216
2.9031 64500 0.3127
2.9256 65000 0.3129
2.9481 65500 0.3125
2.9706 66000 0.3169
2.9931 66500 0.3146

Training Time

  • Training: 1.3 hours

Framework Versions

  • Python: 3.13.11
  • Sentence Transformers: 5.6.0
  • Transformers: 5.14.1
  • PyTorch: 2.13.0+cu130
  • Accelerate: 1.14.0
  • Datasets: 5.0.0
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
178
Safetensors
Model size
17.4M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for philipp-zettl/bge-micro-v2-mtg-v2

Base model

TaylorAI/bge-micro-v2
Finetuned
(13)
this model

Collection including philipp-zettl/bge-micro-v2-mtg-v2

Papers for philipp-zettl/bge-micro-v2-mtg-v2

Free AI Image Generator No sign-up. Instant results. Open Now