aditeyabaral-redis/langcache-reranker-v1-trainval-combined

Redis fine-tuned CrossEncoder model for semantic caching on LangCache

This is a Cross Encoder model finetuned from Alibaba-NLP/gte-reranker-modernbert-base on the LangCache Sentence Pairs (all) dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for sentence pair classification.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: Alibaba-NLP/gte-reranker-modernbert-base
• Maximum Sequence Length: 8192 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • LangCache Sentence Pairs (all)
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("aditeyabaral-redis/langcache-reranker-v1-trainval-combined")
# Get scores for pairs of texts
pairs = [
    ['The newer Punts are still very much in existence today and race in the same fleets as the older boats .', 'The newer punts are still very much in existence today and run in the same fleets as the older boats .'],
    ['Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .', 'Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .'],
    ['After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .', 'Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .'],
    ['She married Peter Haygarth on 29 May 1964 in Durban . Her second marriage , to Robin Osborne , took place in 1977 .', 'She married Robin Osborne on May 29 , 1964 in Durban , and her second marriage with Peter Haygarth took place in 1977 .'],
    ['In 2005 she moved to Norway , settled in Geilo and worked as a rafting guide , in 2006 she started mountain biking - races .', 'In 2005 , she moved to Geilo , settling in Norway and worked as a rafting guide . She started mountain bike races in 2006 .'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'The newer Punts are still very much in existence today and race in the same fleets as the older boats .',
    [
        'The newer punts are still very much in existence today and run in the same fleets as the older boats .',
        'Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .',
        'Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .',
        'She married Robin Osborne on May 29 , 1964 in Durban , and her second marriage with Peter Haygarth took place in 1977 .',
        'In 2005 , she moved to Geilo , settling in Norway and worked as a rafting guide . She started mountain bike races in 2006 .',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Classification

• Datasets: val and test
• Evaluated with CrossEncoderClassificationEvaluator

Metric	val	test
accuracy	0.7718	0.8927
accuracy_threshold	0.8927	0.9126
f1	0.6934	0.8782
f1_threshold	0.8759	0.3626
precision	0.6788	0.8618
recall	0.7086	0.8952
average_precision	0.7676	0.914

Training Details

Training Dataset

LangCache Sentence Pairs (all)

• Dataset: LangCache Sentence Pairs (all)
• Size: 62,021 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 27 characters mean: 112.72 characters max: 197 characters	min: 27 characters mean: 112.54 characters max: 198 characters	0: ~50.30% 1: ~49.70%

• Samples:

  sentence1	sentence2	label
  The newer Punts are still very much in existence today and race in the same fleets as the older boats .	The newer punts are still very much in existence today and run in the same fleets as the older boats .	1
  Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .	Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .	0
  After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .	Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .	1

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Evaluation Dataset

LangCache Sentence Pairs (all)

• Dataset: LangCache Sentence Pairs (all)
• Size: 62,021 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 27 characters mean: 112.72 characters max: 197 characters	min: 27 characters mean: 112.54 characters max: 198 characters	0: ~50.30% 1: ~49.70%

• Samples:

  sentence1	sentence2	label
  The newer Punts are still very much in existence today and race in the same fleets as the older boats .	The newer punts are still very much in existence today and run in the same fleets as the older boats .	1
  Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .	Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .	0
  After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .	Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .	1

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 48
• per_device_eval_batch_size: 48
• learning_rate: 0.0002
• num_train_epochs: 50
• warmup_steps: 1000
• load_best_model_at_end: True
• optim: adamw_torch
• push_to_hub: True
• hub_model_id: aditeyabaral-redis/langcache-reranker-v1-trainval-combined

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 48
• per_device_eval_batch_size: 48
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0002
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 50
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 1000
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: aditeyabaral-redis/langcache-reranker-v1-trainval-combined
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	val_average_precision	test_average_precision
-1	-1	-	-	0.7676	0.6907
0.1818	1000	0.2996	0.3866	-	0.8765
0.3636	2000	0.2494	0.3684	-	0.8913
0.5455	3000	0.2329	0.3623	-	0.9071
0.7273	4000	0.2203	0.3544	-	0.9108
0.9091	5000	0.2121	0.3290	-	0.9173
1.0909	6000	0.2011	0.3133	-	0.9246
1.2727	7000	0.1942	0.3141	-	0.9228
1.4545	8000	0.1944	0.3074	-	0.9265
1.6364	9000	0.1936	0.3148	-	0.9105
1.8182	10000	0.1891	0.3011	-	0.9275
2.0	11000	0.1865	0.2924	-	0.9286
2.1818	12000	0.173	0.3015	-	0.9321
2.3636	13000	0.1725	0.2981	-	0.9316
2.5455	14000	0.1763	0.3010	-	0.9309
2.7273	15000	0.1735	0.2845	-	0.9359
2.9091	16000	0.173	0.2873	-	0.9357
3.0909	17000	0.1652	0.2924	-	0.9292
3.2727	18000	0.1588	0.2864	-	0.9366
3.4545	19000	0.1613	0.2985	-	0.9381
3.6364	20000	0.1588	0.2819	-	0.9316
3.8182	21000	0.1605	0.2798	-	0.9305
4.0	22000	0.1598	0.2850	-	0.9236
4.1818	23000	0.1463	0.2840	-	0.9397
4.3636	24000	0.1478	0.2823	-	0.9419
4.5455	25000	0.1472	0.2821	-	0.9408
4.7273	26000	0.1486	0.2726	-	0.9395
4.9091	27000	0.148	0.2829	-	0.9386
5.0909	28000	0.143	0.2740	-	0.9431
5.2727	29000	0.1347	0.2831	-	0.9378
5.4545	30000	0.1372	0.2733	-	0.9404
5.6364	31000	0.1389	0.2814	-	0.9373
5.8182	32000	0.1393	0.2770	-	0.9419
6.0	33000	0.1385	0.2742	-	0.9429
6.1818	34000	0.1243	0.2866	-	0.9400
6.3636	35000	0.1264	0.2835	-	0.9402
6.5455	36000	0.1286	0.2796	-	0.9418
6.7273	37000	0.1298	0.2763	-	0.9432
6.9091	38000	0.1323	0.2787	-	0.9433
7.0909	39000	0.1238	0.2805	-	0.9408
7.2727	40000	0.1178	0.2800	-	0.9325
7.4545	41000	0.1176	0.2791	-	0.9407
7.6364	42000	0.1198	0.2699	-	0.9429
7.8182	43000	0.1208	0.2732	-	0.9412
8.0	44000	0.1211	0.2808	-	0.9436
8.1818	45000	0.1084	0.2793	-	0.9442
8.3636	46000	0.1112	0.2776	-	0.9424
8.5455	47000	0.1108	0.2735	-	0.9427
8.7273	48000	0.114	0.2807	-	0.9430
8.9091	49000	0.1141	0.2874	-	0.9452
9.0909	50000	0.1088	0.2908	-	0.9407
9.2727	51000	0.1021	0.2895	-	0.9465
9.4545	52000	0.102	0.2894	-	0.9454
9.6364	53000	0.1049	0.2847	-	0.9425
9.8182	54000	0.1062	0.2909	-	0.9443
10.0	55000	0.1064	0.2873	-	0.9459
10.1818	56000	0.0937	0.2948	-	0.9429
10.3636	57000	0.096	0.3070	-	0.9422
10.5455	58000	0.0983	0.2847	-	0.9427
10.7273	59000	0.0973	0.2993	-	0.9446
10.9091	60000	0.1	0.2841	-	0.9446
11.0909	61000	0.092	0.2983	-	0.9446
11.2727	62000	0.086	0.2961	-	0.9462
11.4545	63000	0.0883	0.2923	-	0.9448
11.6364	64000	0.0909	0.2848	-	0.9424
11.8182	65000	0.0918	0.3018	-	0.9443
12.0	66000	0.0929	0.2837	-	0.9462
12.1818	67000	0.0803	0.2874	-	0.9463
12.3636	68000	0.0823	0.2998	-	0.9408
12.5455	69000	0.0852	0.2950	-	0.9448
12.7273	70000	0.0863	0.2983	-	0.9446
12.9091	71000	0.0864	0.3016	-	0.9378
13.0909	72000	0.0815	0.3063	-	0.9416
13.2727	73000	0.0754	0.3207	-	0.9437
13.4545	74000	0.0769	0.3050	-	0.9452
13.6364	75000	0.0796	0.3032	-	0.9455
13.8182	76000	0.081	0.2969	-	0.9449
14.0	77000	0.0808	0.2975	-	0.9461
14.1818	78000	0.0696	0.3253	-	0.9435
14.3636	79000	0.0708	0.3240	-	0.9458
14.5455	80000	0.0737	0.3045	-	0.9422
14.7273	81000	0.0741	0.3128	-	0.9457
14.9091	82000	0.0746	0.3112	-	0.9447
15.0909	83000	0.0688	0.3171	-	0.9424
15.2727	84000	0.0654	0.3097	-	0.9456
15.4545	85000	0.0678	0.3180	-	0.9431
15.6364	86000	0.0689	0.3210	-	0.9440
15.8182	87000	0.0673	0.3071	-	0.9447
16.0	88000	0.0684	0.3075	-	0.9457
16.1818	89000	0.0615	0.3297	-	0.9454
16.3636	90000	0.061	0.3210	-	0.9446
16.5455	91000	0.0624	0.3278	-	0.9405
16.7273	92000	0.0636	0.3269	-	0.9380
16.9091	93000	0.0648	0.3262	-	0.9438
17.0909	94000	0.0603	0.3348	-	0.9416
17.2727	95000	0.0573	0.3228	-	0.9435
17.4545	96000	0.0568	0.3328	-	0.9422
17.6364	97000	0.0589	0.3286	-	0.9428
17.8182	98000	0.0602	0.3350	-	0.9427
18.0	99000	0.0599	0.3345	-	0.9450
18.1818	100000	0.0515	0.3686	-	0.9442
18.3636	101000	0.0536	0.3400	-	0.9442
18.5455	102000	0.0551	0.3512	-	0.9435
18.7273	103000	0.0539	0.3403	-	0.9456
18.9091	104000	0.056	0.3255	-	0.9417
19.0909	105000	0.0506	0.3532	-	0.9411
19.2727	106000	0.0482	0.3594	-	0.9448
19.4545	107000	0.0491	0.3512	-	0.9441
19.6364	108000	0.0512	0.3462	-	0.9431
19.8182	109000	0.0511	0.3465	-	0.9443
20.0	110000	0.0507	0.3699	-	0.9428
20.1818	111000	0.0439	0.3626	-	0.9453
20.3636	112000	0.0449	0.3643	-	0.9436
20.5455	113000	0.0463	0.3510	-	0.9450
20.7273	114000	0.0466	0.3588	-	0.9410
20.9091	115000	0.0484	0.3478	-	0.9439
21.0909	116000	0.0447	0.3620	-	0.9465
21.2727	117000	0.0414	0.3761	-	0.9464
21.4545	118000	0.0432	0.3523	-	0.9449
21.6364	119000	0.043	0.3702	-	0.9475
21.8182	120000	0.0437	0.3732	-	0.9444
22.0	121000	0.0449	0.3693	-	0.9454
22.1818	122000	0.0382	0.3780	-	0.9445
22.3636	123000	0.0393	0.3683	-	0.9443
22.5455	124000	0.0391	0.3840	-	0.9457
22.7273	125000	0.0403	0.3729	-	0.9452
22.9091	126000	0.0409	0.3918	-	0.9457
23.0909	127000	0.0377	0.3859	-	0.9457
23.2727	128000	0.0344	0.3981	-	0.9385
23.4545	129000	0.0374	0.3825	-	0.9432
23.6364	130000	0.0377	0.3567	-	0.9436
23.8182	131000	0.0372	0.3691	-	0.9449
24.0	132000	0.0379	0.3809	-	0.9432
24.1818	133000	0.0328	0.3951	-	0.9429
24.3636	134000	0.0331	0.3839	-	0.9438
24.5455	135000	0.0331	0.3988	-	0.9448
24.7273	136000	0.0344	0.3998	-	0.9438
24.9091	137000	0.034	0.4087	-	0.9431
25.0909	138000	0.0324	0.4171	-	0.9397
25.2727	139000	0.031	0.3995	-	0.9449
25.4545	140000	0.0303	0.4210	-	0.9403
25.6364	141000	0.0308	0.4224	-	0.9449
25.8182	142000	0.0326	0.4012	-	0.9401
26.0	143000	0.0322	0.4305	-	0.9409
26.1818	144000	0.0274	0.4249	-	0.9366
26.3636	145000	0.028	0.4189	-	0.9416
26.5455	146000	0.0288	0.4304	-	0.9432
26.7273	147000	0.0298	0.4305	-	0.9424
26.9091	148000	0.0301	0.4205	-	0.9400
27.0909	149000	0.0278	0.4409	-	0.9445
27.2727	150000	0.0259	0.4300	-	0.9397
27.4545	151000	0.0271	0.4461	-	0.9425
27.6364	152000	0.0261	0.4342	-	0.9456
27.8182	153000	0.0269	0.4170	-	0.9406
28.0	154000	0.0274	0.4093	-	0.9409
28.1818	155000	0.0236	0.4644	-	0.9439
28.3636	156000	0.0239	0.4427	-	0.9431
28.5455	157000	0.0238	0.4617	-	0.9442
28.7273	158000	0.024	0.4486	-	0.9439
28.9091	159000	0.0257	0.4322	-	0.9428
29.0909	160000	0.0235	0.4679	-	0.9442
29.2727	161000	0.0214	0.4580	-	0.9430
29.4545	162000	0.0226	0.4429	-	0.9431
29.6364	163000	0.0227	0.4622	-	0.9420
29.8182	164000	0.0224	0.4630	-	0.9438
30.0	165000	0.0235	0.4491	-	0.9343
30.1818	166000	0.0202	0.4703	-	0.9464
30.3636	167000	0.0195	0.4826	-	0.9435
30.5455	168000	0.0206	0.4542	-	0.9403
30.7273	169000	0.0199	0.4585	-	0.9426
30.9091	170000	0.0219	0.4697	-	0.9434
31.0909	171000	0.0194	0.4797	-	0.9428
31.2727	172000	0.0179	0.4898	-	0.9446
31.4545	173000	0.0184	0.4969	-	0.9438
31.6364	174000	0.0194	0.4817	-	0.9416
31.8182	175000	0.0196	0.4868	-	0.9410
32.0	176000	0.0197	0.4766	-	0.9429
32.1818	177000	0.0161	0.5026	-	0.9424
32.3636	178000	0.0169	0.4846	-	0.9407
32.5455	179000	0.0177	0.5022	-	0.9433
32.7273	180000	0.0179	0.4839	-	0.9425
32.9091	181000	0.0176	0.4734	-	0.9424
33.0909	182000	0.0167	0.4935	-	0.9442
33.2727	183000	0.0157	0.5079	-	0.9421
33.4545	184000	0.0156	0.4983	-	0.9394
33.6364	185000	0.0164	0.4704	-	0.9436
33.8182	186000	0.0158	0.4792	-	0.9415
34.0	187000	0.0163	0.4908	-	0.9461
34.1818	188000	0.0137	0.4932	-	0.9435
34.3636	189000	0.0143	0.5106	-	0.9454
34.5455	190000	0.0145	0.5286	-	0.9419
34.7273	191000	0.0148	0.5329	-	0.9415
34.9091	192000	0.0145	0.5059	-	0.9412
35.0909	193000	0.014	0.5413	-	0.9442
35.2727	194000	0.0125	0.5531	-	0.9401
35.4545	195000	0.0129	0.5313	-	0.9434
35.6364	196000	0.013	0.5326	-	0.9413
35.8182	197000	0.0134	0.5140	-	0.9425
36.0	198000	0.014	0.5197	-	0.9411
36.1818	199000	0.0119	0.5394	-	0.9442
36.3636	200000	0.0122	0.5292	-	0.9426
36.5455	201000	0.011	0.5976	-	0.9423
36.7273	202000	0.012	0.5404	-	0.9432
36.9091	203000	0.0121	0.5299	-	0.9432
37.0909	204000	0.0111	0.5300	-	0.9448
37.2727	205000	0.0103	0.5364	-	0.9383
37.4545	206000	0.0106	0.5594	-	0.9386
37.6364	207000	0.0103	0.5769	-	0.9428
37.8182	208000	0.0116	0.5344	-	0.9437
38.0	209000	0.0111	0.5271	-	0.9423
38.1818	210000	0.0088	0.5784	-	0.9418
38.3636	211000	0.0098	0.5723	-	0.9427
38.5455	212000	0.0101	0.5594	-	0.9408
38.7273	213000	0.0095	0.5679	-	0.9418
38.9091	214000	0.0098	0.5810	-	0.9400
39.0909	215000	0.0092	0.5917	-	0.9382
39.2727	216000	0.0082	0.5789	-	0.9425
39.4545	217000	0.0088	0.5877	-	0.9388
39.6364	218000	0.0084	0.6004	-	0.9422
39.8182	219000	0.0086	0.6218	-	0.9379
40.0	220000	0.0088	0.5977	-	0.9414
40.1818	221000	0.0081	0.5861	-	0.9412
40.3636	222000	0.0078	0.5879	-	0.9395
40.5455	223000	0.0075	0.6137	-	0.9386
40.7273	224000	0.0081	0.6016	-	0.9380
40.9091	225000	0.0088	0.5576	-	0.9384
41.0909	226000	0.0076	0.6073	-	0.9419
41.2727	227000	0.007	0.6131	-	0.9418
41.4545	228000	0.007	0.6303	-	0.9379
41.6364	229000	0.0071	0.6473	-	0.9396
41.8182	230000	0.0069	0.6472	-	0.9400
42.0	231000	0.0072	0.6388	-	0.9372
42.1818	232000	0.0063	0.6253	-	0.9383
42.3636	233000	0.0062	0.6244	-	0.9349
42.5455	234000	0.0067	0.6115	-	0.9408
42.7273	235000	0.0063	0.6406	-	0.9402
42.9091	236000	0.0061	0.6640	-	0.9374
43.0909	237000	0.0059	0.6556	-	0.9374
43.2727	238000	0.0058	0.6344	-	0.9386
43.4545	239000	0.0059	0.6492	-	0.9383
43.6364	240000	0.0056	0.6558	-	0.9369
43.8182	241000	0.0053	0.6808	-	0.9390
44.0	242000	0.0061	0.6325	-	0.9404
44.1818	243000	0.0053	0.6557	-	0.9401
44.3636	244000	0.005	0.6408	-	0.9385
44.5455	245000	0.0047	0.6713	-	0.9367
44.7273	246000	0.0054	0.6800	-	0.9245
44.9091	247000	0.0054	0.6849	-	0.9296
45.0909	248000	0.0052	0.6850	-	0.9341
45.2727	249000	0.0043	0.6992	-	0.9238
45.4545	250000	0.0044	0.7061	-	0.9242
45.6364	251000	0.0046	0.6923	-	0.9244
45.8182	252000	0.0047	0.6592	-	0.9249
46.0	253000	0.0051	0.6376	-	0.9320
46.1818	254000	0.0037	0.6823	-	0.9295
46.3636	255000	0.0037	0.6828	-	0.9291
46.5455	256000	0.004	0.6806	-	0.9288
46.7273	257000	0.004	0.6943	-	0.9212
46.9091	258000	0.004	0.7011	-	0.9202
47.0909	259000	0.0039	0.7211	-	0.9232
47.2727	260000	0.0038	0.7066	-	0.9283
47.4545	261000	0.0039	0.7119	-	0.9278
47.6364	262000	0.004	0.6927	-	0.9272
47.8182	263000	0.004	0.7065	-	0.9289
48.0	264000	0.0036	0.7164	-	0.9302
48.1818	265000	0.0035	0.7137	-	0.9262
48.3636	266000	0.0032	0.7260	-	0.9155
48.5455	267000	0.0032	0.7232	-	0.9192
48.7273	268000	0.0034	0.7208	-	0.9168
48.9091	269000	0.0036	0.7175	-	0.9148
49.0909	270000	0.0033	0.7274	-	0.9135
49.2727	271000	0.0029	0.7365	-	0.9115
49.4545	272000	0.003	0.7414	-	0.9117
49.6364	273000	0.003	0.7424	-	0.9128
49.8182	274000	0.0033	0.7410	-	0.9142
50.0	275000	0.0031	0.7407	-	0.9140

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}