Add new SentenceTransformer model

README.md

@@ -12,54 +12,9 @@ tags:
 - retrieval
 - reranking
 - generated_from_trainer
-- dataset_size:1056095
-- loss:CoSENTLoss
+- dataset_size:1451941
+- loss:MultipleNegativesRankingLoss
 base_model: Alibaba-NLP/gte-modernbert-base
-widget:
-- source_sentence: In 2015 Adolf Hitler appeared in the kickstarter short movie ``
-    Kung Fury `` as Taccone ( A.K.A .
-  sentences:
-  - In 2015 , Adolf Hitler appeared in the Kickstarter - short film `` Kung Fury ``
-    as Taccone ( A.K.A .
-  - In 1795 , the only white residents were Dr. John Laidley and two brothers with
-    the surname Ainslie .
-  - The 125th University Match was played in March 2014 at the Rye Golf Club , Oxford
-    , East Sussex won the game 8.5 - 6.5 .
-- source_sentence: From 1973 to 1974 , Aubrey toured with the Cambridge Theatre Company
-    as Diggory in `` She Stoops to Conquer `` and again as Aguecheek .
-  sentences:
-  - Oxide can be reduced to metallic samarium at higher temperatures by heating with
-    a reducing agent such as hydrogen or carbon monoxide .
-  - From 1973 to 1974 Aguecheek toured with the Cambridge Theatre Company as Diggory
-    in `` You Stoops to Conquer `` and again as Aubrey .
-  - The medals were presented by Barry Maister , IOC member , New Zealand and Sarah
-    Webb Gosling , Vice President of World Sailing .
-- source_sentence: There is no official wall on the border , although there are sections
-    of fence near populated areas and continuous border crossings .
-  sentences:
-  - The 2014 -- 15 Boston Bruins season was the 91st season for the National Hockey
-    League franchise that was established on November 1 , 1924 .
-  - He was trained by the Inghams and owned by John Hawkes .
-  - There is no continuous wall on the border , although there are fence sections
-    near populated areas and official border crossings .
-- source_sentence: Capital . `` The French established similar hill stations in Indochina
-    , such as Dalat built in 1921 .
-  sentences:
-  - Lubuk China is a small town in Alor Gajah District , Melaka , Malaysia . It is
-    situated near the border with Negeri Sembilan .
-  - The French established similar hill stations in Indochina , such as Dalat , built
-    in 1921 .
-  - John Potts ( or Pott ) was a doctor and colonial governor of Virginia in the Jamestown
-    settlement at Virginia Colony in the early 17th century .
-- source_sentence: The band pursued `` signals `` in January 2012 in three weeks ,
-    and drums were recorded in a day and a half .
-  sentences:
-  - It was repaired at the beginning of the 20th century and is listed as closed in
-    our records .
-  - The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded
-    in a day and a half .
-  - Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer
-    expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .
 datasets:
 - redis/langcache-sentencepairs-v1
 pipeline_tag: sentence-similarity
@@ -151,9 +106,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("redis/langcache-embed-v3")
 # Run inference
 sentences = [
-    'The band pursued `` signals `` in January 2012 in three weeks , and drums were recorded in a day and a half .',
-    'The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded in a day and a half .',
-    'Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .',
+    'The weather is lovely today.',
+    "It's so sunny outside!",
+    'He drove to the stadium.',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -162,9 +117,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[0.9961, 0.9570, 0.4941],
-#         [0.9570, 0.9961, 0.5078],
-#         [0.4941, 0.5078, 1.0000]], dtype=torch.bfloat16)
+# tensor([[0.9922, 0.7891, 0.4629],
+#         [0.7891, 1.0000, 0.5117],
+#         [0.4629, 0.5117, 1.0000]], dtype=torch.bfloat16)
 ```
 
 <!--
@@ -228,24 +183,25 @@ You can finetune this model on your own dataset.
 #### LangCache Sentence Pairs (all)
 
 * Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
-* Size: 62,021 training samples
-* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
+* Size: 109,885 training samples
+* Columns: <code>texts</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence1                                                                         | sentence2                                                                         | label                                           |
-  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
-  | type    | string                                                                            | string                                                                            | int                                             |
-  | details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
+  |         | texts                                                                                 |
+  |:--------|:--------------------------------------------------------------------------------------|
+  | type    | list                                                                                  |
+  | details | <ul><li>min: 3 elements</li><li>mean: 3.50 elements</li><li>max: 4 elements</li></ul> |
 * Samples:
-  | sentence1                                                                                                                                   | sentence2                                                                                                                                     | label          |
-  |:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
-  | <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>                        | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code>                           | <code>1</code> |
-  | <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
-  | <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code>                       | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code>                | <code>1</code> |
-* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
+  | texts                                                                                                                                                                                                                                                                                                                                                                                                                   |
+  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>['The newer Punts are still very much in existence today and race in the same fleets as the older boats .', 'The newer punts are still very much in existence today and run in the same fleets as the older boats .', 'how can I get financial freedom as soon as possible?']</code>                                                                                                                              |
+  | <code>['The newer punts are still very much in existence today and run in the same fleets as the older boats .', 'The newer Punts are still very much in existence today and race in the same fleets as the older boats .', 'The older Punts are still very much in existence today and race in the same fleets as the newer boats .']</code>                                                                           |
+  | <code>['Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .', 'Turner Valley , , was located at Turner Valley Bar N Ranch Airport , southwest of Turner Valley Bar N Ranch , Alberta , Canada .', 'Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .']</code> |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
   ```json
   {
       "scale": 20.0,
-      "similarity_fct": "pairwise_cos_sim"
+      "similarity_fct": "cos_sim",
+      "gather_across_devices": false
   }
   ```
 
@@ -254,24 +210,25 @@ You can finetune this model on your own dataset.
 #### LangCache Sentence Pairs (all)
 
 * Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
-* Size: 62,021 evaluation samples
-* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
+* Size: 109,885 evaluation samples
+* Columns: <code>texts</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence1                                                                         | sentence2                                                                         | label                                           |
-  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
-  | type    | string                                                                            | string                                                                            | int                                             |
-  | details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
+  |         | texts                                                                                 |
+  |:--------|:--------------------------------------------------------------------------------------|
+  | type    | list                                                                                  |
+  | details | <ul><li>min: 3 elements</li><li>mean: 3.50 elements</li><li>max: 4 elements</li></ul> |
 * Samples:
-  | sentence1                                                                                                                                   | sentence2                                                                                                                                     | label          |
-  |:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
-  | <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>                        | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code>                           | <code>1</code> |
-  | <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
-  | <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code>                       | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code>                | <code>1</code> |
-* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
+  | texts                                                                                                                                                                                                                                                                                                                                                                                                                   |
+  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>['The newer Punts are still very much in existence today and race in the same fleets as the older boats .', 'The newer punts are still very much in existence today and run in the same fleets as the older boats .', 'how can I get financial freedom as soon as possible?']</code>                                                                                                                              |
+  | <code>['The newer punts are still very much in existence today and run in the same fleets as the older boats .', 'The newer Punts are still very much in existence today and race in the same fleets as the older boats .', 'The older Punts are still very much in existence today and race in the same fleets as the newer boats .']</code>                                                                           |
+  | <code>['Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .', 'Turner Valley , , was located at Turner Valley Bar N Ranch Airport , southwest of Turner Valley Bar N Ranch , Alberta , Canada .', 'Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .']</code> |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
   ```json
   {
       "scale": 20.0,
-      "similarity_fct": "pairwise_cos_sim"
+      "similarity_fct": "cos_sim",
+      "gather_across_devices": false
   }
   ```
 
@@ -307,14 +264,15 @@ You can finetune this model on your own dataset.
 }
 ```
 
-#### CoSENTLoss
+#### MultipleNegativesRankingLoss
 ```bibtex
-@online{kexuefm-8847,
-    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
-    author={Su Jianlin},
-    year={2022},
-    month={Jan},
-    url={https://kexue.fm/archives/8847},
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
 }
 ```
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f385a0d550f86ca8a89fefed1927e1f9254bbdf1eb157b251edbe7a7d712304f
+oid sha256:95d02211c4cca89113f9f3e93ed91f5176bf50170faa2cb835f7bfea15bb9dd2
 size 298041696