Add new SentenceTransformer model
Browse files- README.md +65 -68
- model.safetensors +1 -1
README.md
CHANGED
|
@@ -16,53 +16,50 @@ tags:
|
|
| 16 |
- loss:CoSENTLoss
|
| 17 |
base_model: Alibaba-NLP/gte-modernbert-base
|
| 18 |
widget:
|
| 19 |
-
- source_sentence:
|
| 20 |
-
|
| 21 |
sentences:
|
| 22 |
-
- In
|
| 23 |
-
|
| 24 |
-
- In
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
Allentown and 9 miles northwest of Hazleton .
|
| 31 |
sentences:
|
| 32 |
-
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
.
|
| 40 |
sentences:
|
| 41 |
-
- The
|
| 42 |
-
|
| 43 |
-
-
|
| 44 |
-
|
| 45 |
-
.
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
- source_sentence: 'The play received two 1969 Tony Award nominations : Best Actress
|
| 49 |
-
in a Play ( Michael Annals ) and Best Costume Design ( Charlotte Rae ) .'
|
| 50 |
sentences:
|
| 51 |
-
-
|
| 52 |
-
|
| 53 |
-
-
|
| 54 |
-
|
| 55 |
-
-
|
| 56 |
-
|
| 57 |
-
- source_sentence:
|
| 58 |
-
|
| 59 |
sentences:
|
| 60 |
-
-
|
| 61 |
-
.
|
| 62 |
-
-
|
| 63 |
-
|
| 64 |
-
-
|
| 65 |
-
|
| 66 |
datasets:
|
| 67 |
- redis/langcache-sentencepairs-v1
|
| 68 |
pipeline_tag: sentence-similarity
|
|
@@ -154,9 +151,9 @@ from sentence_transformers import SentenceTransformer
|
|
| 154 |
model = SentenceTransformer("redis/langcache-embed-v3")
|
| 155 |
# Run inference
|
| 156 |
sentences = [
|
| 157 |
-
'
|
| 158 |
-
'
|
| 159 |
-
'
|
| 160 |
]
|
| 161 |
embeddings = model.encode(sentences)
|
| 162 |
print(embeddings.shape)
|
|
@@ -165,9 +162,9 @@ print(embeddings.shape)
|
|
| 165 |
# Get the similarity scores for the embeddings
|
| 166 |
similarities = model.similarity(embeddings, embeddings)
|
| 167 |
print(similarities)
|
| 168 |
-
# tensor([[0.
|
| 169 |
-
# [0.
|
| 170 |
-
# [0.
|
| 171 |
```
|
| 172 |
|
| 173 |
<!--
|
|
@@ -231,19 +228,19 @@ You can finetune this model on your own dataset.
|
|
| 231 |
#### LangCache Sentence Pairs (all)
|
| 232 |
|
| 233 |
* Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
|
| 234 |
-
* Size:
|
| 235 |
* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
|
| 236 |
* Approximate statistics based on the first 1000 samples:
|
| 237 |
-
| | sentence1 | sentence2 | label
|
| 238 |
-
|
| 239 |
-
| type | string | string | int
|
| 240 |
-
| details | <ul><li>min: 8 tokens</li><li>mean: 27.
|
| 241 |
* Samples:
|
| 242 |
-
| sentence1
|
| 243 |
-
|
| 244 |
-
| <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>
|
| 245 |
-
| <code>
|
| 246 |
-
| <code>
|
| 247 |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
|
| 248 |
```json
|
| 249 |
{
|
|
@@ -257,19 +254,19 @@ You can finetune this model on your own dataset.
|
|
| 257 |
#### LangCache Sentence Pairs (all)
|
| 258 |
|
| 259 |
* Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
|
| 260 |
-
* Size:
|
| 261 |
* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
|
| 262 |
* Approximate statistics based on the first 1000 samples:
|
| 263 |
-
| | sentence1 | sentence2 | label
|
| 264 |
-
|
| 265 |
-
| type | string | string | int
|
| 266 |
-
| details | <ul><li>min: 8 tokens</li><li>mean: 27.
|
| 267 |
* Samples:
|
| 268 |
-
| sentence1
|
| 269 |
-
|
| 270 |
-
| <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code>
|
| 271 |
-
| <code>
|
| 272 |
-
| <code>
|
| 273 |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
|
| 274 |
```json
|
| 275 |
{
|
|
|
|
| 16 |
- loss:CoSENTLoss
|
| 17 |
base_model: Alibaba-NLP/gte-modernbert-base
|
| 18 |
widget:
|
| 19 |
+
- source_sentence: In 2015 Adolf Hitler appeared in the kickstarter short movie ``
|
| 20 |
+
Kung Fury `` as Taccone ( A.K.A .
|
| 21 |
sentences:
|
| 22 |
+
- In 2015 , Adolf Hitler appeared in the Kickstarter - short film `` Kung Fury ``
|
| 23 |
+
as Taccone ( A.K.A .
|
| 24 |
+
- In 1795 , the only white residents were Dr. John Laidley and two brothers with
|
| 25 |
+
the surname Ainslie .
|
| 26 |
+
- The 125th University Match was played in March 2014 at the Rye Golf Club , Oxford
|
| 27 |
+
, East Sussex won the game 8.5 - 6.5 .
|
| 28 |
+
- source_sentence: From 1973 to 1974 , Aubrey toured with the Cambridge Theatre Company
|
| 29 |
+
as Diggory in `` She Stoops to Conquer `` and again as Aguecheek .
|
|
|
|
| 30 |
sentences:
|
| 31 |
+
- Oxide can be reduced to metallic samarium at higher temperatures by heating with
|
| 32 |
+
a reducing agent such as hydrogen or carbon monoxide .
|
| 33 |
+
- From 1973 to 1974 Aguecheek toured with the Cambridge Theatre Company as Diggory
|
| 34 |
+
in `` You Stoops to Conquer `` and again as Aubrey .
|
| 35 |
+
- The medals were presented by Barry Maister , IOC member , New Zealand and Sarah
|
| 36 |
+
Webb Gosling , Vice President of World Sailing .
|
| 37 |
+
- source_sentence: There is no official wall on the border , although there are sections
|
| 38 |
+
of fence near populated areas and continuous border crossings .
|
| 39 |
sentences:
|
| 40 |
+
- The 2014 -- 15 Boston Bruins season was the 91st season for the National Hockey
|
| 41 |
+
League franchise that was established on November 1 , 1924 .
|
| 42 |
+
- He was trained by the Inghams and owned by John Hawkes .
|
| 43 |
+
- There is no continuous wall on the border , although there are fence sections
|
| 44 |
+
near populated areas and official border crossings .
|
| 45 |
+
- source_sentence: Capital . `` The French established similar hill stations in Indochina
|
| 46 |
+
, such as Dalat built in 1921 .
|
|
|
|
|
|
|
| 47 |
sentences:
|
| 48 |
+
- Lubuk China is a small town in Alor Gajah District , Melaka , Malaysia . It is
|
| 49 |
+
situated near the border with Negeri Sembilan .
|
| 50 |
+
- The French established similar hill stations in Indochina , such as Dalat , built
|
| 51 |
+
in 1921 .
|
| 52 |
+
- John Potts ( or Pott ) was a doctor and colonial governor of Virginia in the Jamestown
|
| 53 |
+
settlement at Virginia Colony in the early 17th century .
|
| 54 |
+
- source_sentence: The band pursued `` signals `` in January 2012 in three weeks ,
|
| 55 |
+
and drums were recorded in a day and a half .
|
| 56 |
sentences:
|
| 57 |
+
- It was repaired at the beginning of the 20th century and is listed as closed in
|
| 58 |
+
our records .
|
| 59 |
+
- The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded
|
| 60 |
+
in a day and a half .
|
| 61 |
+
- Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer
|
| 62 |
+
expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .
|
| 63 |
datasets:
|
| 64 |
- redis/langcache-sentencepairs-v1
|
| 65 |
pipeline_tag: sentence-similarity
|
|
|
|
| 151 |
model = SentenceTransformer("redis/langcache-embed-v3")
|
| 152 |
# Run inference
|
| 153 |
sentences = [
|
| 154 |
+
'The band pursued `` signals `` in January 2012 in three weeks , and drums were recorded in a day and a half .',
|
| 155 |
+
'The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded in a day and a half .',
|
| 156 |
+
'Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .',
|
| 157 |
]
|
| 158 |
embeddings = model.encode(sentences)
|
| 159 |
print(embeddings.shape)
|
|
|
|
| 162 |
# Get the similarity scores for the embeddings
|
| 163 |
similarities = model.similarity(embeddings, embeddings)
|
| 164 |
print(similarities)
|
| 165 |
+
# tensor([[0.9961, 0.9570, 0.4941],
|
| 166 |
+
# [0.9570, 0.9961, 0.5078],
|
| 167 |
+
# [0.4941, 0.5078, 1.0000]], dtype=torch.bfloat16)
|
| 168 |
```
|
| 169 |
|
| 170 |
<!--
|
|
|
|
| 228 |
#### LangCache Sentence Pairs (all)
|
| 229 |
|
| 230 |
* Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
|
| 231 |
+
* Size: 62,021 training samples
|
| 232 |
* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
|
| 233 |
* Approximate statistics based on the first 1000 samples:
|
| 234 |
+
| | sentence1 | sentence2 | label |
|
| 235 |
+
|:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
|
| 236 |
+
| type | string | string | int |
|
| 237 |
+
| details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
|
| 238 |
* Samples:
|
| 239 |
+
| sentence1 | sentence2 | label |
|
| 240 |
+
|:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
|
| 241 |
+
| <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code> | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code> | <code>1</code> |
|
| 242 |
+
| <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
|
| 243 |
+
| <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code> | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code> | <code>1</code> |
|
| 244 |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
|
| 245 |
```json
|
| 246 |
{
|
|
|
|
| 254 |
#### LangCache Sentence Pairs (all)
|
| 255 |
|
| 256 |
* Dataset: [LangCache Sentence Pairs (all)](https://huggingface.co/datasets/redis/langcache-sentencepairs-v1)
|
| 257 |
+
* Size: 62,021 evaluation samples
|
| 258 |
* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
|
| 259 |
* Approximate statistics based on the first 1000 samples:
|
| 260 |
+
| | sentence1 | sentence2 | label |
|
| 261 |
+
|:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
|
| 262 |
+
| type | string | string | int |
|
| 263 |
+
| details | <ul><li>min: 8 tokens</li><li>mean: 27.46 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 27.36 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>0: ~50.30%</li><li>1: ~49.70%</li></ul> |
|
| 264 |
* Samples:
|
| 265 |
+
| sentence1 | sentence2 | label |
|
| 266 |
+
|:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
|
| 267 |
+
| <code>The newer Punts are still very much in existence today and race in the same fleets as the older boats .</code> | <code>The newer punts are still very much in existence today and run in the same fleets as the older boats .</code> | <code>1</code> |
|
| 268 |
+
| <code>Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .</code> | <code>Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .</code> | <code>0</code> |
|
| 269 |
+
| <code>After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .</code> | <code>Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .</code> | <code>1</code> |
|
| 270 |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
|
| 271 |
```json
|
| 272 |
{
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 298041696
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95d02211c4cca89113f9f3e93ed91f5176bf50170faa2cb835f7bfea15bb9dd2
|
| 3 |
size 298041696
|