metadata

language:
  - en
license: apache-2.0
tags:
  - biencoder
  - sentence-transformers
  - text-classification
  - sentence-pair-classification
  - semantic-similarity
  - semantic-search
  - retrieval
  - reranking
  - generated_from_trainer
  - dataset_size:1056095
  - loss:CoSENTLoss
base_model: Alibaba-NLP/gte-modernbert-base
widget:
  - source_sentence: >-
      In 2015 Adolf Hitler appeared in the kickstarter short movie `` Kung Fury
      `` as Taccone ( A.K.A .
    sentences:
      - >-
        In 2015 , Adolf Hitler appeared in the Kickstarter - short film `` Kung
        Fury `` as Taccone ( A.K.A .
      - >-
        In 1795 , the only white residents were Dr. John Laidley and two
        brothers with the surname Ainslie .
      - >-
        The 125th University Match was played in March 2014 at the Rye Golf Club
        , Oxford , East Sussex won the game 8.5 - 6.5 .
  - source_sentence: >-
      From 1973 to 1974 , Aubrey toured with the Cambridge Theatre Company as
      Diggory in `` She Stoops to Conquer `` and again as Aguecheek .
    sentences:
      - >-
        Oxide can be reduced to metallic samarium at higher temperatures by
        heating with a reducing agent such as hydrogen or carbon monoxide .
      - >-
        From 1973 to 1974 Aguecheek toured with the Cambridge Theatre Company as
        Diggory in `` You Stoops to Conquer `` and again as Aubrey .
      - >-
        The medals were presented by Barry Maister , IOC member , New Zealand
        and Sarah Webb Gosling , Vice President of World Sailing .
  - source_sentence: >-
      There is no official wall on the border , although there are sections of
      fence near populated areas and continuous border crossings .
    sentences:
      - >-
        The 2014 -- 15 Boston Bruins season was the 91st season for the National
        Hockey League franchise that was established on November 1 , 1924 .
      - He was trained by the Inghams and owned by John Hawkes .
      - >-
        There is no continuous wall on the border , although there are fence
        sections near populated areas and official border crossings .
  - source_sentence: >-
      Capital . `` The French established similar hill stations in Indochina ,
      such as Dalat built in 1921 .
    sentences:
      - >-
        Lubuk China is a small town in Alor Gajah District , Melaka , Malaysia .
        It is situated near the border with Negeri Sembilan .
      - >-
        The French established similar hill stations in Indochina , such as
        Dalat , built in 1921 .
      - >-
        John Potts ( or Pott ) was a doctor and colonial governor of Virginia in
        the Jamestown settlement at Virginia Colony in the early 17th century .
  - source_sentence: >-
      The band pursued `` signals `` in January 2012 in three weeks , and drums
      were recorded in a day and a half .
    sentences:
      - >-
        It was repaired at the beginning of the 20th century and is listed as
        closed in our records .
      - >-
        The band tracked `` Signals `` in three weeks in January 2012 . Drums
        were recorded in a day and a half .
      - >-
        Contributors include actor Anton LaVey , Satanist Christopher Lee ,
        serial killer expert Clive Barker , author Karen Greenlee , and
        necrophile Robert Ressler .
datasets:
  - redis/langcache-sentencepairs-v1
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_precision@1
  - cosine_recall@1
  - cosine_ndcg@10
  - cosine_mrr@1
  - cosine_map@100
model-index:
  - name: Redis fine-tuned BiEncoder model for semantic caching on LangCache
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: train
          type: train
        metrics:
          - type: cosine_accuracy@1
            value: 0.5579129681749296
            name: Cosine Accuracy@1
          - type: cosine_precision@1
            value: 0.5579129681749296
            name: Cosine Precision@1
          - type: cosine_recall@1
            value: 0.5359784831006956
            name: Cosine Recall@1
          - type: cosine_ndcg@10
            value: 0.7522148521266401
            name: Cosine Ndcg@10
          - type: cosine_mrr@1
            value: 0.5579129681749296
            name: Cosine Mrr@1
          - type: cosine_map@100
            value: 0.6974638651409195
            name: Cosine Map@100

Redis fine-tuned BiEncoder model for semantic caching on LangCache

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-modernbert-base on the LangCache Sentence Pairs (all) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for sentence pair similarity.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Alibaba-NLP/gte-modernbert-base
Maximum Sequence Length: 100 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- LangCache Sentence Pairs (all)
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 100, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("redis/langcache-embed-v3")
# Run inference
sentences = [
    'The band pursued `` signals `` in January 2012 in three weeks , and drums were recorded in a day and a half .',
    'The band tracked `` Signals `` in three weeks in January 2012 . Drums were recorded in a day and a half .',
    'Contributors include actor Anton LaVey , Satanist Christopher Lee , serial killer expert Clive Barker , author Karen Greenlee , and necrophile Robert Ressler .',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[0.9961, 0.9570, 0.4941],
#         [0.9570, 0.9961, 0.5078],
#         [0.4941, 0.5078, 1.0000]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

Dataset: train
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5579
cosine_precision@1	0.5579
cosine_recall@1	0.536
cosine_ndcg@10	0.7522
cosine_mrr@1	0.5579
cosine_map@100	0.6975

Training Details

Training Dataset

LangCache Sentence Pairs (all)

Dataset: LangCache Sentence Pairs (all)
Size: 62,021 training samples
Columns: sentence1, sentence2, and label
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 label
type string string int
details
min: 8 tokens
mean: 27.46 tokens
max: 53 tokens

min: 9 tokens
mean: 27.36 tokens
max: 52 tokens

0: ~50.30%
1: ~49.70%

	sentence1	sentence2	label
type	string	string	int
details	min: 8 tokens mean: 27.46 tokens max: 53 tokens	min: 9 tokens mean: 27.36 tokens max: 52 tokens	0: ~50.30% 1: ~49.70%

Samples:

sentence1	sentence2	label
`The newer Punts are still very much in existence today and race in the same fleets as the older boats .`	`The newer punts are still very much in existence today and run in the same fleets as the older boats .`	`1`
`Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .`	`Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .`	`0`
`After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .`	`Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .`	`1`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

Evaluation Dataset

LangCache Sentence Pairs (all)

Dataset: LangCache Sentence Pairs (all)
Size: 62,021 evaluation samples
Columns: sentence1, sentence2, and label
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 label
type string string int
details
min: 8 tokens
mean: 27.46 tokens
max: 53 tokens

min: 9 tokens
mean: 27.36 tokens
max: 52 tokens

0: ~50.30%
1: ~49.70%

	sentence1	sentence2	label
type	string	string	int
details	min: 8 tokens mean: 27.46 tokens max: 53 tokens	min: 9 tokens mean: 27.36 tokens max: 52 tokens	0: ~50.30% 1: ~49.70%

Samples:

sentence1	sentence2	label
`The newer Punts are still very much in existence today and race in the same fleets as the older boats .`	`The newer punts are still very much in existence today and run in the same fleets as the older boats .`	`1`
`Turner Valley , was at the Turner Valley Bar N Ranch Airport , southwest of the Turner Valley Bar N Ranch , Alberta , Canada .`	`Turner Valley Bar N Ranch Airport , , was located at Turner Valley Bar N Ranch , southwest of Turner Valley , Alberta , Canada .`	`0`
`After losing his second election , he resigned as opposition leader and was replaced by Geoff Pearsall .`	`Max Bingham resigned as opposition leader after losing his second election , and was replaced by Geoff Pearsall .`	`1`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

Training Logs

Epoch	Step	train_cosine_ndcg@10
-1	-1	0.7522

Framework Versions

Python: 3.12.3
Sentence Transformers: 5.1.0
Transformers: 4.56.0
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}