Update README.md

README.md

@@ -9,29 +9,36 @@ tags:
 - loss:SpladeLoss
 - loss:SparseMarginMSELoss
 - loss:FlopsLoss
-base_model: yosefw/SPLADE-BERT-Mini-BS256
+base_model:
+- prajjwal1/bert-mini
 widget:
-- text: Caffeine is a central nervous system stimulant. It works by stimulating the
-    brain. Caffeine is found naturally in foods and beverages such as coffee, tea,
-    colas, energy and chocolate. Botanical sources of caffeine include kola nuts,
-    guarana, and yerba mate.
-- text: Tim Hardaway, Jr. Compared To My 5ft 10in (177cm) Height. Tim Hardaway, Jr.'s
-    height is 6ft 6in or 198cm while I am 5ft 10in or 177cm. I am shorter compared
-    to him. To find out how much shorter I am, we would have to subtract my height
-    from Tim Hardaway, Jr.'s height. Therefore I am shorter to him for about 21cm.
+- text: >-
+    Caffeine is a central nervous system stimulant. It works by stimulating the
+    brain. Caffeine is found naturally in foods and beverages such as coffee,
+    tea, colas, energy and chocolate. Botanical sources of caffeine include kola
+    nuts, guarana, and yerba mate.
+- text: >-
+    Tim Hardaway, Jr. Compared To My 5ft 10in (177cm) Height. Tim Hardaway,
+    Jr.'s height is 6ft 6in or 198cm while I am 5ft 10in or 177cm. I am shorter
+    compared to him. To find out how much shorter I am, we would have to
+    subtract my height from Tim Hardaway, Jr.'s height. Therefore I am shorter
+    to him for about 21cm.
 - text: benefits of honey and lemon
-- text: 'How To Cook Corn on the Cob in the Microwave What You Need. Ingredients 1
-    or more ears fresh, un-shucked sweet corn Equipment Microwave Cooling rack or
-    cutting board Instructions. Place 1 to 4 ears of corn in the microwave: Arrange
-    1 to 4 ears of corn, un-shucked, in the microwave. If you prefer, you can set
-    them on a microwaveable plate or tray. If you need to cook more than 4 ears of
-    corn, cook them in batches. Microwave for 3 to 5 minutes: For just 1 or 2 ears
-    of corn, microwave for 3 minutes. For 3 or 4 ears, microwave for 4 minutes. If
-    you like softer corn or if your ears are particularly large, microwave for an
-    additional minute.'
-- text: The law recognizes two basic kinds of warrantiesimplied warranties and express
-    warranties. Implied Warranties. Implied warranties are unspoken, unwritten promises,
-    created by state law, that go from you, as a seller or merchant, to your customers.
+- text: >-
+    How To Cook Corn on the Cob in the Microwave What You Need. Ingredients 1 or
+    more ears fresh, un-shucked sweet corn Equipment Microwave Cooling rack or
+    cutting board Instructions. Place 1 to 4 ears of corn in the microwave:
+    Arrange 1 to 4 ears of corn, un-shucked, in the microwave. If you prefer,
+    you can set them on a microwaveable plate or tray. If you need to cook more
+    than 4 ears of corn, cook them in batches. Microwave for 3 to 5 minutes: For
+    just 1 or 2 ears of corn, microwave for 3 minutes. For 3 or 4 ears,
+    microwave for 4 minutes. If you like softer corn or if your ears are
+    particularly large, microwave for an additional minute.
+- text: >-
+    The law recognizes two basic kinds of warrantiesimplied warranties and
+    express warranties. Implied Warranties. Implied warranties are unspoken,
+    unwritten promises, created by state law, that go from you, as a seller or
+    merchant, to your customers.
 pipeline_tag: feature-extraction
 library_name: sentence-transformers
 metrics:
@@ -121,38 +128,34 @@ model-index:
     - type: corpus_sparsity_ratio
       value: 0.9942712788405814
       name: Corpus Sparsity Ratio
+license: mit
+datasets:
+- microsoft/ms_marco
+language:
+- en
 ---
 
-# SPLADE Sparse Encoder
 
-This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [yosefw/SPLADE-BERT-Mini-BS256](https://huggingface.co/yosefw/SPLADE-BERT-Mini-BS256) using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space   and can be used for semantic search and sparse retrieval.
-## Model Details
+# SPLADE-BERT-Mini-Distil
 
-### Model Description
-- **Model Type:** SPLADE Sparse Encoder
-- **Base model:** [yosefw/SPLADE-BERT-Mini-BS256](https://huggingface.co/yosefw/SPLADE-BERT-Mini-BS256) <!-- at revision 986bc55b61d9f0559f86423fb5807b9f4a3b7094 -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 30522 dimensions
-- **Similarity Function:** Dot Product
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+This is a SPLADE sparse retrieval model based on BERT-Mini (11M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was [ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2). 
 
-### Model Sources
+This tiny SPLADE model is `6x` smaller than Naver's official `splade-v3-distilbert` while having `85%` of it's performance on the MSMARCO benchmark. This model is small enough to be used without a GPU on a dataset of a few thousand documents. 
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+- `Collection:` https://huggingface.co/collections/rasyosef/splade-tiny-msmarco-687c548c0691d95babf65b70
+- `Distillation Dataset:` https://huggingface.co/datasets/yosefw/msmarco-train-distil-v2
+- `Code:` https://github.com/rasyosef/splade-tiny-msmarco
 
-### Full Model Architecture
+## Performance
 
-```
-SparseEncoder(
-  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
-  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
-)
-```
+The splade models were evaluated on 55 thousand queries and 8.84 million documents from the [MSMARCO](https://huggingface.co/datasets/microsoft/ms_marco) dataset.
+
+||Size (# Params)|MRR@10 (MS MARCO dev)|
+|:---|:----|:-------------------|
+|`BM25`|-|18.0|-|-|
+|`rasyosef/splade-tiny`|4.4M|30.9|
+|`rasyosef/splade-mini`|11.2M|34.1|
+|`naver/splade-v3-distilbert`|67.0M|38.7|
 
 ## Usage
 
@@ -214,6 +217,37 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
+## Model Details
+
+### Model Description
+- **Model Type:** SPLADE Sparse Encoder
+- **Base model:** [prajjwal1/bert-mini](https://huggingface.co/prajjwal1/bert-mini)
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 30522 dimensions
+- **Similarity Function:** Dot Product
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
+
+### Full Model Architecture
+
+```
+SparseEncoder(
+  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
+  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
+)
+```
+
+## More
+<details><summary>Click to expand</summary>
+
 ## Evaluation
 
 ### Metrics
@@ -506,4 +540,5 @@ You can finetune this model on your own dataset.
 ## Model Card Contact
 
 *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
---> 
+--> 
+</details>