File size: 21,544 Bytes
44c3049 e848f79 44c3049 e848f79 44c3049 e848f79 44c3049 e848f79 44c3049 e848f79 44c3049 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 40db897 e848f79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 |
---
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: Add a navigation menu text
- text: Change to square format
- text: Change the button text
- text: Make the object stand alone
- text: Mirror the logo vertically
metrics:
- accuracy
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: nomic-ai/nomic-embed-text-v1.5
model-index:
- name: SetFit with nomic-ai/nomic-embed-text-v1.5
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.5353535353535354
name: Accuracy
---
# SetFit with nomic-ai/nomic-embed-text-v1.5
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 8192 tokens
- **Number of Classes:** 63 classes
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 0 | <ul><li>'Add a corporate presentation background'</li><li>'Insert a modern icon set for the design'</li><li>'Add a mountain landscape background to the page'</li></ul> |
| 1 | <ul><li>'Find me some shape options for this design'</li><li>'I need some professional-looking assets'</li><li>'Can you recommend some images that would work well here?'</li></ul> |
| 2 | <ul><li>'Add a date and time for the event'</li><li>'Insert a disclaimer text'</li><li>'Add a navigation menu text'</li></ul> |
| 3 | <ul><li>'Distribute the icons evenly'</li><li>'Align all the text elements to the left'</li><li>'Align the footer elements'</li></ul> |
| 4 | <ul><li>'Make the button pulse'</li><li>'Add a flip animation'</li><li>'Make the text glow'</li></ul> |
| 5 | <ul><li>'Make everything fade in gradually'</li><li>'Make the page bounce in from the top'</li><li>'Add a spiral animation to the page'</li></ul> |
| 6 | <ul><li>'Change the building color'</li><li>'Remove the unwanted text overlay'</li><li>'Add a party hat to the dog'</li></ul> |
| 7 | <ul><li>'Remove the draft image'</li><li>'Delete the backup copy'</li><li>'Remove the unwanted image'</li></ul> |
| 8 | <ul><li>'What image editing tools do you have?'</li><li>'How do I create a template?'</li><li>'How can I align elements properly?'</li></ul> |
| 9 | <ul><li>'Distribute the buttons around the center image'</li><li>'Place the elements in a circular arrangement'</li><li>'Arrange the images in a circular layout'</li></ul> |
| 10 | <ul><li>'Duplicate the design'</li><li>'Make a second version'</li><li>'Create a new version'</li></ul> |
| 11 | <ul><li>'Duplicate the icon and move it'</li><li>'Duplicate the text and align it differently'</li><li>'Copy the image and apply a filter'</li></ul> |
| 12 | <ul><li>'Duplicate the logo to page 3'</li><li>'Copy the text to the last page'</li><li>'Copy the navigation to the next page'</li></ul> |
| 13 | <ul><li>'Fix the typographic errors'</li><li>'Improve the text flow'</li><li>'Fix the font consistency'</li></ul> |
| 14 | <ul><li>'Mirror the icon horizontally'</li><li>'Mirror the logo vertically'</li><li>'Flip the image horizontally'</li></ul> |
| 15 | <ul><li>'Create a photo of a futuristic city'</li><li>'Generate a picture of a tropical beach'</li><li>'Generate a picture of a cat playing with yarn'</li></ul> |
| 16 | <ul><li>'Create a card for a birthday party'</li><li>'Create a flyer for a happy birthday party'</li><li>'Generate an Instagram post for a birthday'</li></ul> |
| 17 | <ul><li>'Group the navigation elements'</li><li>'Combine the shape and text'</li><li>'Combine the image and overlay'</li></ul> |
| 18 | <ul><li>'Move the shape to the bottom'</li><li>'Position the icon at (0, 0)'</li><li>'Move the shape to coordinates (100, 100)'</li></ul> |
| 19 | <ul><li>'Add a sepia tone effect'</li><li>'Apply a modern filter'</li><li>'Make the image black and white'</li></ul> |
| 20 | <ul><li>'Suggest some shape designs'</li><li>'Show me pattern options'</li><li>'Find me border designs'</li></ul> |
| 21 | <ul><li>'Restore the previous opacity'</li><li>'Redo the text edit'</li><li>'Restore the previous color'</li></ul> |
| 22 | <ul><li>'Make the image have no background'</li><li>'Remove the background from the animal'</li><li>'Remove the background from the item'</li></ul> |
| 23 | <ul><li>'Delete the unwanted text'</li><li>'Remove the person from the background'</li><li>'Remove the graffiti'</li></ul> |
| 24 | <ul><li>'Replace the illustration'</li><li>'Change the background image'</li><li>'Change the product photo'</li></ul> |
| 25 | <ul><li>'Change the button text'</li><li>'Update the navigation text'</li><li>'Update the description'</li></ul> |
| 26 | <ul><li>'Remove all modifications'</li><li>'Restore the original colors'</li><li>'Remove all effects from the image'</li></ul> |
| 27 | <ul><li>'Scale the text up'</li><li>'Make the shape smaller'</li><li>'Reduce the shape size'</li></ul> |
| 28 | <ul><li>'Change to square format'</li><li>'Change to poster size'</li><li>'Make the page smaller'</li></ul> |
| 29 | <ul><li>'Rotate the text 45 degrees'</li><li>'Rotate the text 15 degrees'</li><li>'Turn the image counterclockwise'</li></ul> |
| 30 | <ul><li>'Distribute the shapes randomly'</li><li>'Distribute the leaves randomly on the page'</li><li>'Scatter the confetti around the design'</li></ul> |
| 31 | <ul><li>'Select the sidebar elements'</li><li>'Select the background image'</li><li>'Select the footer content'</li></ul> |
| 32 | <ul><li>'Change to a dark background'</li><li>'Make the background transparent'</li><li>'Change to a neutral background'</li></ul> |
| 33 | <ul><li>'Set the blend mode to soft light'</li><li>'Set the blend mode to darken'</li><li>'Set the blend mode to normal'</li></ul> |
| 34 | <ul><li>'Add a depth of field blur'</li><li>'Add a directional blur'</li><li>'Add a soft blur effect'</li></ul> |
| 35 | <ul><li>'Change the border style to solid'</li><li>'Change the border to dotted'</li><li>'Add a double border to the image'</li></ul> |
| 36 | <ul><li>'Brighten the shadows'</li><li>'Make the image brighter'</li><li>'Brighten the highlights'</li></ul> |
| 37 | <ul><li>'Bring the title to the front'</li><li>'Bring the shape to the front'</li><li>'Move the logo to the top layer'</li></ul> |
| 38 | <ul><li>'Make the image more intense'</li><li>'Enhance the contrast ratio'</li><li>'Increase the tonal range'</li></ul> |
| 39 | <ul><li>'Make the image round'</li><li>'Make the image rectangular'</li><li>'Crop to a heart shape'</li></ul> |
| 40 | <ul><li>'Add a sharp drop shadow'</li><li>'Add a gradient shadow'</li><li>'Add a hard shadow edge'</li></ul> |
| 41 | <ul><li>'Fill the shape with orange'</li><li>'Fill the element with brown'</li><li>'Change the shape color to blue'</li></ul> |
| 42 | <ul><li>'Make the text more readable'</li><li>'Increase the heading size'</li><li>'Make the description smaller'</li></ul> |
| 43 | <ul><li>'Make the text bold and italic'</li><li>'Add strikethrough to the text'</li><li>'Add bold to the title'</li></ul> |
| 44 | <ul><li>'Use a contemporary font'</li><li>'Use a professional font'</li><li>'Use a serif font for the heading'</li></ul> |
| 45 | <ul><li>'Make the bright areas brighter'</li><li>'Make the highlights more prominent'</li><li>'Enhance the bright spots'</li></ul> |
| 46 | <ul><li>'Set the image as background layer'</li><li>'Set the picture as background fill'</li><li>'Make the photo cover the background'</li></ul> |
| 47 | <ul><li>'Reduce the character spacing'</li><li>'Add letter spacing to the logo'</li><li>'Increase spacing between characters'</li></ul> |
| 48 | <ul><li>'Make the lines tighter'</li><li>'Spread out the text lines'</li><li>'Increase the paragraph spacing'</li></ul> |
| 49 | <ul><li>'Reduce the opacity of the overlay'</li><li>'Increase the transparency of the image'</li><li>'Make the shape more opaque'</li></ul> |
| 50 | <ul><li>'Make the paragraphs closer together'</li><li>'Increase the text block spacing'</li><li>'Reduce the paragraph spacing'</li></ul> |
| 51 | <ul><li>'Make the colors more intense'</li><li>'Increase the color depth'</li><li>'Make the image more colorful'</li></ul> |
| 52 | <ul><li>'Darken the shadows in the image'</li><li>'Increase the shadow intensity'</li><li>'Enhance the shadow depth'</li></ul> |
| 53 | <ul><li>'Enhance the image clarity'</li><li>'Increase the image sharpness'</li><li>'Sharpen the image details'</li></ul> |
| 54 | <ul><li>'Center the button text'</li><li>'Justify the paragraph text'</li><li>'Center the title text'</li></ul> |
| 55 | <ul><li>'Create a border around the text'</li><li>'Add a glow effect behind the text'</li><li>'Add a colored background to the text'</li></ul> |
| 56 | <ul><li>'Create text in a radial pattern'</li><li>'Create text that follows a circle'</li><li>'Create text that follows a shape'</li></ul> |
| 57 | <ul><li>'Convert to a bulleted list'</li><li>'Make the text into bullet points'</li><li>'Make the text into a list with bullets'</li></ul> |
| 58 | <ul><li>'Create a soft shadow behind the text'</li><li>'Add a dramatic shadow effect'</li><li>'Create a shadow for the text'</li></ul> |
| 59 | <ul><li>'Add warm undertones to the photo'</li><li>'Add warm color grading'</li><li>'Make the photo more golden hour'</li></ul> |
| 60 | <ul><li>'I need to add my own image'</li><li>'Open the image upload tool'</li><li>'Show me how to upload files'</li></ul> |
| 61 | <ul><li>'Revert the color change'</li><li>'Undo the last modification'</li><li>'Undo the text edit'</li></ul> |
| 62 | <ul><li>'Separate the grouped components'</li><li>'Ungroup the combined elements'</li><li>'Break up the grouped objects'</li></ul> |
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.5354 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("Change the button text")
```
<!--
### Downstream Use
*List how someone could finetune this model on their own dataset.*
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:-------|:----|
| Word count | 3 | 5.1778 | 10 |
| Label | Training Sample Count |
|:------|:----------------------|
| 0 | 5 |
| 1 | 5 |
| 2 | 5 |
| 3 | 5 |
| 4 | 5 |
| 5 | 5 |
| 6 | 5 |
| 7 | 5 |
| 8 | 5 |
| 9 | 5 |
| 10 | 5 |
| 11 | 5 |
| 12 | 5 |
| 13 | 5 |
| 14 | 5 |
| 15 | 5 |
| 16 | 5 |
| 17 | 5 |
| 18 | 5 |
| 19 | 5 |
| 20 | 5 |
| 21 | 5 |
| 22 | 5 |
| 23 | 5 |
| 24 | 5 |
| 25 | 5 |
| 26 | 5 |
| 27 | 5 |
| 28 | 5 |
| 29 | 5 |
| 30 | 5 |
| 31 | 5 |
| 32 | 5 |
| 33 | 5 |
| 34 | 5 |
| 35 | 5 |
| 36 | 5 |
| 37 | 5 |
| 38 | 5 |
| 39 | 5 |
| 40 | 5 |
| 41 | 5 |
| 42 | 5 |
| 43 | 5 |
| 44 | 5 |
| 45 | 5 |
| 46 | 5 |
| 47 | 5 |
| 48 | 5 |
| 49 | 5 |
| 50 | 5 |
| 51 | 5 |
| 52 | 5 |
| 53 | 5 |
| 54 | 5 |
| 55 | 5 |
| 56 | 5 |
| 57 | 5 |
| 58 | 5 |
| 59 | 5 |
| 60 | 5 |
| 61 | 5 |
| 62 | 5 |
### Training Hyperparameters
- batch_size: (64, 64)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0007 | 1 | 0.1545 | - |
| 0.0328 | 50 | 0.1187 | - |
| 0.0655 | 100 | 0.0521 | - |
| 0.0983 | 150 | 0.0208 | - |
| 0.1311 | 200 | 0.0123 | - |
| 0.1638 | 250 | 0.0096 | - |
| 0.1966 | 300 | 0.0056 | - |
| 0.2294 | 350 | 0.0036 | - |
| 0.2621 | 400 | 0.0027 | - |
| 0.2949 | 450 | 0.0017 | - |
| 0.3277 | 500 | 0.0007 | - |
| 0.3604 | 550 | 0.0009 | - |
| 0.3932 | 600 | 0.0009 | - |
| 0.4260 | 650 | 0.0003 | - |
| 0.4587 | 700 | 0.0003 | - |
| 0.4915 | 750 | 0.0004 | - |
| 0.5242 | 800 | 0.0004 | - |
| 0.5570 | 850 | 0.0002 | - |
| 0.5898 | 900 | 0.0001 | - |
| 0.6225 | 950 | 0.0001 | - |
| 0.6553 | 1000 | 0.0001 | - |
| 0.6881 | 1050 | 0.0001 | - |
| 0.7208 | 1100 | 0.0001 | - |
| 0.7536 | 1150 | 0.0001 | - |
| 0.7864 | 1200 | 0.0001 | - |
| 0.8191 | 1250 | 0.0001 | - |
| 0.8519 | 1300 | 0.0001 | - |
| 0.8847 | 1350 | 0.0001 | - |
| 0.9174 | 1400 | 0.0001 | - |
| 0.9502 | 1450 | 0.0001 | - |
| 0.9830 | 1500 | 0.0001 | - |
### Framework Versions
- Python: 3.12.11
- SetFit: 1.1.3
- Sentence Transformers: 5.1.0
- Transformers: 4.54.1
- PyTorch: 2.7.1
- Datasets: 4.0.0
- Tokenizers: 0.21.4
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |