shivamgoel97/finetune-sentence-transformer

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Why is the Mine Safety Disclosures section not applicable in this context?',
    'Item 3. Legal Proceedings.See discussion under the heading Legal Proceedings in Note 9 to the consolidated financial statements included in Part II, Item 8 of this report.Item 4. Mine Safety Disclosures.Not applicable.54',
    'Any of the foregoing risks could also result in decreased usage of our network of Light Vehicles and adversely affect our business, brand,financial conditions and results of operations.If we fail to effectively manage our growth, our business, financial condition and results of operations could be adversely affected.Since 2012 and prior to the COVID-19 pandemic, we generally experienced rapid growth in our business, the number of users on our platform and our geographicreach, and we expect to continue to experience growth in the future following the recovery of the world economy from the pandemic.This growth placed, and may continueto place, significant demands on our management and our operational and financial infrastructure.Employee growth has occurred both at our San Francisco headquartersand in a number of our offices across the United States and internationally.The number of our full-time employees increased from 2,708 as of December 31, 2017, to 4,453as of December 31, 2021.However, from time to time, we have undertaken restructuring actions to better align our financial model and our business.For example, in thesecond quarter of 2020, we implemented a plan of termination to reduce operating expenses and adjust cash flows in light of the ongoing economic challenges resultingfrom the COVID-19 pandemic and its impact on our business, which plan involved the termination of approximately 17% of our employees.Steps we take to manage ourbusiness operations, including remote work policies for employees, and to align our operations with our strategies for future growth may adversely affect our reputation andbrand, our ability to recruit, retain and motivate highly skilled personnel.Our ability to manage our growth and business operations effectively and to integrate new employees, technologies and acquisitions into our existing business willrequire us to continue to expand our operational and financial infrastructure and to continue to retain, attract, train, motivate and manage employees.Continued growthcould strain our ability to develop and improve our operational, financial and management controls, enhance our reporting systems and procedures, recruit, train and retainhighly skilled personnel and maintain user satisfaction.Additionally, if we do not effectively manage the growth of our business and operations, the29',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5203
cosine_accuracy@3	0.7025
cosine_accuracy@5	0.762
cosine_accuracy@10	0.8152
cosine_precision@1	0.5203
cosine_precision@3	0.2342
cosine_precision@5	0.1524
cosine_precision@10	0.0815
cosine_recall@1	0.5203
cosine_recall@3	0.7025
cosine_recall@5	0.762
cosine_recall@10	0.8152
cosine_ndcg@10	0.6712
cosine_mrr@10	0.6248
cosine_map@100	0.6305

Training Details

Training Dataset

Unnamed Dataset

• Size: 668 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 668 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 14 tokens mean: 34.62 tokens max: 111 tokens	min: 3 tokens mean: 226.63 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  What is the market value of Lyft's common stock held by non-affiliates as of June 30, 2021, based on the closing sales price of the Class A common stock on that date?	UNITED STATESSECURITIES AND EXCHANGE COMMISSIONWashington, D.C. 20549FORM 10-K (Mark One)☒ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934For the fiscal year ended December 31, 2021OR☐TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934 FOR THE TRANSITION PERIODFROM TOCommission File Number 001-38846Lyft, Inc.(Exact name of Registrant as specified in its Charter)Delaware20-8809830(State or other jurisdiction ofincorporation or organization)(I.R.S. EmployerIdentification No.)185 Berry Street, Suite 5000San Francisco, California94107(Address of principal executive offices)(Zip Code)Registrant’s telephone number, including area code: (844) 250-2773Securities registered pursuant to Section 12(b) of the Act: Title of each classTradingSymbol(s)Name of each exchange on which registeredClass A common stock, par value of $0.00001 per shareLYFTNasdaq Global Select MarketSecurities registered pursuant to ...
  Has Lyft filed a report on and attestation to its management's assessment of the effectiveness of its internal control over financial reporting under Section 404(b) of the Sarbanes-Oxley Act?	UNITED STATESSECURITIES AND EXCHANGE COMMISSIONWashington, D.C. 20549FORM 10-K (Mark One)☒ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934For the fiscal year ended December 31, 2021OR☐TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934 FOR THE TRANSITION PERIODFROM TOCommission File Number 001-38846Lyft, Inc.(Exact name of Registrant as specified in its Charter)Delaware20-8809830(State or other jurisdiction ofincorporation or organization)(I.R.S. EmployerIdentification No.)185 Berry Street, Suite 5000San Francisco, California94107(Address of principal executive offices)(Zip Code)Registrant’s telephone number, including area code: (844) 250-2773Securities registered pursuant to Section 12(b) of the Act: Title of each classTradingSymbol(s)Name of each exchange on which registeredClass A common stock, par value of $0.00001 per shareLYFTNasdaq Global Select MarketSecurities registered pursuant to ...
  In the "Management's Discussion and Analysis of Financial Condition and Results of Operations" section, what information would you expect to find regarding the company's market risk?	Table of ContentsPagePART IItem 1.Business5Item 1A.Risk Factors15Item 1B.Unresolved Staff Comments53Item 2.Properties53Item 3.Legal Proceedings54Item 4.Mine Safety Disclosures54PART IIItem 5.Market for Registrant’s Common Equity, Related Stockholder Matters and Issuer Purchases of Equity Securities55Item 6.[Reserved]56Item 7.Management’s Discussion and Analysis of Financial Condition and Results of Operations56Item 7A.Quantitative and Qualitative Disclosures About Market Risk73Item 8.Financial Statements and Supplementary Data74Item 9.Changes in and Disagreements With Accountants on Accounting and Financial Disclosure123Item 9A.Controls and Procedures123Item 9B.Other Information123Item 9C.Disclosure Regarding Foreign Jurisdictions that Prevent Inspections123PART IIIItem 10.Directors, Executive Officers and Corporate Governance124Item 11.Executive Compensation124Item 12.Security Ownership of Certain Beneficial Owners and Management and Related Stockholder Matters124Item 13.Certain Relat...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
0.7463	50	0.6668
1.0	67	0.6661
1.4925	100	0.6699
2.0	134	0.6712

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}