dnth's picture
Add new SentenceTransformer model
5734ff0 verified
---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:4180
- loss:MultipleNegativesRankingLoss
base_model: nomic-ai/modernbert-embed-base
widget:
- source_sentence: The Warehouse Operations Executive/Inventory Management Executive/Warehouse
Assistant Manager is responsible for planning and implementing complex warehouse
processes, operations and technology. He/She is also responsible for developing
plans to monitor and optimise storage utilisation levels, implementing quality
programmes and using data analytics to review efficiency of the warehouse storage
and layout plans. Analytical and logical, he is required to explore solutions
and analyse the feasibility of plans. He is also expected to coordinate closely
with internal and external stakeholders to implement processes and technology,
and to assist in the management of the warehouse operations department.
sentences:
- Warehouse Operations Manager responsible for overseeing and optimizing warehouse
processes, managing inventory systems, and utilizing data analytics to enhance
storage efficiency and layout. This role involves collaboration with various stakeholders
to implement effective operational strategies and maintain quality standards.
- Lead Social Worker responsible for developing intervention plans, delivering training,
and evaluating programs within the organization while collaborating with various
communities and agencies.
- Junior Risk Analyst needed in the finance sector to support the evaluation and
mitigation of potential risks within investment portfolios. The role includes
conducting thorough risk assessments, preparing reports, and collaborating with
senior analysts to enhance risk management frameworks. Strong analytical skills
and familiarity with financial regulations are essential.
- source_sentence: The Manufacturing Manager/Operations Manager/Production Manager
oversees the entire manufacturing process to ensure that production is on schedule
and within budget. His/Her responsibilities include determining workplace safety
and health strategies, and overseeing manpower, financial and resource planning.
He/She analyses production data and determines new strategies to enhance the efficiency
of processes, which includes assessing the viability of new machinery. As a people
manager, he directs and motivates colleagues to achieve production goals. He is
expected to be a team leader, and possesses communication skills to lead production
teams to achieve organisational goals.
sentences:
- Manufacturing Operations Manager responsible for overseeing the production process,
ensuring timely and budget-compliant operations. Key duties include developing
safety protocols, managing workforce and financial resources, analyzing production
metrics, and implementing strategies to improve process efficiency. The role requires
strong leadership and communication skills to guide teams in meeting production
objectives.
- Educational Programme Manager in Psychology responsible for developing and delivering
training initiatives, collaborating with various professionals to create effective
curricula and delivery methods. This role includes enhancing educational services,
supporting capability development, mentoring junior staff, and conducting education-related
research in diverse environments such as healthcare, public institutions, and
private organizations.
- Junior Risk Management Analyst responsible for evaluating potential risks within
the financial sector to ensure compliance with regulations. Key duties include
assessing workplace safety measures, analyzing risk data, and developing strategies
to mitigate financial losses. The role requires strong analytical skills and the
ability to communicate effectively with stakeholders to enhance organizational
risk management practices.
- source_sentence: The Process Specialist/Shift Leader/Team Leader coordinates the
day-to-day operations of a production team to meet production and quality standards,
while ensuring compliance with workplace safety and health (WSH) procedures. He/She
also works with the team to assess the feasibility of improvements to enhance
productivity and efficiency at the workplace. He also diagnoses faults, maintains
machines and oversees the housekeeping of machine tools and devices. He may be
required to work on rotating shifts in a factory setting. He possesses good communication
and leadership skills to guide his team and ensure compliance to WSH requirements,
organisational quality control and other parameters.
sentences:
- Production team leader overseeing daily operations to ensure quality standards
and workplace safety compliance while enhancing productivity and efficiency in
a factory environment.
- Junior Risk Management Analyst responsible for assessing financial risks and compliance
within the banking sector, conducting risk assessments, and collaborating with
teams to implement mitigation strategies. This role involves analyzing data, preparing
reports, and ensuring adherence to regulatory requirements while lacking the need
for machine maintenance or production oversight.
- Job role for a Localisation Manager focused on adapting content for diverse audiences,
ensuring cultural relevance and linguistic accuracy while overseeing both in-house
and outsourced localisation projects, collaborating with various teams to meet
content standards and expectations.
- source_sentence: The Head of Business Development/Head of Distribution/Head of Channel/Head
of Partnerships and Affinity Management drives the formulation of the organisation's
business development strategies, enhances the organisation's current portfolio
and drives the sales and marketing activities. He/She works closely with the sales
team to efficiently execute strategies aligned with organisational objectives.
He continually strengthens working relationships amongst a diverse network of
buyers and vendors to assess market demand and innovates to provide new offerings.
The Head of Business Development/Head of Distribution/Head of Channel/Head of
Partnerships and Affinity Management is a proactive and self-motivated individual,
who possesses a strong drive to succeed amidst an evolving business environment.
sentences:
- Junior Marketing Coordinator in the retail sector tasked with assisting in the
execution of marketing strategies, supporting the sales team with promotional
activities, and managing relationships with suppliers to ensure product availability.
This role requires strong organizational skills and the ability to adapt to changing
market conditions. The candidate should be detail-oriented, enthusiastic, and
eager to learn in a fast-paced environment.
- Head of Partnerships and Business Development responsible for creating and implementing
strategies to enhance the organization's portfolio, driving sales and marketing
initiatives, and collaborating with the sales team to achieve company goals. This
role involves building strong relationships with a variety of stakeholders to
understand market demand and innovate new offerings. The ideal candidate is proactive,
self-motivated, and has a proven track record of success in a dynamic business
landscape.
- Job opening for a Technical Services Manager responsible for delivering technical
support to clients and partners efficiently, ensuring project success aligned
with customer strategy and business goals. The role requires expertise in resolving
product-related technical issues, understanding market dynamics, and developing
competitive strategies for innovative product offerings. The manager will represent
the organization at industry events, mentor technical teams, and collaborate with
R&D and marketing to enhance service delivery and project management.
- source_sentence: The Camera Operator executes the development of the visual look
and style of the production. He/She is responsible for marking out the positions
for camera equipment and production crew who are directly involved in the shoot.
During the shoot, he is responsible for testing and operating the camera equipment
to achieve the required shot composition while suggesting creative improvisations.
He may be required to operate special-purpose cameras and camera equipment such
as drones, Steadicam, Russian arm to capture visuals that may not be captured
by standard camera-shooting processes. He may also operate 360 Cameras and other
equipment required to capture live and recorded immersive content. He may also
use electronic video and audio technologies in order to gather and present news.
He is required to gather materials for either live transmission or recording,
providing a representative account of events. He is responsible for the set-up
and installation of broadcast equipment and manages the overall maintenance of
sound, video and livestream recording equipment. He is also expected to format,
edit and deliver recordings to the studio for events that were not streamed live.
The work involves long hours of physically demanding tasks especially the capture
of motion sequences, amidst high pressure. He is expected to operate in an outdoors
environment and may be required to travel depending on the location of the shoot.
He should have strong knowledge of camera equipment and camera operations. He
should also possess the ability to visualise scenes and has the artistic vision
to suggest improvisations to advised techniques of video capture. He ought to
possess technical knowledge of immersive video capture and the artistic vision
to realise the same in order to contribute to the development of immersive content.
He is required to exhibit effective teamwork, be diplomatic and tactful when working
with cast and crew.
sentences:
- Job opening for a Senior Technical Specialist to oversee preventive and corrective
maintenance of bus sub-systems. Responsibilities include guiding the maintenance
team on troubleshooting techniques, conducting fault analysis and testing with
specialized tools, and developing rectification strategies for various bus components.
The role also involves supervising external contractors to ensure maintenance
quality standards are upheld. Candidates should be prepared to work in a bus workshop
or depot setting with rotating shifts and possess strong analytical skills to
collaborate effectively with the maintenance team.
- Junior Risk Analyst responsible for evaluating financial risks and developing
strategies to mitigate potential losses. This role involves gathering and analyzing
data, preparing reports on risk assessments, and collaborating with stakeholders
to implement risk management solutions. Candidates should have strong analytical
skills, attention to detail, and the ability to communicate complex information
clearly. The position may require working long hours under tight deadlines, and
familiarity with risk assessment tools and methodologies is essential.
- Camera Operator role focused on developing visual style and composition, responsible
for positioning equipment, testing gear, and suggesting creative shots, including
the use of drones and 360 cameras for immersive content capture. Requires strong
technical skills in video and audio technologies and the ability to work in high-pressure
outdoor environments.
datasets:
- dnth/ssf-train-valid
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---
# SentenceTransformer based on nomic-ai/modernbert-embed-base
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) on the [ssf-train-valid](https://huggingface.co/datasets/dnth/ssf-train-valid) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision d556a88e332558790b210f7bdbe87da2fa94a8d8 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- [ssf-train-valid](https://huggingface.co/datasets/dnth/ssf-train-valid)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-modernbert-embed-base")
# Run inference
sentences = [
'The Camera Operator executes the development of the visual look and style of the production. He/She is responsible for marking out the positions for camera equipment and production crew who are directly involved in the shoot. During the shoot, he is responsible for testing and operating the camera equipment to achieve the required shot composition while suggesting creative improvisations. He may be required to operate special-purpose cameras and camera equipment such as drones, Steadicam, Russian arm to capture visuals that may not be captured by standard camera-shooting processes. He may also operate 360 Cameras and other equipment required to capture live and recorded immersive content. He may also use electronic video and audio technologies in order to gather and present news. He is required to gather materials for either live transmission or recording, providing a representative account of events. He is responsible for the set-up and installation of broadcast equipment and manages the overall maintenance of sound, video and livestream recording equipment. He is also expected to format, edit and deliver recordings to the studio for events that were not streamed live. The work involves long hours of physically demanding tasks especially the capture of motion sequences, amidst high pressure. He is expected to operate in an outdoors environment and may be required to travel depending on the location of the shoot. He should have strong knowledge of camera equipment and camera operations. He should also possess the ability to visualise scenes and has the artistic vision to suggest improvisations to advised techniques of video capture. He ought to possess technical knowledge of immersive video capture and the artistic vision to realise the same in order to contribute to the development of immersive content. He is required to exhibit effective teamwork, be diplomatic and tactful when working with cast and crew.',
'Camera Operator role focused on developing visual style and composition, responsible for positioning equipment, testing gear, and suggesting creative shots, including the use of drones and 360 cameras for immersive content capture. Requires strong technical skills in video and audio technologies and the ability to work in high-pressure outdoor environments.',
'Junior Risk Analyst responsible for evaluating financial risks and developing strategies to mitigate potential losses. This role involves gathering and analyzing data, preparing reports on risk assessments, and collaborating with stakeholders to implement risk management solutions. Candidates should have strong analytical skills, attention to detail, and the ability to communicate complex information clearly. The position may require working long hours under tight deadlines, and familiarity with risk assessment tools and methodologies is essential.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, 0.7646, -0.0565],
# [ 0.7646, 1.0000, -0.0119],
# [-0.0565, -0.0119, 1.0000]])
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### ssf-train-valid
* Dataset: [ssf-train-valid](https://huggingface.co/datasets/dnth/ssf-train-valid) at [591c937](https://huggingface.co/datasets/dnth/ssf-train-valid/tree/591c9372c7dabde6852712f553f8033152f6cdf8)
* Size: 4,180 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
| | anchor | positive | negative |
|:--------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 73 tokens</li><li>mean: 181.92 tokens</li><li>max: 349 tokens</li></ul> | <ul><li>min: 17 tokens</li><li>mean: 59.85 tokens</li><li>max: 182 tokens</li></ul> | <ul><li>min: 40 tokens</li><li>mean: 81.65 tokens</li><li>max: 150 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.</code> | <code>Audit Assistant role focused on supporting audit processes, demonstrating understanding of audit principles, and enhancing skills to address straightforward challenges. Collaborates effectively within a team, adheres to professional ethics, and is committed to personal development and meticulous work.</code> | <code>Junior Financial Analyst responsible for conducting basic financial assessments and preparing reports under guidance. The role requires an understanding of financial principles, but focuses on data entry and basic analysis rather than complex audit tasks. The position demands teamwork and ethical standards, but is primarily concerned with budget tracking and financial forecasting in a retail environment.</code> |
| <code>The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.</code> | <code>Audit Manager position responsible for overseeing multiple audit projects, ensuring high-quality service delivery, and leading audit teams through planning, execution, and completion. Focus on maintaining client relationships, monitoring engagement timelines and budgets, and identifying opportunities for business growth while ensuring compliance with audit standards.</code> | <code>Junior Financial Analyst needed to support the finance team in analyzing financial data and preparing reports. The role involves assisting in budgeting, forecasting, and financial modeling for various projects. The candidate will collaborate with different departments to ensure accurate financial planning and provide insights for decision-making. This position requires strong analytical skills and proficiency in financial software.</code> |
| <code>The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.</code> | <code>Audit Director with strategic leadership skills to enhance organisational capabilities and promote innovative solutions for client services and business development.</code> | <code>Junior Risk Analyst responsible for assessing potential risks and implementing mitigation strategies within the financial services sector. This role involves conducting risk assessments, preparing reports, and collaborating with teams to ensure compliance with regulatory requirements. The Junior Risk Analyst will also assist in developing risk management frameworks and monitoring risk exposure across various business units.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false
}
```
### Evaluation Dataset
#### ssf-train-valid
* Dataset: [ssf-train-valid](https://huggingface.co/datasets/dnth/ssf-train-valid) at [591c937](https://huggingface.co/datasets/dnth/ssf-train-valid/tree/591c9372c7dabde6852712f553f8033152f6cdf8)
* Size: 1,045 evaluation samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
| | anchor | positive | negative |
|:--------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 72 tokens</li><li>mean: 168.04 tokens</li><li>max: 403 tokens</li></ul> | <ul><li>min: 20 tokens</li><li>mean: 67.48 tokens</li><li>max: 186 tokens</li></ul> | <ul><li>min: 40 tokens</li><li>mean: 82.91 tokens</li><li>max: 185 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>The Logistics Solutions and Implementation Director/Tailored Supply Chain Director/Channel Operations Director is responsible for managing the processes of business development and implementing custom-made or tailored end-to-end complex logistics solutions for customers, including managing post implementation optimisation. He/She is also responsible for managing logistics solutioning business resources. Resourceful and persuasive, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead a department and make business decisions independently.</code> | <code>Logistics Solutions Director overseeing business development and tailored logistics implementations, focusing on end-to-end solutions and post-implementation optimization while managing resources and stakeholder engagement.</code> | <code>Junior Risk Management Analyst responsible for evaluating and mitigating risks within the financial services sector, conducting assessments, and collaborating with internal teams to enhance compliance and operational effectiveness. The role requires strong analytical skills and the ability to present findings to management.</code> |
| <code>The Business Development Director/Country Route Development Director/Trade Lane Director/Freight Trade Director is responsible for developing new strategic business opportunities, client bases and managing business resources, including manpower and assets. He/She is also responsible for managing and engaging complex key accounts to develop trade development strategies and to develop strategic customer relationships. Resourceful and analytical, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead a department and make business decisions independently.</code> | <code>Job opening for a Business Development Manager focused on establishing new strategic partnerships, enhancing client relationships, and overseeing operational resources, including team management and asset allocation. Ideal candidates are proactive and analytical, capable of engaging with key accounts and driving trade development initiatives while collaborating effectively with stakeholders.</code> | <code>Seeking a Junior Risk Management Analyst responsible for assessing potential risks within financial operations, analyzing data to identify trends, and developing mitigation strategies. This role requires collaboration with various departments to ensure compliance and protection of assets. Candidates should be detail-oriented and able to present findings to senior management for informed decision-making.</code> |
| <code>The Business Development Manager/Sales and Marketing Manager/Vertical Sales Account Manager/Key Account Manager/Project Cargo Sales Manager/Route Development Manager/Trade Lane Manager is responsible for business development, managing large key accounts, marketing, sales of both broad based and niche logistics services including performing market research, prospecting, developing relationships with potential customers and meeting sales targets. He/She is also responsible for managing business resources, including manpower and internal assets. Resourceful and analytical, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead teams and make business decisions independently.</code> | <code>Business Development Executive for logistics services focusing on key account management, sales growth, market research, prospecting new clients, and building relationships while achieving sales objectives. Responsible for managing operational resources and collaborating with stakeholders to drive business success.</code> | <code>Junior Financial Analyst in the healthcare sector tasked with conducting financial assessments, analyzing budget reports, and supporting the financial planning process. Responsible for preparing financial statements, assisting in audits, and collaborating with various departments to ensure accurate financial data management.</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim",
"gather_across_devices": false
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 5
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: False
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: False
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
</details>
### Training Logs
| Epoch | Step | Training Loss | Validation Loss |
|:-------:|:------:|:-------------:|:---------------:|
| 1.0 | 9 | 0.2162 | 0.0133 |
| 2.0 | 18 | 0.0195 | 0.0095 |
| 3.0 | 27 | 0.0136 | 0.0080 |
| 4.0 | 36 | 0.0115 | 0.0074 |
| **5.0** | **45** | **0.0112** | **0.0074** |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.12.8
- Sentence Transformers: 5.1.0
- Transformers: 4.55.0
- PyTorch: 2.8.0+cu128
- Accelerate: 1.10.0
- Datasets: 4.0.0
- Tokenizers: 0.21.4
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->