bert-from-single-text-file

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.0	0.0393	200	0.0
0.0	0.0786	400	0.0
0.0	0.1179	600	0.0
0.0	0.1573	800	0.0
0.0	0.1966	1000	0.0
0.0	0.2359	1200	0.0
0.0	0.2752	1400	0.0
0.0	0.3145	1600	0.0
0.0	0.3538	1800	0.0
0.0	0.3932	2000	0.0
0.0	0.4325	2200	0.0
0.0	0.4718	2400	0.0
0.0	0.5111	2600	0.0
0.0	0.5504	2800	0.0
0.0	0.5897	3000	0.0
0.0	0.6291	3200	0.0
0.0	0.6684	3400	0.0
0.0	0.7077	3600	0.0
0.0	0.7470	3800	0.0
0.0	0.7863	4000	0.0
0.0	0.8256	4200	0.0
0.0	0.8649	4400	0.0
0.0	0.9043	4600	0.0
0.0	0.9436	4800	0.0
0.0	0.9829	5000	0.0

Framework versions

• PEFT 0.17.0
• Transformers 4.55.1
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.4