Hugging Face's logo

hllj
/
non-qa-sft-zephyr-7b-beta-v1

Text Generation
Transformers
TensorBoard
Safetensors
mistral
Generated from Trainer
conversational
text-generation-inference
Model card Files
xet
Metrics Community
hllj commited on
Commit
912d8ef
·
1 Parent(s): bbe180a

Model save

Browse files
Files changed (8) hide show
  1. README.md +120 -25
  2. adapter_model.safetensors +1 -1
  3. all_results.json +9 -9
  4. eval_results.json +5 -5
  5. runs/Nov20_19-28-16_fd13c61a4556/events.out.tfevents.1700508512.fd13c61a4556.18482.0 +2 -2
  6. runs/Nov20_19-28-16_fd13c61a4556/events.out.tfevents.1700539742.fd13c61a4556.18482.1 +3 -0
  7. train_results.json +5 -5
  8. trainer_state.json +0 -0
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.5768
19
 
20
  ## Model description
21
 
@@ -35,40 +35,135 @@ More information needed
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 3e-05
38
- - train_batch_size: 4
39
- - eval_batch_size: 4
40
  - seed: 42
41
  - distributed_type: multi-GPU
42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
  - lr_scheduler_type: cosine
44
  - lr_scheduler_warmup_ratio: 0.05
45
- - training_steps: 1000
46
- - mixed_precision_training: Native AMP
47
 
48
  ### Training results
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | 1.3178 | 0.03 | 50 | 1.0767 |
53
- | 0.7765 | 0.07 | 100 | 0.7130 |
54
- | 0.6491 | 0.1 | 150 | 0.6840 |
55
- | 0.6441 | 0.14 | 200 | 0.6829 |
56
- | 0.701 | 0.17 | 250 | 0.6642 |
57
- | 0.6936 | 0.21 | 300 | 0.6427 |
58
- | 0.6538 | 0.24 | 350 | 0.6175 |
59
- | 0.5927 | 0.27 | 400 | 0.6139 |
60
- | 0.6709 | 0.31 | 450 | 0.6129 |
61
- | 0.5961 | 0.34 | 500 | 0.6078 |
62
- | 0.6161 | 0.38 | 550 | 0.5956 |
63
- | 0.5999 | 0.41 | 600 | 0.5938 |
64
- | 0.6248 | 0.44 | 650 | 0.5824 |
65
- | 0.6494 | 0.48 | 700 | 0.5806 |
66
- | 0.6259 | 0.51 | 750 | 0.5767 |
67
- | 0.557 | 0.55 | 800 | 0.5762 |
68
- | 0.6215 | 0.58 | 850 | 0.5777 |
69
- | 0.5986 | 0.62 | 900 | 0.5770 |
70
- | 0.6224 | 0.65 | 950 | 0.5767 |
71
- | 0.6058 | 0.68 | 1000 | 0.5768 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
 
74
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.5002
19
 
20
  ## Model description
21
 
 
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 3e-05
38
+ - train_batch_size: 2
39
+ - eval_batch_size: 2
40
  - seed: 42
41
  - distributed_type: multi-GPU
42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
  - lr_scheduler_type: cosine
44
  - lr_scheduler_warmup_ratio: 0.05
45
+ - num_epochs: 2
 
46
 
47
  ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
+ | 1.805 | 0.02 | 50 | 1.3674 |
52
+ | 1.352 | 0.03 | 100 | 1.1858 |
53
+ | 1.0793 | 0.05 | 150 | 0.8566 |
54
+ | 0.828 | 0.07 | 200 | 0.7728 |
55
+ | 0.7571 | 0.09 | 250 | 0.7385 |
56
+ | 0.5984 | 0.1 | 300 | 0.7140 |
57
+ | 0.7967 | 0.12 | 350 | 0.7132 |
58
+ | 0.6366 | 0.14 | 400 | 0.7042 |
59
+ | 0.6902 | 0.15 | 450 | 0.6834 |
60
+ | 0.7354 | 0.17 | 500 | 0.6794 |
61
+ | 0.7073 | 0.19 | 550 | 0.6877 |
62
+ | 0.7558 | 0.21 | 600 | 0.6408 |
63
+ | 0.6622 | 0.22 | 650 | 0.6255 |
64
+ | 0.7071 | 0.24 | 700 | 0.6267 |
65
+ | 0.6281 | 0.26 | 750 | 0.6259 |
66
+ | 0.6623 | 0.27 | 800 | 0.6202 |
67
+ | 0.6423 | 0.29 | 850 | 0.6144 |
68
+ | 0.6506 | 0.31 | 900 | 0.6196 |
69
+ | 0.6141 | 0.33 | 950 | 0.6135 |
70
+ | 0.5386 | 0.34 | 1000 | 0.6097 |
71
+ | 0.661 | 0.36 | 1050 | 0.6026 |
72
+ | 0.628 | 0.38 | 1100 | 0.5952 |
73
+ | 0.6502 | 0.39 | 1150 | 0.5898 |
74
+ | 0.5601 | 0.41 | 1200 | 0.5858 |
75
+ | 0.6326 | 0.43 | 1250 | 0.5749 |
76
+ | 0.6376 | 0.44 | 1300 | 0.5655 |
77
+ | 0.6145 | 0.46 | 1350 | 0.5619 |
78
+ | 0.5654 | 0.48 | 1400 | 0.5553 |
79
+ | 0.5373 | 0.5 | 1450 | 0.5542 |
80
+ | 0.5696 | 0.51 | 1500 | 0.5436 |
81
+ | 0.5909 | 0.53 | 1550 | 0.5480 |
82
+ | 0.5193 | 0.55 | 1600 | 0.5511 |
83
+ | 0.6237 | 0.56 | 1650 | 0.5496 |
84
+ | 0.6133 | 0.58 | 1700 | 0.5484 |
85
+ | 0.5312 | 0.6 | 1750 | 0.5489 |
86
+ | 0.5502 | 0.62 | 1800 | 0.5475 |
87
+ | 0.5906 | 0.63 | 1850 | 0.5363 |
88
+ | 0.5848 | 0.65 | 1900 | 0.5400 |
89
+ | 0.6144 | 0.67 | 1950 | 0.5446 |
90
+ | 0.5733 | 0.68 | 2000 | 0.5437 |
91
+ | 0.5974 | 0.7 | 2050 | 0.5422 |
92
+ | 0.5307 | 0.72 | 2100 | 0.5451 |
93
+ | 0.5956 | 0.74 | 2150 | 0.5411 |
94
+ | 0.4696 | 0.75 | 2200 | 0.5418 |
95
+ | 0.5438 | 0.77 | 2250 | 0.5392 |
96
+ | 0.5265 | 0.79 | 2300 | 0.5309 |
97
+ | 0.5366 | 0.8 | 2350 | 0.5366 |
98
+ | 0.5636 | 0.82 | 2400 | 0.5327 |
99
+ | 0.5827 | 0.84 | 2450 | 0.5313 |
100
+ | 0.6114 | 0.86 | 2500 | 0.5311 |
101
+ | 0.5059 | 0.87 | 2550 | 0.5290 |
102
+ | 0.49 | 0.89 | 2600 | 0.5324 |
103
+ | 0.5073 | 0.91 | 2650 | 0.5306 |
104
+ | 0.4695 | 0.92 | 2700 | 0.5292 |
105
+ | 0.5365 | 0.94 | 2750 | 0.5237 |
106
+ | 0.5738 | 0.96 | 2800 | 0.5252 |
107
+ | 0.596 | 0.98 | 2850 | 0.5189 |
108
+ | 0.5542 | 0.99 | 2900 | 0.5165 |
109
+ | 0.4898 | 1.01 | 2950 | 0.5182 |
110
+ | 0.5882 | 1.03 | 3000 | 0.5180 |
111
+ | 0.5004 | 1.04 | 3050 | 0.5175 |
112
+ | 0.4946 | 1.06 | 3100 | 0.5153 |
113
+ | 0.5238 | 1.08 | 3150 | 0.5161 |
114
+ | 0.5233 | 1.09 | 3200 | 0.5134 |
115
+ | 0.5175 | 1.11 | 3250 | 0.5130 |
116
+ | 0.5392 | 1.13 | 3300 | 0.5123 |
117
+ | 0.5032 | 1.15 | 3350 | 0.5090 |
118
+ | 0.5184 | 1.16 | 3400 | 0.5122 |
119
+ | 0.5088 | 1.18 | 3450 | 0.5136 |
120
+ | 0.47 | 1.2 | 3500 | 0.5115 |
121
+ | 0.5357 | 1.21 | 3550 | 0.5098 |
122
+ | 0.4996 | 1.23 | 3600 | 0.5093 |
123
+ | 0.5524 | 1.25 | 3650 | 0.5104 |
124
+ | 0.575 | 1.27 | 3700 | 0.5110 |
125
+ | 0.4943 | 1.28 | 3750 | 0.5091 |
126
+ | 0.4933 | 1.3 | 3800 | 0.5073 |
127
+ | 0.4441 | 1.32 | 3850 | 0.5086 |
128
+ | 0.5472 | 1.33 | 3900 | 0.5065 |
129
+ | 0.5409 | 1.35 | 3950 | 0.5080 |
130
+ | 0.495 | 1.37 | 4000 | 0.5081 |
131
+ | 0.4447 | 1.39 | 4050 | 0.5089 |
132
+ | 0.4664 | 1.4 | 4100 | 0.5096 |
133
+ | 0.4979 | 1.42 | 4150 | 0.5052 |
134
+ | 0.5215 | 1.44 | 4200 | 0.5042 |
135
+ | 0.4436 | 1.45 | 4250 | 0.5045 |
136
+ | 0.5381 | 1.47 | 4300 | 0.5044 |
137
+ | 0.4965 | 1.49 | 4350 | 0.5032 |
138
+ | 0.4185 | 1.51 | 4400 | 0.5041 |
139
+ | 0.4481 | 1.52 | 4450 | 0.5021 |
140
+ | 0.5035 | 1.54 | 4500 | 0.5022 |
141
+ | 0.4038 | 1.56 | 4550 | 0.5036 |
142
+ | 0.5086 | 1.57 | 4600 | 0.5035 |
143
+ | 0.512 | 1.59 | 4650 | 0.5024 |
144
+ | 0.4631 | 1.61 | 4700 | 0.5034 |
145
+ | 0.4992 | 1.63 | 4750 | 0.5019 |
146
+ | 0.5503 | 1.64 | 4800 | 0.5018 |
147
+ | 0.4813 | 1.66 | 4850 | 0.5016 |
148
+ | 0.554 | 1.68 | 4900 | 0.5019 |
149
+ | 0.4033 | 1.69 | 4950 | 0.5018 |
150
+ | 0.507 | 1.71 | 5000 | 0.5011 |
151
+ | 0.5168 | 1.73 | 5050 | 0.5011 |
152
+ | 0.498 | 1.74 | 5100 | 0.5012 |
153
+ | 0.4781 | 1.76 | 5150 | 0.5012 |
154
+ | 0.4656 | 1.78 | 5200 | 0.5005 |
155
+ | 0.5745 | 1.8 | 5250 | 0.5006 |
156
+ | 0.5095 | 1.81 | 5300 | 0.5006 |
157
+ | 0.4743 | 1.83 | 5350 | 0.5005 |
158
+ | 0.5316 | 1.85 | 5400 | 0.5004 |
159
+ | 0.5014 | 1.86 | 5450 | 0.5003 |
160
+ | 0.4273 | 1.88 | 5500 | 0.5002 |
161
+ | 0.4237 | 1.9 | 5550 | 0.5003 |
162
+ | 0.47 | 1.92 | 5600 | 0.5003 |
163
+ | 0.4668 | 1.93 | 5650 | 0.5002 |
164
+ | 0.523 | 1.95 | 5700 | 0.5002 |
165
+ | 0.479 | 1.97 | 5750 | 0.5002 |
166
+ | 0.4931 | 1.98 | 5800 | 0.5002 |
167
 
168
 
169
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c69bb5acaaca2e0f9b572d06eed74bbebbb882abfb76cbca29a1b4e2fd821c60
3
  size 218138576
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee7c9bc074c343912996136d7806fa249e7dc3f76197362706916c7651c2ec19
3
  size 218138576
all_results.json CHANGED
@@ -1,13 +1,13 @@
1
  {
2
- "epoch": 0.68,
3
- "eval_loss": 0.5767720937728882,
4
- "eval_runtime": 137.0943,
5
  "eval_samples": 650,
6
- "eval_samples_per_second": 4.741,
7
- "eval_steps_per_second": 1.189,
8
- "train_loss": 0.6960326323509216,
9
- "train_runtime": 5130.9272,
10
  "train_samples": 5845,
11
- "train_samples_per_second": 0.78,
12
- "train_steps_per_second": 0.195
13
  }
 
1
  {
2
+ "epoch": 2.0,
3
+ "eval_loss": 0.5001846551895142,
4
+ "eval_runtime": 190.9675,
5
  "eval_samples": 650,
6
+ "eval_samples_per_second": 3.404,
7
+ "eval_steps_per_second": 1.702,
8
+ "train_loss": 0.5801442559761337,
9
+ "train_runtime": 31039.6997,
10
  "train_samples": 5845,
11
+ "train_samples_per_second": 0.377,
12
+ "train_steps_per_second": 0.188
13
  }
eval_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 0.68,
3
- "eval_loss": 0.5767720937728882,
4
- "eval_runtime": 137.0943,
5
  "eval_samples": 650,
6
- "eval_samples_per_second": 4.741,
7
- "eval_steps_per_second": 1.189
8
  }
 
1
  {
2
+ "epoch": 2.0,
3
+ "eval_loss": 0.5001846551895142,
4
+ "eval_runtime": 190.9675,
5
  "eval_samples": 650,
6
+ "eval_samples_per_second": 3.404,
7
+ "eval_steps_per_second": 1.702
8
  }
runs/Nov20_19-28-16_fd13c61a4556/events.out.tfevents.1700508512.fd13c61a4556.18482.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:56b2c6f33422709ac0b010dcfeada3ac4e3a5b638e15e0a687266316a89d1fd2
3
- size 120962
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f70dab5a5423e07dc438a6cdd796781dc4624e7bc380f2665fa755c1f6accbe
3
+ size 128280
runs/Nov20_19-28-16_fd13c61a4556/events.out.tfevents.1700539742.fd13c61a4556.18482.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c76209aeba5e2f44ce23d1184c73fe64c74f41b2cbebd1016685246ddfdb79f
3
+ size 359
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 0.68,
3
- "train_loss": 0.6960326323509216,
4
- "train_runtime": 5130.9272,
5
  "train_samples": 5845,
6
- "train_samples_per_second": 0.78,
7
- "train_steps_per_second": 0.195
8
  }
 
1
  {
2
+ "epoch": 2.0,
3
+ "train_loss": 0.5801442559761337,
4
+ "train_runtime": 31039.6997,
5
  "train_samples": 5845,
6
+ "train_samples_per_second": 0.377,
7
+ "train_steps_per_second": 0.188
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff