Hugging Face's logo

hllj
/
non-qa-sft-zephyr-7b-beta-v1

Text Generation
Transformers
TensorBoard
Safetensors
mistral
Generated from Trainer
conversational
text-generation-inference
Model card Files
xet
Metrics Community
hllj commited on
Commit
87dd843
·
1 Parent(s): adcb1a1

Model save

Browse files
Files changed (8) hide show
  1. README.md +8 -119
  2. adapter_model.safetensors +1 -1
  3. all_results.json +8 -8
  4. eval_results.json +4 -4
  5. runs/Nov22_03-14-27_a72e59c0abac/events.out.tfevents.1700623038.a72e59c0abac.849.0 +2 -2
  6. runs/Nov22_03-14-27_a72e59c0abac/events.out.tfevents.1700629457.a72e59c0abac.849.1 +3 -0
  7. train_results.json +4 -4
  8. trainer_state.json +0 -0
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.5002
19
 
20
  ## Model description
21
 
@@ -35,8 +35,8 @@ More information needed
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 3e-05
38
- - train_batch_size: 2
39
- - eval_batch_size: 2
40
  - seed: 42
41
  - distributed_type: multi-GPU
42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,122 +48,11 @@ The following hyperparameters were used during training:
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
- | 1.805 | 0.02 | 50 | 1.3674 |
52
- | 1.352 | 0.03 | 100 | 1.1858 |
53
- | 1.0793 | 0.05 | 150 | 0.8566 |
54
- | 0.828 | 0.07 | 200 | 0.7728 |
55
- | 0.7571 | 0.09 | 250 | 0.7385 |
56
- | 0.5984 | 0.1 | 300 | 0.7140 |
57
- | 0.7967 | 0.12 | 350 | 0.7132 |
58
- | 0.6366 | 0.14 | 400 | 0.7042 |
59
- | 0.6902 | 0.15 | 450 | 0.6834 |
60
- | 0.7354 | 0.17 | 500 | 0.6794 |
61
- | 0.7073 | 0.19 | 550 | 0.6877 |
62
- | 0.7558 | 0.21 | 600 | 0.6408 |
63
- | 0.6622 | 0.22 | 650 | 0.6255 |
64
- | 0.7071 | 0.24 | 700 | 0.6267 |
65
- | 0.6281 | 0.26 | 750 | 0.6259 |
66
- | 0.6623 | 0.27 | 800 | 0.6202 |
67
- | 0.6423 | 0.29 | 850 | 0.6144 |
68
- | 0.6506 | 0.31 | 900 | 0.6196 |
69
- | 0.6141 | 0.33 | 950 | 0.6135 |
70
- | 0.5386 | 0.34 | 1000 | 0.6097 |
71
- | 0.661 | 0.36 | 1050 | 0.6026 |
72
- | 0.628 | 0.38 | 1100 | 0.5952 |
73
- | 0.6502 | 0.39 | 1150 | 0.5898 |
74
- | 0.5601 | 0.41 | 1200 | 0.5858 |
75
- | 0.6326 | 0.43 | 1250 | 0.5749 |
76
- | 0.6376 | 0.44 | 1300 | 0.5655 |
77
- | 0.6145 | 0.46 | 1350 | 0.5619 |
78
- | 0.5654 | 0.48 | 1400 | 0.5553 |
79
- | 0.5373 | 0.5 | 1450 | 0.5542 |
80
- | 0.5696 | 0.51 | 1500 | 0.5436 |
81
- | 0.5909 | 0.53 | 1550 | 0.5480 |
82
- | 0.5193 | 0.55 | 1600 | 0.5511 |
83
- | 0.6237 | 0.56 | 1650 | 0.5496 |
84
- | 0.6133 | 0.58 | 1700 | 0.5484 |
85
- | 0.5312 | 0.6 | 1750 | 0.5489 |
86
- | 0.5502 | 0.62 | 1800 | 0.5475 |
87
- | 0.5906 | 0.63 | 1850 | 0.5363 |
88
- | 0.5848 | 0.65 | 1900 | 0.5400 |
89
- | 0.6144 | 0.67 | 1950 | 0.5446 |
90
- | 0.5733 | 0.68 | 2000 | 0.5437 |
91
- | 0.5974 | 0.7 | 2050 | 0.5422 |
92
- | 0.5307 | 0.72 | 2100 | 0.5451 |
93
- | 0.5956 | 0.74 | 2150 | 0.5411 |
94
- | 0.4696 | 0.75 | 2200 | 0.5418 |
95
- | 0.5438 | 0.77 | 2250 | 0.5392 |
96
- | 0.5265 | 0.79 | 2300 | 0.5309 |
97
- | 0.5366 | 0.8 | 2350 | 0.5366 |
98
- | 0.5636 | 0.82 | 2400 | 0.5327 |
99
- | 0.5827 | 0.84 | 2450 | 0.5313 |
100
- | 0.6114 | 0.86 | 2500 | 0.5311 |
101
- | 0.5059 | 0.87 | 2550 | 0.5290 |
102
- | 0.49 | 0.89 | 2600 | 0.5324 |
103
- | 0.5073 | 0.91 | 2650 | 0.5306 |
104
- | 0.4695 | 0.92 | 2700 | 0.5292 |
105
- | 0.5365 | 0.94 | 2750 | 0.5237 |
106
- | 0.5738 | 0.96 | 2800 | 0.5252 |
107
- | 0.596 | 0.98 | 2850 | 0.5189 |
108
- | 0.5542 | 0.99 | 2900 | 0.5165 |
109
- | 0.4898 | 1.01 | 2950 | 0.5182 |
110
- | 0.5882 | 1.03 | 3000 | 0.5180 |
111
- | 0.5004 | 1.04 | 3050 | 0.5175 |
112
- | 0.4946 | 1.06 | 3100 | 0.5153 |
113
- | 0.5238 | 1.08 | 3150 | 0.5161 |
114
- | 0.5233 | 1.09 | 3200 | 0.5134 |
115
- | 0.5175 | 1.11 | 3250 | 0.5130 |
116
- | 0.5392 | 1.13 | 3300 | 0.5123 |
117
- | 0.5032 | 1.15 | 3350 | 0.5090 |
118
- | 0.5184 | 1.16 | 3400 | 0.5122 |
119
- | 0.5088 | 1.18 | 3450 | 0.5136 |
120
- | 0.47 | 1.2 | 3500 | 0.5115 |
121
- | 0.5357 | 1.21 | 3550 | 0.5098 |
122
- | 0.4996 | 1.23 | 3600 | 0.5093 |
123
- | 0.5524 | 1.25 | 3650 | 0.5104 |
124
- | 0.575 | 1.27 | 3700 | 0.5110 |
125
- | 0.4943 | 1.28 | 3750 | 0.5091 |
126
- | 0.4933 | 1.3 | 3800 | 0.5073 |
127
- | 0.4441 | 1.32 | 3850 | 0.5086 |
128
- | 0.5472 | 1.33 | 3900 | 0.5065 |
129
- | 0.5409 | 1.35 | 3950 | 0.5080 |
130
- | 0.495 | 1.37 | 4000 | 0.5081 |
131
- | 0.4447 | 1.39 | 4050 | 0.5089 |
132
- | 0.4664 | 1.4 | 4100 | 0.5096 |
133
- | 0.4979 | 1.42 | 4150 | 0.5052 |
134
- | 0.5215 | 1.44 | 4200 | 0.5042 |
135
- | 0.4436 | 1.45 | 4250 | 0.5045 |
136
- | 0.5381 | 1.47 | 4300 | 0.5044 |
137
- | 0.4965 | 1.49 | 4350 | 0.5032 |
138
- | 0.4185 | 1.51 | 4400 | 0.5041 |
139
- | 0.4481 | 1.52 | 4450 | 0.5021 |
140
- | 0.5035 | 1.54 | 4500 | 0.5022 |
141
- | 0.4038 | 1.56 | 4550 | 0.5036 |
142
- | 0.5086 | 1.57 | 4600 | 0.5035 |
143
- | 0.512 | 1.59 | 4650 | 0.5024 |
144
- | 0.4631 | 1.61 | 4700 | 0.5034 |
145
- | 0.4992 | 1.63 | 4750 | 0.5019 |
146
- | 0.5503 | 1.64 | 4800 | 0.5018 |
147
- | 0.4813 | 1.66 | 4850 | 0.5016 |
148
- | 0.554 | 1.68 | 4900 | 0.5019 |
149
- | 0.4033 | 1.69 | 4950 | 0.5018 |
150
- | 0.507 | 1.71 | 5000 | 0.5011 |
151
- | 0.5168 | 1.73 | 5050 | 0.5011 |
152
- | 0.498 | 1.74 | 5100 | 0.5012 |
153
- | 0.4781 | 1.76 | 5150 | 0.5012 |
154
- | 0.4656 | 1.78 | 5200 | 0.5005 |
155
- | 0.5745 | 1.8 | 5250 | 0.5006 |
156
- | 0.5095 | 1.81 | 5300 | 0.5006 |
157
- | 0.4743 | 1.83 | 5350 | 0.5005 |
158
- | 0.5316 | 1.85 | 5400 | 0.5004 |
159
- | 0.5014 | 1.86 | 5450 | 0.5003 |
160
- | 0.4273 | 1.88 | 5500 | 0.5002 |
161
- | 0.4237 | 1.9 | 5550 | 0.5003 |
162
- | 0.47 | 1.92 | 5600 | 0.5003 |
163
- | 0.4668 | 1.93 | 5650 | 0.5002 |
164
- | 0.523 | 1.95 | 5700 | 0.5002 |
165
- | 0.479 | 1.97 | 5750 | 0.5002 |
166
- | 0.4931 | 1.98 | 5800 | 0.5002 |
167
 
168
 
169
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.5488
19
 
20
  ## Model description
21
 
 
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 3e-05
38
+ - train_batch_size: 4
39
+ - eval_batch_size: 4
40
  - seed: 42
41
  - distributed_type: multi-GPU
42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
+ | 0.6891 | 0.34 | 500 | 0.6660 |
52
+ | 0.6423 | 0.68 | 1000 | 0.6075 |
53
+ | 0.5553 | 1.03 | 1500 | 0.5688 |
54
+ | 0.5175 | 1.37 | 2000 | 0.5533 |
55
+ | 0.5614 | 1.71 | 2500 | 0.5487 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
 
57
 
58
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:19f4392f13bcf5a2d862fbd339c734132b4007e3b0cfbbb9c5450b66374352b3
3
  size 218138576
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d46968635d204791c216cb5c4480f957d7213dec1bf01ea88434a883766bc8c
3
  size 218138576
all_results.json CHANGED
@@ -1,13 +1,13 @@
1
  {
2
  "epoch": 2.0,
3
- "eval_loss": 0.5001846551895142,
4
- "eval_runtime": 190.9675,
5
  "eval_samples": 650,
6
- "eval_samples_per_second": 3.404,
7
- "eval_steps_per_second": 1.702,
8
- "train_loss": 0.5801442559761337,
9
- "train_runtime": 31039.6997,
10
  "train_samples": 5845,
11
- "train_samples_per_second": 0.377,
12
- "train_steps_per_second": 0.188
13
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "eval_loss": 0.5487806797027588,
4
+ "eval_runtime": 112.8332,
5
  "eval_samples": 650,
6
+ "eval_samples_per_second": 5.761,
7
+ "eval_steps_per_second": 1.445,
8
+ "train_loss": 0.628653114618257,
9
+ "train_runtime": 6305.9173,
10
  "train_samples": 5845,
11
+ "train_samples_per_second": 1.854,
12
+ "train_steps_per_second": 0.464
13
  }
eval_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 2.0,
3
- "eval_loss": 0.5001846551895142,
4
- "eval_runtime": 190.9675,
5
  "eval_samples": 650,
6
- "eval_samples_per_second": 3.404,
7
- "eval_steps_per_second": 1.702
8
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "eval_loss": 0.5487806797027588,
4
+ "eval_runtime": 112.8332,
5
  "eval_samples": 650,
6
+ "eval_samples_per_second": 5.761,
7
+ "eval_steps_per_second": 1.445
8
  }
runs/Nov22_03-14-27_a72e59c0abac/events.out.tfevents.1700623038.a72e59c0abac.849.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:57ad87a2b57078a89de1edacd6dee4718de8ea50d11da4064cd7a5fa3ec42512
3
- size 45417
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48e3c11675e19959480c962a4dfbfa939c5afab8f5237874bfc25a07d81c3273
3
+ size 52365
runs/Nov22_03-14-27_a72e59c0abac/events.out.tfevents.1700629457.a72e59c0abac.849.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d98f214e168b0bfba95ddf772c8b7bde62f9838947eb7da239d041aee61fbff
3
+ size 359
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 2.0,
3
- "train_loss": 0.5801442559761337,
4
- "train_runtime": 31039.6997,
5
  "train_samples": 5845,
6
- "train_samples_per_second": 0.377,
7
- "train_steps_per_second": 0.188
8
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "train_loss": 0.628653114618257,
4
+ "train_runtime": 6305.9173,
5
  "train_samples": 5845,
6
+ "train_samples_per_second": 1.854,
7
+ "train_steps_per_second": 0.464
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff