Hugging Face's logo

neurocoder
/
logsQwen2.5-0.5B-Instruct-math-gsm8k

Text Generation
Transformers
TensorBoard
Safetensors
qwen2
Generated from Trainer
trl
sft
conversational
text-generation-inference
Model card Files
xet
Metrics Community
1
neurocoder commited on
Commit
96aac03
·
verified ·
1 Parent(s): 8026a48

Model save

Browse files
Files changed (8) hide show
  1. README.md +58 -0
  2. all_results.json +9 -0
  3. config.json +1 -1
  4. generation_config.json +14 -0
  5. model.safetensors +1 -1
  6. runs/Mar11_17-20-56_h100-m-07/events.out.tfevents.1741674065.h100-m-07.1151483.0 +2 -2
  7. train_results.json +9 -0
  8. trainer_state.json +756 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-0.5B-Instruct
3
+ library_name: transformers
4
+ model_name: logsQwen2.5-0.5B-Instruct-math-gsm8k
5
+ tags:
6
+ - generated_from_trainer
7
+ - trl
8
+ - sft
9
+ licence: license
10
+ ---
11
+
12
+ # Model Card for logsQwen2.5-0.5B-Instruct-math-gsm8k
13
+
14
+ This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl).
16
+
17
+ ## Quick start
18
+
19
+ ```python
20
+ from transformers import pipeline
21
+
22
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="neurocoder/logsQwen2.5-0.5B-Instruct-math-gsm8k", device="cuda")
24
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
+ print(output["generated_text"])
26
+ ```
27
+
28
+ ## Training procedure
29
+
30
+
31
+
32
+
33
+ This model was trained with SFT.
34
+
35
+ ### Framework versions
36
+
37
+ - TRL: 0.14.0
38
+ - Transformers: 4.48.2
39
+ - Pytorch: 2.5.1
40
+ - Datasets: 3.2.0
41
+ - Tokenizers: 0.21.0
42
+
43
+ ## Citations
44
+
45
+
46
+
47
+ Cite TRL as:
48
+
49
+ ```bibtex
50
+ @misc{vonwerra2022trl,
51
+ title = {{TRL: Transformer Reinforcement Learning}},
52
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
53
+ year = 2020,
54
+ journal = {GitHub repository},
55
+ publisher = {GitHub},
56
+ howpublished = {\url{https://github.com/huggingface/trl}}
57
+ }
58
+ ```
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.9995119570522206,
3
+ "total_flos": 3518281600204800.0,
4
+ "train_loss": 1.3758189086802304,
5
+ "train_runtime": 549.4956,
6
+ "train_samples": 7473,
7
+ "train_samples_per_second": 14.915,
8
+ "train_steps_per_second": 0.932
9
+ }
config.json CHANGED
@@ -23,7 +23,7 @@
23
  "tie_word_embeddings": true,
24
  "torch_dtype": "bfloat16",
25
  "transformers_version": "4.48.2",
26
- "use_cache": false,
27
  "use_sliding_window": false,
28
  "vocab_size": 151936
29
  }
 
23
  "tie_word_embeddings": true,
24
  "torch_dtype": "bfloat16",
25
  "transformers_version": "4.48.2",
26
+ "use_cache": true,
27
  "use_sliding_window": false,
28
  "vocab_size": 151936
29
  }
generation_config.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151643,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 151645,
6
+ 151643
7
+ ],
8
+ "pad_token_id": 151643,
9
+ "repetition_penalty": 1.1,
10
+ "temperature": 0.7,
11
+ "top_k": 20,
12
+ "top_p": 0.8,
13
+ "transformers_version": "4.48.2"
14
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:441f88c7cb29db407e074ac4cf0d227a254396cb3c5365a1064679d4984902b7
3
  size 988097824
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:266a2dc7d9bc80e446f38e6d728f239890773ce51e34e46c8ca42f05348f3401
3
  size 988097824
runs/Mar11_17-20-56_h100-m-07/events.out.tfevents.1741674065.h100-m-07.1151483.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2bc4c472859d4dc42c248242abdd5c02ea8076f4d8a5eea63d8039a69c52760
3
- size 26545
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff5a893d14961013a6ede0adc3e69aae937f25ffbabeaba1a3dfe53267e64545
3
+ size 27321
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.9995119570522206,
3
+ "total_flos": 3518281600204800.0,
4
+ "train_loss": 1.3758189086802304,
5
+ "train_runtime": 549.4956,
6
+ "train_samples": 7473,
7
+ "train_samples_per_second": 14.915,
8
+ "train_steps_per_second": 0.932
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,756 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.9995119570522206,
5
+ "eval_steps": 500,
6
+ "global_step": 512,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.009760858955588092,
13
+ "grad_norm": 23.375,
14
+ "learning_rate": 0.0002,
15
+ "loss": 2.8524,
16
+ "step": 5
17
+ },
18
+ {
19
+ "epoch": 0.019521717911176184,
20
+ "grad_norm": 12.9375,
21
+ "learning_rate": 0.0002,
22
+ "loss": 1.7471,
23
+ "step": 10
24
+ },
25
+ {
26
+ "epoch": 0.029282576866764276,
27
+ "grad_norm": 5.34375,
28
+ "learning_rate": 0.0002,
29
+ "loss": 1.4572,
30
+ "step": 15
31
+ },
32
+ {
33
+ "epoch": 0.03904343582235237,
34
+ "grad_norm": 16.625,
35
+ "learning_rate": 0.0002,
36
+ "loss": 1.3997,
37
+ "step": 20
38
+ },
39
+ {
40
+ "epoch": 0.04880429477794046,
41
+ "grad_norm": 6.875,
42
+ "learning_rate": 0.0002,
43
+ "loss": 1.5013,
44
+ "step": 25
45
+ },
46
+ {
47
+ "epoch": 0.05856515373352855,
48
+ "grad_norm": 5.25,
49
+ "learning_rate": 0.0002,
50
+ "loss": 1.4325,
51
+ "step": 30
52
+ },
53
+ {
54
+ "epoch": 0.06832601268911664,
55
+ "grad_norm": 4.5,
56
+ "learning_rate": 0.0002,
57
+ "loss": 1.3938,
58
+ "step": 35
59
+ },
60
+ {
61
+ "epoch": 0.07808687164470474,
62
+ "grad_norm": 4.53125,
63
+ "learning_rate": 0.0002,
64
+ "loss": 1.4496,
65
+ "step": 40
66
+ },
67
+ {
68
+ "epoch": 0.08784773060029283,
69
+ "grad_norm": 4.90625,
70
+ "learning_rate": 0.0002,
71
+ "loss": 1.4919,
72
+ "step": 45
73
+ },
74
+ {
75
+ "epoch": 0.09760858955588092,
76
+ "grad_norm": 3.96875,
77
+ "learning_rate": 0.0002,
78
+ "loss": 1.4757,
79
+ "step": 50
80
+ },
81
+ {
82
+ "epoch": 0.10736944851146901,
83
+ "grad_norm": 5.8125,
84
+ "learning_rate": 0.0002,
85
+ "loss": 1.5236,
86
+ "step": 55
87
+ },
88
+ {
89
+ "epoch": 0.1171303074670571,
90
+ "grad_norm": 3.71875,
91
+ "learning_rate": 0.0002,
92
+ "loss": 1.464,
93
+ "step": 60
94
+ },
95
+ {
96
+ "epoch": 0.1268911664226452,
97
+ "grad_norm": 5.0,
98
+ "learning_rate": 0.0002,
99
+ "loss": 1.4982,
100
+ "step": 65
101
+ },
102
+ {
103
+ "epoch": 0.1366520253782333,
104
+ "grad_norm": 4.15625,
105
+ "learning_rate": 0.0002,
106
+ "loss": 1.3583,
107
+ "step": 70
108
+ },
109
+ {
110
+ "epoch": 0.14641288433382138,
111
+ "grad_norm": 3.203125,
112
+ "learning_rate": 0.0002,
113
+ "loss": 1.3943,
114
+ "step": 75
115
+ },
116
+ {
117
+ "epoch": 0.15617374328940947,
118
+ "grad_norm": 3.796875,
119
+ "learning_rate": 0.0002,
120
+ "loss": 1.4915,
121
+ "step": 80
122
+ },
123
+ {
124
+ "epoch": 0.16593460224499756,
125
+ "grad_norm": 4.15625,
126
+ "learning_rate": 0.0002,
127
+ "loss": 1.5084,
128
+ "step": 85
129
+ },
130
+ {
131
+ "epoch": 0.17569546120058566,
132
+ "grad_norm": 3.1875,
133
+ "learning_rate": 0.0002,
134
+ "loss": 1.4788,
135
+ "step": 90
136
+ },
137
+ {
138
+ "epoch": 0.18545632015617375,
139
+ "grad_norm": 7.28125,
140
+ "learning_rate": 0.0002,
141
+ "loss": 1.4738,
142
+ "step": 95
143
+ },
144
+ {
145
+ "epoch": 0.19521717911176184,
146
+ "grad_norm": 3.734375,
147
+ "learning_rate": 0.0002,
148
+ "loss": 1.4498,
149
+ "step": 100
150
+ },
151
+ {
152
+ "epoch": 0.20497803806734993,
153
+ "grad_norm": 7.15625,
154
+ "learning_rate": 0.0002,
155
+ "loss": 1.5028,
156
+ "step": 105
157
+ },
158
+ {
159
+ "epoch": 0.21473889702293802,
160
+ "grad_norm": 7.3125,
161
+ "learning_rate": 0.0002,
162
+ "loss": 1.4605,
163
+ "step": 110
164
+ },
165
+ {
166
+ "epoch": 0.22449975597852612,
167
+ "grad_norm": 3.53125,
168
+ "learning_rate": 0.0002,
169
+ "loss": 1.4047,
170
+ "step": 115
171
+ },
172
+ {
173
+ "epoch": 0.2342606149341142,
174
+ "grad_norm": 3.640625,
175
+ "learning_rate": 0.0002,
176
+ "loss": 1.4216,
177
+ "step": 120
178
+ },
179
+ {
180
+ "epoch": 0.2440214738897023,
181
+ "grad_norm": 3.5,
182
+ "learning_rate": 0.0002,
183
+ "loss": 1.4401,
184
+ "step": 125
185
+ },
186
+ {
187
+ "epoch": 0.2537823328452904,
188
+ "grad_norm": 2.78125,
189
+ "learning_rate": 0.0002,
190
+ "loss": 1.4301,
191
+ "step": 130
192
+ },
193
+ {
194
+ "epoch": 0.2635431918008785,
195
+ "grad_norm": 2.96875,
196
+ "learning_rate": 0.0002,
197
+ "loss": 1.4325,
198
+ "step": 135
199
+ },
200
+ {
201
+ "epoch": 0.2733040507564666,
202
+ "grad_norm": 2.8125,
203
+ "learning_rate": 0.0002,
204
+ "loss": 1.3988,
205
+ "step": 140
206
+ },
207
+ {
208
+ "epoch": 0.28306490971205467,
209
+ "grad_norm": 3.234375,
210
+ "learning_rate": 0.0002,
211
+ "loss": 1.3768,
212
+ "step": 145
213
+ },
214
+ {
215
+ "epoch": 0.29282576866764276,
216
+ "grad_norm": 5.0,
217
+ "learning_rate": 0.0002,
218
+ "loss": 1.4328,
219
+ "step": 150
220
+ },
221
+ {
222
+ "epoch": 0.30258662762323085,
223
+ "grad_norm": 4.0625,
224
+ "learning_rate": 0.0002,
225
+ "loss": 1.4189,
226
+ "step": 155
227
+ },
228
+ {
229
+ "epoch": 0.31234748657881894,
230
+ "grad_norm": 2.78125,
231
+ "learning_rate": 0.0002,
232
+ "loss": 1.3605,
233
+ "step": 160
234
+ },
235
+ {
236
+ "epoch": 0.32210834553440704,
237
+ "grad_norm": 3.40625,
238
+ "learning_rate": 0.0002,
239
+ "loss": 1.4092,
240
+ "step": 165
241
+ },
242
+ {
243
+ "epoch": 0.33186920448999513,
244
+ "grad_norm": 3.421875,
245
+ "learning_rate": 0.0002,
246
+ "loss": 1.3854,
247
+ "step": 170
248
+ },
249
+ {
250
+ "epoch": 0.3416300634455832,
251
+ "grad_norm": 3.375,
252
+ "learning_rate": 0.0002,
253
+ "loss": 1.3923,
254
+ "step": 175
255
+ },
256
+ {
257
+ "epoch": 0.3513909224011713,
258
+ "grad_norm": 2.609375,
259
+ "learning_rate": 0.0002,
260
+ "loss": 1.3395,
261
+ "step": 180
262
+ },
263
+ {
264
+ "epoch": 0.3611517813567594,
265
+ "grad_norm": 2.671875,
266
+ "learning_rate": 0.0002,
267
+ "loss": 1.3532,
268
+ "step": 185
269
+ },
270
+ {
271
+ "epoch": 0.3709126403123475,
272
+ "grad_norm": 2.765625,
273
+ "learning_rate": 0.0002,
274
+ "loss": 1.3764,
275
+ "step": 190
276
+ },
277
+ {
278
+ "epoch": 0.3806734992679356,
279
+ "grad_norm": 3.109375,
280
+ "learning_rate": 0.0002,
281
+ "loss": 1.3179,
282
+ "step": 195
283
+ },
284
+ {
285
+ "epoch": 0.3904343582235237,
286
+ "grad_norm": 2.71875,
287
+ "learning_rate": 0.0002,
288
+ "loss": 1.309,
289
+ "step": 200
290
+ },
291
+ {
292
+ "epoch": 0.4001952171791118,
293
+ "grad_norm": 2.9375,
294
+ "learning_rate": 0.0002,
295
+ "loss": 1.4215,
296
+ "step": 205
297
+ },
298
+ {
299
+ "epoch": 0.40995607613469986,
300
+ "grad_norm": 2.71875,
301
+ "learning_rate": 0.0002,
302
+ "loss": 1.4362,
303
+ "step": 210
304
+ },
305
+ {
306
+ "epoch": 0.41971693509028796,
307
+ "grad_norm": 2.875,
308
+ "learning_rate": 0.0002,
309
+ "loss": 1.3835,
310
+ "step": 215
311
+ },
312
+ {
313
+ "epoch": 0.42947779404587605,
314
+ "grad_norm": 2.46875,
315
+ "learning_rate": 0.0002,
316
+ "loss": 1.3099,
317
+ "step": 220
318
+ },
319
+ {
320
+ "epoch": 0.43923865300146414,
321
+ "grad_norm": 2.609375,
322
+ "learning_rate": 0.0002,
323
+ "loss": 1.3739,
324
+ "step": 225
325
+ },
326
+ {
327
+ "epoch": 0.44899951195705223,
328
+ "grad_norm": 2.890625,
329
+ "learning_rate": 0.0002,
330
+ "loss": 1.3374,
331
+ "step": 230
332
+ },
333
+ {
334
+ "epoch": 0.4587603709126403,
335
+ "grad_norm": 2.453125,
336
+ "learning_rate": 0.0002,
337
+ "loss": 1.3693,
338
+ "step": 235
339
+ },
340
+ {
341
+ "epoch": 0.4685212298682284,
342
+ "grad_norm": 2.625,
343
+ "learning_rate": 0.0002,
344
+ "loss": 1.3478,
345
+ "step": 240
346
+ },
347
+ {
348
+ "epoch": 0.4782820888238165,
349
+ "grad_norm": 2.78125,
350
+ "learning_rate": 0.0002,
351
+ "loss": 1.3685,
352
+ "step": 245
353
+ },
354
+ {
355
+ "epoch": 0.4880429477794046,
356
+ "grad_norm": 2.53125,
357
+ "learning_rate": 0.0002,
358
+ "loss": 1.2963,
359
+ "step": 250
360
+ },
361
+ {
362
+ "epoch": 0.4978038067349927,
363
+ "grad_norm": 2.359375,
364
+ "learning_rate": 0.0002,
365
+ "loss": 1.3452,
366
+ "step": 255
367
+ },
368
+ {
369
+ "epoch": 0.5075646656905808,
370
+ "grad_norm": 2.515625,
371
+ "learning_rate": 0.0002,
372
+ "loss": 1.3321,
373
+ "step": 260
374
+ },
375
+ {
376
+ "epoch": 0.5173255246461689,
377
+ "grad_norm": 2.65625,
378
+ "learning_rate": 0.0002,
379
+ "loss": 1.3702,
380
+ "step": 265
381
+ },
382
+ {
383
+ "epoch": 0.527086383601757,
384
+ "grad_norm": 3.296875,
385
+ "learning_rate": 0.0002,
386
+ "loss": 1.4063,
387
+ "step": 270
388
+ },
389
+ {
390
+ "epoch": 0.5368472425573451,
391
+ "grad_norm": 2.59375,
392
+ "learning_rate": 0.0002,
393
+ "loss": 1.2899,
394
+ "step": 275
395
+ },
396
+ {
397
+ "epoch": 0.5466081015129332,
398
+ "grad_norm": 2.453125,
399
+ "learning_rate": 0.0002,
400
+ "loss": 1.309,
401
+ "step": 280
402
+ },
403
+ {
404
+ "epoch": 0.5563689604685212,
405
+ "grad_norm": 2.5,
406
+ "learning_rate": 0.0002,
407
+ "loss": 1.3354,
408
+ "step": 285
409
+ },
410
+ {
411
+ "epoch": 0.5661298194241093,
412
+ "grad_norm": 2.578125,
413
+ "learning_rate": 0.0002,
414
+ "loss": 1.3682,
415
+ "step": 290
416
+ },
417
+ {
418
+ "epoch": 0.5758906783796974,
419
+ "grad_norm": 2.40625,
420
+ "learning_rate": 0.0002,
421
+ "loss": 1.3351,
422
+ "step": 295
423
+ },
424
+ {
425
+ "epoch": 0.5856515373352855,
426
+ "grad_norm": 2.65625,
427
+ "learning_rate": 0.0002,
428
+ "loss": 1.3483,
429
+ "step": 300
430
+ },
431
+ {
432
+ "epoch": 0.5954123962908736,
433
+ "grad_norm": 2.421875,
434
+ "learning_rate": 0.0002,
435
+ "loss": 1.292,
436
+ "step": 305
437
+ },
438
+ {
439
+ "epoch": 0.6051732552464617,
440
+ "grad_norm": 2.53125,
441
+ "learning_rate": 0.0002,
442
+ "loss": 1.3021,
443
+ "step": 310
444
+ },
445
+ {
446
+ "epoch": 0.6149341142020498,
447
+ "grad_norm": 2.453125,
448
+ "learning_rate": 0.0002,
449
+ "loss": 1.3805,
450
+ "step": 315
451
+ },
452
+ {
453
+ "epoch": 0.6246949731576379,
454
+ "grad_norm": 2.71875,
455
+ "learning_rate": 0.0002,
456
+ "loss": 1.3212,
457
+ "step": 320
458
+ },
459
+ {
460
+ "epoch": 0.634455832113226,
461
+ "grad_norm": 2.40625,
462
+ "learning_rate": 0.0002,
463
+ "loss": 1.2793,
464
+ "step": 325
465
+ },
466
+ {
467
+ "epoch": 0.6442166910688141,
468
+ "grad_norm": 2.53125,
469
+ "learning_rate": 0.0002,
470
+ "loss": 1.3733,
471
+ "step": 330
472
+ },
473
+ {
474
+ "epoch": 0.6539775500244022,
475
+ "grad_norm": 2.53125,
476
+ "learning_rate": 0.0002,
477
+ "loss": 1.2849,
478
+ "step": 335
479
+ },
480
+ {
481
+ "epoch": 0.6637384089799903,
482
+ "grad_norm": 2.59375,
483
+ "learning_rate": 0.0002,
484
+ "loss": 1.3962,
485
+ "step": 340
486
+ },
487
+ {
488
+ "epoch": 0.6734992679355783,
489
+ "grad_norm": 2.796875,
490
+ "learning_rate": 0.0002,
491
+ "loss": 1.3527,
492
+ "step": 345
493
+ },
494
+ {
495
+ "epoch": 0.6832601268911664,
496
+ "grad_norm": 2.828125,
497
+ "learning_rate": 0.0002,
498
+ "loss": 1.3286,
499
+ "step": 350
500
+ },
501
+ {
502
+ "epoch": 0.6930209858467545,
503
+ "grad_norm": 2.671875,
504
+ "learning_rate": 0.0002,
505
+ "loss": 1.3914,
506
+ "step": 355
507
+ },
508
+ {
509
+ "epoch": 0.7027818448023426,
510
+ "grad_norm": 5.125,
511
+ "learning_rate": 0.0002,
512
+ "loss": 1.3359,
513
+ "step": 360
514
+ },
515
+ {
516
+ "epoch": 0.7125427037579307,
517
+ "grad_norm": 2.609375,
518
+ "learning_rate": 0.0002,
519
+ "loss": 1.3115,
520
+ "step": 365
521
+ },
522
+ {
523
+ "epoch": 0.7223035627135188,
524
+ "grad_norm": 2.109375,
525
+ "learning_rate": 0.0002,
526
+ "loss": 1.3279,
527
+ "step": 370
528
+ },
529
+ {
530
+ "epoch": 0.7320644216691069,
531
+ "grad_norm": 7.15625,
532
+ "learning_rate": 0.0002,
533
+ "loss": 1.2885,
534
+ "step": 375
535
+ },
536
+ {
537
+ "epoch": 0.741825280624695,
538
+ "grad_norm": 3.390625,
539
+ "learning_rate": 0.0002,
540
+ "loss": 1.322,
541
+ "step": 380
542
+ },
543
+ {
544
+ "epoch": 0.7515861395802831,
545
+ "grad_norm": 3.109375,
546
+ "learning_rate": 0.0002,
547
+ "loss": 1.3047,
548
+ "step": 385
549
+ },
550
+ {
551
+ "epoch": 0.7613469985358712,
552
+ "grad_norm": 2.65625,
553
+ "learning_rate": 0.0002,
554
+ "loss": 1.2534,
555
+ "step": 390
556
+ },
557
+ {
558
+ "epoch": 0.7711078574914593,
559
+ "grad_norm": 2.734375,
560
+ "learning_rate": 0.0002,
561
+ "loss": 1.2539,
562
+ "step": 395
563
+ },
564
+ {
565
+ "epoch": 0.7808687164470474,
566
+ "grad_norm": 2.71875,
567
+ "learning_rate": 0.0002,
568
+ "loss": 1.277,
569
+ "step": 400
570
+ },
571
+ {
572
+ "epoch": 0.7906295754026355,
573
+ "grad_norm": 2.8125,
574
+ "learning_rate": 0.0002,
575
+ "loss": 1.2932,
576
+ "step": 405
577
+ },
578
+ {
579
+ "epoch": 0.8003904343582235,
580
+ "grad_norm": 2.296875,
581
+ "learning_rate": 0.0002,
582
+ "loss": 1.2782,
583
+ "step": 410
584
+ },
585
+ {
586
+ "epoch": 0.8101512933138116,
587
+ "grad_norm": 2.359375,
588
+ "learning_rate": 0.0002,
589
+ "loss": 1.294,
590
+ "step": 415
591
+ },
592
+ {
593
+ "epoch": 0.8199121522693997,
594
+ "grad_norm": 2.296875,
595
+ "learning_rate": 0.0002,
596
+ "loss": 1.2311,
597
+ "step": 420
598
+ },
599
+ {
600
+ "epoch": 0.8296730112249878,
601
+ "grad_norm": 2.28125,
602
+ "learning_rate": 0.0002,
603
+ "loss": 1.3014,
604
+ "step": 425
605
+ },
606
+ {
607
+ "epoch": 0.8394338701805759,
608
+ "grad_norm": 2.859375,
609
+ "learning_rate": 0.0002,
610
+ "loss": 1.325,
611
+ "step": 430
612
+ },
613
+ {
614
+ "epoch": 0.849194729136164,
615
+ "grad_norm": 2.125,
616
+ "learning_rate": 0.0002,
617
+ "loss": 1.187,
618
+ "step": 435
619
+ },
620
+ {
621
+ "epoch": 0.8589555880917521,
622
+ "grad_norm": 2.359375,
623
+ "learning_rate": 0.0002,
624
+ "loss": 1.2626,
625
+ "step": 440
626
+ },
627
+ {
628
+ "epoch": 0.8687164470473402,
629
+ "grad_norm": 2.3125,
630
+ "learning_rate": 0.0002,
631
+ "loss": 1.1967,
632
+ "step": 445
633
+ },
634
+ {
635
+ "epoch": 0.8784773060029283,
636
+ "grad_norm": 2.125,
637
+ "learning_rate": 0.0002,
638
+ "loss": 1.3065,
639
+ "step": 450
640
+ },
641
+ {
642
+ "epoch": 0.8882381649585164,
643
+ "grad_norm": 2.421875,
644
+ "learning_rate": 0.0002,
645
+ "loss": 1.2892,
646
+ "step": 455
647
+ },
648
+ {
649
+ "epoch": 0.8979990239141045,
650
+ "grad_norm": 2.3125,
651
+ "learning_rate": 0.0002,
652
+ "loss": 1.2817,
653
+ "step": 460
654
+ },
655
+ {
656
+ "epoch": 0.9077598828696926,
657
+ "grad_norm": 2.109375,
658
+ "learning_rate": 0.0002,
659
+ "loss": 1.2344,
660
+ "step": 465
661
+ },
662
+ {
663
+ "epoch": 0.9175207418252807,
664
+ "grad_norm": 2.359375,
665
+ "learning_rate": 0.0002,
666
+ "loss": 1.349,
667
+ "step": 470
668
+ },
669
+ {
670
+ "epoch": 0.9272816007808687,
671
+ "grad_norm": 2.21875,
672
+ "learning_rate": 0.0002,
673
+ "loss": 1.255,
674
+ "step": 475
675
+ },
676
+ {
677
+ "epoch": 0.9370424597364568,
678
+ "grad_norm": 2.03125,
679
+ "learning_rate": 0.0002,
680
+ "loss": 1.2741,
681
+ "step": 480
682
+ },
683
+ {
684
+ "epoch": 0.9468033186920449,
685
+ "grad_norm": 2.328125,
686
+ "learning_rate": 0.0002,
687
+ "loss": 1.3024,
688
+ "step": 485
689
+ },
690
+ {
691
+ "epoch": 0.956564177647633,
692
+ "grad_norm": 2.34375,
693
+ "learning_rate": 0.0002,
694
+ "loss": 1.297,
695
+ "step": 490
696
+ },
697
+ {
698
+ "epoch": 0.9663250366032211,
699
+ "grad_norm": 2.125,
700
+ "learning_rate": 0.0002,
701
+ "loss": 1.2095,
702
+ "step": 495
703
+ },
704
+ {
705
+ "epoch": 0.9760858955588092,
706
+ "grad_norm": 2.1875,
707
+ "learning_rate": 0.0002,
708
+ "loss": 1.3445,
709
+ "step": 500
710
+ },
711
+ {
712
+ "epoch": 0.9858467545143973,
713
+ "grad_norm": 2.78125,
714
+ "learning_rate": 0.0002,
715
+ "loss": 1.349,
716
+ "step": 505
717
+ },
718
+ {
719
+ "epoch": 0.9956076134699854,
720
+ "grad_norm": 2.296875,
721
+ "learning_rate": 0.0002,
722
+ "loss": 1.2542,
723
+ "step": 510
724
+ },
725
+ {
726
+ "epoch": 0.9995119570522206,
727
+ "step": 512,
728
+ "total_flos": 3518281600204800.0,
729
+ "train_loss": 1.3758189086802304,
730
+ "train_runtime": 549.4956,
731
+ "train_samples_per_second": 14.915,
732
+ "train_steps_per_second": 0.932
733
+ }
734
+ ],
735
+ "logging_steps": 5,
736
+ "max_steps": 512,
737
+ "num_input_tokens_seen": 0,
738
+ "num_train_epochs": 1,
739
+ "save_steps": 100,
740
+ "stateful_callbacks": {
741
+ "TrainerControl": {
742
+ "args": {
743
+ "should_epoch_stop": false,
744
+ "should_evaluate": false,
745
+ "should_log": false,
746
+ "should_save": true,
747
+ "should_training_stop": true
748
+ },
749
+ "attributes": {}
750
+ }
751
+ },
752
+ "total_flos": 3518281600204800.0,
753
+ "train_batch_size": 2,
754
+ "trial_name": null,
755
+ "trial_params": null
756
+ }