tonyzhao123 commited on
Commit
6ec8c1d
·
1 Parent(s): 286fbeb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -26
README.md CHANGED
@@ -12,13 +12,7 @@ pipeline_tag: text-generation
12
 
13
  # tonyzhao123/dummy_llama4
14
 
15
- Llama 4 for small size EP debug and dist
16
-
17
- ## Model Details
18
-
19
- - **Base Model**: meta-llama/Llama-3.2-1B
20
- - **Training Method**: Knowledge Distillation + Supervised Fine-tuning
21
- - **Dataset**: Custom KD Dataset (1000 samples)
22
 
23
  ## Usage
24
 
@@ -59,25 +53,6 @@ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
59
  print(response)
60
  ```
61
 
62
- ## Training Details
63
-
64
- - **Training Framework**: TRL (Transformers Reinforcement Learning)
65
- - **Optimizer**: AdamW
66
- - **Learning Rate**: 2e-5
67
- - **Batch Size**: 4
68
- - **Epochs**: 5
69
- - **Scheduler**: Cosine
70
- - **Precision**: FP16
71
-
72
- ## Performance
73
-
74
- This model has been fine-tuned using knowledge distillation techniques to maintain performance while potentially reducing model size or improving specific capabilities.
75
-
76
- ## Limitations
77
-
78
- - This is a fine-tuned model and may have inherited biases from the base model
79
- - Performance may vary on different types of tasks
80
- - Always evaluate the model on your specific use case
81
 
82
  ## Citation
83
 
 
12
 
13
  # tonyzhao123/dummy_llama4
14
 
15
+ Dummy Llama 4 for small size EP debug and dist
 
 
 
 
 
 
16
 
17
  ## Usage
18
 
 
53
  print(response)
54
  ```
55
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
 
57
  ## Citation
58