Upload README.md

README.md

@@ -0,0 +1,83 @@
+---
+datasets:
+- TigerResearch/pretrain_zh
+base_model:
+- Qwen/Qwen2.5-14B
+tags:
+- character
+- generation
+license: apache-2.0
+---
+**Qwen2.5-14B-Character**
+
+**Introduction:**
+
+**Qwen2.5-14B-Character** is the Character version of [Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) model.  It is developed based on the [Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) model. It is specifically designed for character-to-character transformation and generation tasks. 
+
+**Core Contributions:**
+
+1. **Modified Token Vocabulary:** The original model's token vocabulary has been revised to remove tokens representing phrases and multiple characters. This refinement enhances the model's focus on individual character processing.
+
+2. **Continued Pre-training:** Based on the modified vocabulary, the model has undergone further pre-training to optimize its performance and adaptability for character-level tasks.
+   
+
+**Training Dataset:**
+
+The model has been trained using the `TigerResearch/pretrain_zh` dataset, a comprehensive Chinese pre-training dataset provided by **TigerResearch**. For more information about the dataset, please visit: [TigerResearch/pretrain_zh](https://huggingface.co/datasets/TigerResearch/pretrain_zh).
+
+
+**Training Code:**
+
+The training process for this model was facilitated by the **LLaMA-Factory**, an open-source project that provides tools and frameworks for training language models. The LLaMa-factory codebase is available at: [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory).
+
+
+**Results**
+
+To assess the efficacy of the Qwen2.5-14B-Character, we evaluated its performance on three widely utilized benchmarks: C-Evel, CMMLU, and MMLU. The results are tabulated as follows:
+
+| Model                             | ceval| cmmlu| mmlu| 
+| :---:                             | :---: | :---: | :---: | 
+| Qwen2.5-14B                       | 85.29| 85.84| 79.86| 
+| Qwen2.5-14B-filter 		        | 83.43| 83.72| 79.75| 
+| Qwen2.5-14B-Character     	    | 84.99| 84.60| 79.61| 
+
+In the table, to discern the model performance more distinctly, we have presented the test results for both the original Qwen2.5-14B (Qwen2.5-14B) and the token-modified Qwen2.5-14B (Qwen2.5-14B-filter).
+
+
+**Quickstart**
+
+The latest version of transformers is recommended (at least 4.37.0). Here we show a code snippet to show you how to use the chat model with transformers:
+
+```shell
+from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+
+model_name = 'Henry94/Qwen2.5-14B-Character'
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
+
+
+prompt = "请简单介绍一下大型语言模型."
+messages = [
+    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+
+print(response)
+```