Update GitHub repository link for broader project context

#20
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +112 -112
README.md CHANGED
@@ -1,112 +1,112 @@
1
- ---
2
- license: mit
3
- language:
4
- - en
5
- - zh
6
- base_model:
7
- - THUDM/GLM-4-9B-0414
8
- pipeline_tag: image-text-to-text
9
- library_name: transformers
10
- tags:
11
- - reasoning
12
- ---
13
-
14
- # GLM-4.1V-9B-Thinking
15
-
16
- <div align="center">
17
- <img src=https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/99c5eb6563236f0ff43605d91d107544da9863b2/resources/logo.svg width="40%"/>
18
- </div>
19
- <p align="center">
20
- 📖 View the GLM-4.1V-9B-Thinking <a href="https://arxiv.org/abs/2507.01006" target="_blank">paper</a>.
21
- <br>
22
- 💡 Try the <a href="https://huggingface.co/spaces/THUDM/GLM-4.1V-9B-Thinking-Demo" target="_blank">Hugging Face</a> or <a href="https://modelscope.cn/studios/ZhipuAI/GLM-4.1V-9B-Thinking-Demo" target="_blank">ModelScope</a> online demo for GLM-4.1V-9B-Thinking.
23
- <br>
24
- 📍 Using GLM-4.1V-9B-Thinking API at <a href="https://www.bigmodel.cn/dev/api/visual-reasoning-model/GLM-4.1V-Thinking">Zhipu Foundation Model Open Platform</a>
25
- </p>
26
-
27
-
28
- ## Model Introduction
29
-
30
- Vision-Language Models (VLMs) have become foundational components of intelligent systems. As real-world AI tasks grow
31
- increasingly complex, VLMs must evolve beyond basic multimodal perception to enhance their reasoning capabilities in
32
- complex tasks. This involves improving accuracy, comprehensiveness, and intelligence, enabling applications such as
33
- complex problem solving, long-context understanding, and multimodal agents.
34
-
35
- Based on the [GLM-4-9B-0414](https://github.com/THUDM/GLM-4) foundation model, we present the new open-source VLM model
36
- **GLM-4.1V-9B-Thinking**, designed to explore the upper limits of reasoning in vision-language models. By introducing
37
- a "thinking paradigm" and leveraging reinforcement learning, the model significantly enhances its capabilities. It
38
- achieves state-of-the-art performance among 10B-parameter VLMs, matching or even surpassing the 72B-parameter
39
- Qwen-2.5-VL-72B on 18 benchmark tasks. We are also open-sourcing the base model GLM-4.1V-9B-Base to
40
- support further research into the boundaries of VLM capabilities.
41
-
42
- ![rl](https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/refs/heads/main/resources/rl.jpeg)
43
-
44
- Compared to the previous generation models CogVLM2 and the GLM-4V series, **GLM-4.1V-Thinking** offers the
45
- following improvements:
46
-
47
- 1. The first reasoning-focused model in the series, achieving world-leading performance not only in mathematics but also
48
- across various sub-domains.
49
- 2. Supports **64k** context length.
50
- 3. Handles **arbitrary aspect ratios** and up to **4K** image resolution.
51
- 4. Provides an open-source version supporting both **Chinese and English bilingual** usage.
52
-
53
- ## Benchmark Performance
54
-
55
- By incorporating the Chain-of-Thought reasoning paradigm, GLM-4.1V-9B-Thinking significantly improves answer accuracy,
56
- richness, and interpretability. It comprehensively surpasses traditional non-reasoning visual models.
57
- Out of 28 benchmark tasks, it achieved the best performance among 10B-level models on 23 tasks,
58
- and even outperformed the 72B-parameter Qwen-2.5-VL-72B on 18 tasks.
59
-
60
- ![bench](https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/refs/heads/main/resources/bench.jpeg)
61
-
62
- ## Quick Inference
63
-
64
- This is a simple example of running single-image inference using the `transformers` library.
65
- First, install the `transformers` library from source:
66
-
67
- ```
68
- pip install git+https://github.com/huggingface/transformers.git
69
- ```
70
-
71
- Then, run the following code:
72
-
73
- ```python
74
- from transformers import AutoProcessor, Glm4vForConditionalGeneration
75
- import torch
76
-
77
- MODEL_PATH = "THUDM/GLM-4.1V-9B-Thinking"
78
- messages = [
79
- {
80
- "role": "user",
81
- "content": [
82
- {
83
- "type": "image",
84
- "url": "https://upload.wikimedia.org/wikipedia/commons/f/fa/Grayscale_8bits_palette_sample_image.png"
85
- },
86
- {
87
- "type": "text",
88
- "text": "describe this image"
89
- }
90
- ],
91
- }
92
- ]
93
- processor = AutoProcessor.from_pretrained(MODEL_PATH, use_fast=True)
94
- model = Glm4vForConditionalGeneration.from_pretrained(
95
- pretrained_model_name_or_path=MODEL_PATH,
96
- torch_dtype=torch.bfloat16,
97
- device_map="auto",
98
- )
99
- inputs = processor.apply_chat_template(
100
- messages,
101
- tokenize=True,
102
- add_generation_prompt=True,
103
- return_dict=True,
104
- return_tensors="pt"
105
- ).to(model.device)
106
- generated_ids = model.generate(**inputs, max_new_tokens=8192)
107
- output_text = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
108
- print(output_text)
109
- ```
110
-
111
- For video reasoning, web demo deployment, and more code, please check
112
- our [GitHub](https://github.com/THUDM/GLM-4.1V-Thinking).
 
1
+ ---
2
+ base_model:
3
+ - THUDM/GLM-4-9B-0414
4
+ language:
5
+ - en
6
+ - zh
7
+ library_name: transformers
8
+ license: mit
9
+ pipeline_tag: image-text-to-text
10
+ tags:
11
+ - reasoning
12
+ ---
13
+
14
+ # GLM-4.1V-9B-Thinking
15
+
16
+ <div align="center">
17
+ <img src=https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/99c5eb6563236f0ff43605d91d107544da9863b2/resources/logo.svg width="40%"/>
18
+ </div>
19
+ <p align="center">
20
+ 📖 View the GLM-4.1V-9B-Thinking <a href="https://arxiv.org/abs/2507.01006" target="_blank">paper</a>.
21
+ <br>
22
+ 💡 Try the <a href="https://huggingface.co/spaces/THUDM/GLM-4.1V-9B-Thinking-Demo" target="_blank">Hugging Face</a> or <a href="https://modelscope.cn/studios/ZhipuAI/GLM-4.1V-9B-Thinking-Demo" target="_blank">ModelScope</a> online demo for GLM-4.1V-9B-Thinking.
23
+ <br>
24
+ 📍 Using GLM-4.1V-9B-Thinking API at <a href="https://www.bigmodel.cn/dev/api/visual-reasoning-model/GLM-4.1V-Thinking">Zhipu Foundation Model Open Platform</a>
25
+ </p>
26
+
27
+
28
+ ## Model Introduction
29
+
30
+ Vision-Language Models (VLMs) have become foundational components of intelligent systems. As real-world AI tasks grow
31
+ increasingly complex, VLMs must evolve beyond basic multimodal perception to enhance their reasoning capabilities in
32
+ complex tasks. This involves improving accuracy, comprehensiveness, and intelligence, enabling applications such as
33
+ complex problem solving, long-context understanding, and multimodal agents.
34
+
35
+ Based on the [GLM-4-9B-0414](https://github.com/THUDM/GLM-4) foundation model, we present the new open-source VLM model
36
+ **GLM-4.1V-9B-Thinking**, designed to explore the upper limits of reasoning in vision-language models. By introducing
37
+ a "thinking paradigm" and leveraging reinforcement learning, the model significantly enhances its capabilities. It
38
+ achieves state-of-the-art performance among 10B-parameter VLMs, matching or even surpassing the 72B-parameter
39
+ Qwen-2.5-VL-72B on 18 benchmark tasks. We are also open-sourcing the base model GLM-4.1V-9B-Base to
40
+ support further research into the boundaries of VLM capabilities.
41
+
42
+ ![rl](https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/refs/heads/main/resources/rl.jpeg)
43
+
44
+ Compared to the previous generation models CogVLM2 and the GLM-4V series, **GLM-4.1V-Thinking** offers the
45
+ following improvements:
46
+
47
+ 1. The first reasoning-focused model in the series, achieving world-leading performance not only in mathematics but also
48
+ across various sub-domains.
49
+ 2. Supports **64k** context length.
50
+ 3. Handles **arbitrary aspect ratios** and up to **4K** image resolution.
51
+ 4. Provides an open-source version supporting both **Chinese and English bilingual** usage.
52
+
53
+ ## Benchmark Performance
54
+
55
+ By incorporating the Chain-of-Thought reasoning paradigm, GLM-4.1V-9B-Thinking significantly improves answer accuracy,
56
+ richness, and interpretability. It comprehensively surpasses traditional non-reasoning visual models.
57
+ Out of 28 benchmark tasks, it achieved the best performance among 10B-level models on 23 tasks,
58
+ and even outperformed the 72B-parameter Qwen-2.5-VL-72B on 18 tasks.
59
+
60
+ ![bench](https://raw.githubusercontent.com/THUDM/GLM-4.1V-Thinking/refs/heads/main/resources/bench.jpeg)
61
+
62
+ ## Quick Inference
63
+
64
+ This is a simple example of running single-image inference using the `transformers` library.
65
+ First, install the `transformers` library from source:
66
+
67
+ ```
68
+ pip install git+https://github.com/huggingface/transformers.git
69
+ ```
70
+
71
+ Then, run the following code:
72
+
73
+ ```python
74
+ from transformers import AutoProcessor, Glm4vForConditionalGeneration
75
+ import torch
76
+
77
+ MODEL_PATH = "THUDM/GLM-4.1V-9B-Thinking"
78
+ messages = [
79
+ {
80
+ "role": "user",
81
+ "content": [
82
+ {
83
+ "type": "image",
84
+ "url": "https://upload.wikimedia.org/wikipedia/commons/f/fa/Grayscale_8bits_palette_sample_image.png"
85
+ },
86
+ {
87
+ "type": "text",
88
+ "text": "describe this image"
89
+ }
90
+ ],
91
+ }
92
+ ]
93
+ processor = AutoProcessor.from_pretrained(MODEL_PATH, use_fast=True)
94
+ model = Glm4vForConditionalGeneration.from_pretrained(
95
+ pretrained_model_name_or_path=MODEL_PATH,
96
+ torch_dtype=torch.bfloat16,
97
+ device_map="auto",
98
+ )
99
+ inputs = processor.apply_chat_template(
100
+ messages,
101
+ tokenize=True,
102
+ add_generation_prompt=True,
103
+ return_dict=True,
104
+ return_tensors="pt"
105
+ ).to(model.device)
106
+ generated_ids = model.generate(**inputs, max_new_tokens=8192)
107
+ output_text = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
108
+ print(output_text)
109
+ ```
110
+
111
+ For video reasoning, web demo deployment, and more code, please check
112
+ our [GitHub](https://github.com/THUDM/GLM-V).