Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,110 @@
+---
+license: mit
+language:
+- en
+- zh
+base_model:
+- zai-org/GLM-4-9B-0414
+pipeline_tag: image-text-to-text
+library_name: transformers
+tags:
+- reasoning
+---
+
+# GLM-4.1V-9B-Thinking
+
+<div align="center">
+<img src=https://raw.githubusercontent.com/zai-org/GLM-4.1V-Thinking/99c5eb6563236f0ff43605d91d107544da9863b2/resources/logo.svg width="40%"/>
+</div>
+<p align="center">
+    📖 View the GLM-4.1V-9B-Thinking <a href="https://arxiv.org/abs/2507.01006" target="_blank">paper</a>.
+    <br>
+    📍 Using GLM-4.1V-9B-Thinking API at <a href="https://www.bigmodel.cn/dev/api/visual-reasoning-model/GLM-4.1V-Thinking">Zhipu Foundation Model Open Platform</a>
+</p>
+
+
+## Model Introduction
+
+Vision-Language Models (VLMs) have become foundational components of intelligent systems. As real-world AI tasks grow
+increasingly complex, VLMs must evolve beyond basic multimodal perception to enhance their reasoning capabilities in
+complex tasks. This involves improving accuracy, comprehensiveness, and intelligence, enabling applications such as
+complex problem solving, long-context understanding, and multimodal agents.
+
+Based on the [GLM-4-9B-0414](https://github.com/zai-org/GLM-4) foundation model, we present the new open-source VLM model
+**GLM-4.1V-9B-Thinking**, designed to explore the upper limits of reasoning in vision-language models. By introducing
+a "thinking paradigm" and leveraging reinforcement learning, the model significantly enhances its capabilities. It
+achieves state-of-the-art performance among 10B-parameter VLMs, matching or even surpassing the 72B-parameter
+Qwen-2.5-VL-72B on 18 benchmark tasks. We are also open-sourcing the base model GLM-4.1V-9B-Base to
+support further research into the boundaries of VLM capabilities.
+
+![rl](https://raw.githubusercontent.com/zai-org/GLM-4.1V-Thinking/refs/heads/main/resources/rl.jpeg)
+
+Compared to the previous generation models CogVLM2 and the GLM-4V series, **GLM-4.1V-Thinking** offers the
+following improvements:
+
+1. The first reasoning-focused model in the series, achieving world-leading performance not only in mathematics but also
+   across various sub-domains.
+2. Supports **64k** context length.
+3. Handles **arbitrary aspect ratios** and up to **4K** image resolution.
+4. Provides an open-source version supporting both **Chinese and English bilingual** usage.
+
+## Benchmark Performance
+
+By incorporating the Chain-of-Thought reasoning paradigm, GLM-4.1V-9B-Thinking significantly improves answer accuracy,
+richness, and interpretability. It comprehensively surpasses traditional non-reasoning visual models.
+Out of 28 benchmark tasks, it achieved the best performance among 10B-level models on 23 tasks,
+and even outperformed the 72B-parameter Qwen-2.5-VL-72B on 18 tasks.
+
+![bench](https://raw.githubusercontent.com/zai-org/GLM-4.1V-Thinking/refs/heads/main/resources/bench.jpeg)
+
+## Quick Inference
+
+This is a simple example of running single-image inference using the `transformers` library.  
+First, install the `transformers` library from source:
+
+```
+pip install transformers>=4.57.1
+```
+
+Then, run the following code:
+
+```python
+from transformers import AutoProcessor, Glm4vForConditionalGeneration
+import torch
+
+MODEL_PATH = "zai-org/GLM-4.1V-9B-Thinking"
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image",
+                "url": "https://upload.wikimedia.org/wikipedia/commons/f/fa/Grayscale_8bits_palette_sample_image.png"
+            },
+            {
+                "type": "text",
+                "text": "describe this image"
+            }
+        ],
+    }
+]
+processor = AutoProcessor.from_pretrained(MODEL_PATH, use_fast=True)
+model = Glm4vForConditionalGeneration.from_pretrained(
+    pretrained_model_name_or_path=MODEL_PATH,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_dict=True,
+    return_tensors="pt"
+).to(model.device)
+generated_ids = model.generate(**inputs, max_new_tokens=8192)
+output_text = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
+print(output_text)
+```
+
+For video reasoning, web demo deployment, and more code, please check
+our [GitHub](https://github.com/zai-org/GLM-V).

chat_template.jinja

@@ -0,0 +1,33 @@
+[gMASK]<sop>
+{%- for msg in messages %}
+    {%- if msg.role == 'system' %}
+<|system|>
+{{ msg.content }}
+    {%- elif msg.role == 'user' %}
+<|user|>{{ '\n' }}
+
+        {%- if msg.content is string %}
+{{ msg.content }}
+        {%- else %}
+            {%- for item in msg.content %}
+                {%- if item.type == 'video' or 'video' in item %}
+<|begin_of_video|><|video|><|end_of_video|>
+                {%- elif item.type == 'image' or 'image' in item %}
+<|begin_of_image|><|image|><|end_of_image|>
+                {%- elif item.type == 'text' %}
+{{ item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- elif msg.role == 'assistant' %}
+        {%- if msg.metadata %}
+<|assistant|>{{ msg.metadata }}
+{{ msg.content }}
+        {%- else %}
+<|assistant|>
+{{ msg.content }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{% if add_generation_prompt %}<|assistant|>
+{% endif %}

config.json

@@ -0,0 +1,77 @@
+{
+    "architectures": [
+        "Glm4vForConditionalGeneration"
+    ],
+    "model_type": "glm4v",
+    "image_start_token_id": 151339,
+    "image_end_token_id": 151340,
+    "video_start_token_id": 151341,
+    "video_end_token_id": 151342,
+    "image_token_id": 151343,
+    "video_token_id": 151344,
+    "tie_word_embeddings": false,
+    "transformers_version": "4.57.1",
+    "text_config": {
+        "model_type": "glm4v_text",
+        "attention_bias": true,
+        "attention_dropout": 0.0,
+        "pad_token_id": 151329,
+        "eos_token_id": [
+            151329,
+            151336,
+            151338,
+            151348
+        ],
+        "hidden_act": "silu",
+        "hidden_size": 4096,
+        "initializer_range": 0.02,
+        "intermediate_size": 13696,
+        "max_position_embeddings": 65536,
+        "num_attention_heads": 32,
+        "num_hidden_layers": 40,
+        "num_key_value_heads": 2,
+        "rms_norm_eps": 1e-05,
+        "dtype": "bfloat16",
+        "use_cache": true,
+        "vocab_size": 151552,
+        "partial_rotary_factor": 0.5,
+        "rope_theta": 10000,
+        "rope_scaling": {
+            "rope_type": "default",
+            "mrope_section": [
+                8,
+                12,
+                12
+            ]
+        }
+    },
+    "vision_config": {
+        "model_type": "glm4v",
+        "hidden_size": 1536,
+        "depth": 24,
+        "num_heads": 12,
+        "attention_bias": false,
+        "intermediate_size": 13696,
+        "hidden_act": "silu",
+        "hidden_dropout_prob": 0.0,
+        "initializer_range": 0.02,
+        "image_size": 336,
+        "patch_size": 14,
+        "out_hidden_size": 4096,
+        "rms_norm_eps": 1e-05,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "quantization_config": {
+        "quant_method": "exl3",
+        "version": "0.0.14",
+        "bits": 2.0,
+        "head_bits": 6,
+        "calibration": {
+            "rows": 250,
+            "cols": 2048
+        },
+        "out_scales": "auto",
+        "codebook": "mcg"
+    }
+}

generation_config.json

@@ -0,0 +1,15 @@
+{
+  "_from_model_config": true,
+  "do_sample": true,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338,
+    151348
+  ],
+  "pad_token_id": 151329,
+  "top_p": 0.6,
+  "temperature": 0.8,
+  "top_k": 2,
+  "transformers_version": "4.57.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49a37c517868c5f40488551a72296c1ab41b84cf1eee92bfdb069c5b4898dc7a
+size 5540027748

preprocessor_config.json

@@ -0,0 +1,11 @@
+{
+    "size": {"shortest_edge": 12544, "longest_edge": 9633792},
+    "do_rescale": true,
+    "patch_size": 14,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "image_processor_type": "Glm4vImageProcessor",
+    "processor_class": "Glm4vProcessor"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76ebeac0d8bd7879ead7b43c16b44981f277e47225de2bd7de9ae1a6cc664a8c
+size 19966496

tokenizer_config.json

@@ -0,0 +1,218 @@
+{
+  "added_tokens_decoder": {
+    "151329": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151330": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151331": {
+      "content": "[gMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151332": {
+      "content": "[sMASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151333": {
+      "content": "<sop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151334": {
+      "content": "<eop>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151335": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151336": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151337": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151338": {
+      "content": "<|observation|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151339": {
+      "content": "<|begin_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151340": {
+      "content": "<|end_of_image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151341": {
+      "content": "<|begin_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151342": {
+      "content": "<|end_of_video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+     "151343": {
+      "content": "<|image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151344": {
+      "content": "<|video|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+     "151345": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+     "151346": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+     "151347": {
+      "content": "<answer>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+     "151348": {
+      "content": "</answer>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151349": {
+      "content": "<|begin_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151350": {
+      "content": "<|end_of_box|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151351": {
+      "content": "<|sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|image|>",
+    "<|video|>"
+  ],
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 65536,
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "PreTrainedTokenizer"
+}

video_preprocessor_config.json

@@ -0,0 +1,11 @@
+{
+    "size": {"shortest_edge": 12544, "longest_edge": 47040000},
+    "do_rescale": true,
+    "patch_size": 14,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "video_processor_type": "Glm4vVideoProcessor",
+    "processor_class": "Glm4vProcessor"
+}