vLLM: 'Replace the chat_template string in your tokenizer_config.json with the file contents.' ... where?
In the model card, it says the following for vLLM:
Replace the
chat_templatestring in yourtokenizer_config.jsonwith the file contents.
I searched in tokenizer_config.json but there is no chat_template in there. Do you mean the Jinja file?
See the example below, this is what a tokenizer_config.json file should look like:
{
"add_prefix_space": false,
"audio_bos_token": "<|audio_start|>",
"audio_eos_token": "<|audio_end|>",
"audio_token": "<|audio_pad|>",
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"image_token": "<|image_pad|>",
"is_local": true,
"max_length": null,
"model_max_length": 262144,
"model_specific_special_tokens": {
"audio_bos_token": "<|audio_start|>",
"audio_eos_token": "<|audio_end|>",
"audio_token": "<|audio_pad|>",
"image_token": "<|image_pad|>",
"video_token": "<|video_pad|>",
"vision_bos_token": "<|vision_start|>",
"vision_eos_token": "<|vision_end|>"
},
"pad_to_multiple_of": null,
"pad_token": "<|endoftext|>",
"pad_token_type_id": 0,
"padding_side": "left",
"pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
"processor_class": "Qwen3VLProcessor",
"split_special_tokens": false,
"tokenizer_class": "TokenizersBackend",
"unk_token": null,
"video_token": "<|video_pad|>",
"vision_bos_token": "<|vision_start|>",
"vision_eos_token": "<|vision_end|>",
"chat_template": "..."
Interesting, it seems that chat_template is in the original Qwen's files, but not Intel or Lorbus files. There is at least one other part that's different too, besides chat_template.
Anyway, I can confirm v9 made improvements for vLLM. Still not 100% fixed with tool calls, but mostly fixed. Thanks!
--chat-template chat_template-v9.jinja works
BTW, I have now included the one line version of the template for tokenizer_config.json and I have also made some big improvements for tool calls in v10. Please test.