ds4sd

/

SmolDocling-256M-preview

like 1.55k

Follow

Docling 802

The chat_template.json you provided seems to output

"<|im_start|>User:{PROMPT_TEXT} <image><end_of_utterance>\nAssistant:"

although your example is

"<|im_start|>User:<image>{PROMPT_TEXT}<end_of_utterance>\nAssistant:"

Shouldn't <image> be before the prompt?
I am using PROMPT_TEXT="Convert this page to docling." and it is not accurate.

When I try to put this template as a jinja file for vLLM it gives bad output.
And when trying this template:

<|im_start|>{% for message in messages %}{{message['role'] | capitalize}}:{% for line in message['content'] %}{% if line['type'] == 'image' or line['type'] == 'image_url' %}{{ '<image>' }}{% endif %}{% endfor %}{% for line in message['content'] %}{% if line['type'] == 'text' %}{{ line['text'] }}{% endif %}{% endfor %}<end_of_utterance>
{% endfor %}{% if add_generation_prompt %}Assistant:{% endif %}

which works in vLLM.
So, in summary, Is there a problem with the provided chat template json file?
Thanks.

Hi, can you please what configuration of chat_template.json and chat_template = f"<|im_start|>User:{PROMPT_TEXT}\nAssistant:" works for you?

I'm also trying to run the vllm example and it is not following the doctags properly.

I'm getting output like this "58>61>432>113>On December 15 and 16, 2023"

Another example

<text>125>113>215>120>February 15, 2025</text>

Hello @nitincypher ,
Regarding your second problem of getting outputs like this <text>125>113>215>120>February 15, 2025</text>, when you decode you need to set skip_special_characters to False.

Did you try the one I provided in my question? Making <image> before prompt text?
Also, the example of vLLM library they provided seems to work. But for a vLLM docker instance, were you able to make sure it loads the file correctly and outputs the template in logs?

Hello @nitincypher ,
Regarding your second problem of getting outputs like this <text>125>113>215>120>February 15, 2025</text>, when you decode you need to set skip_special_characters to False.

where is that to be set?