Qwen2.5-VL-7B的问题

[ffanyt]

我的命令是：
vllm serve /home/share/models/Qwen2.5-VL-3B-Instruct --limit_mm_per_prompt image=5 \ --dtype float16 \ --port 10004 \ --tensor-parallel-size 4 \ --gpu-memory-utilization 0.9 \ --max-model-len 32768

我发现7B的模型会回答感叹号

而3b的模型一切正常

[wangxiyuan]

@ganyi1996ppo Any update?

[ganyi1996ppo]

我的命令是： vllm serve /home/share/models/Qwen2.5-VL-3B-Instruct --limit_mm_per_prompt image=5 \ --dtype float16 \ --port 10004 \ --tensor-parallel-size 4 \ --gpu-memory-utilization 0.9 \ --max-model-len 32768

我发现7B的模型会回答感叹号

而3b的模型一切正常

Thanks for reporting this issue, did you use the latest release version of vllm-ascend? If so, maybe you can try to set a specific seed to the request, this might helps on the accuracy issue.