关于FunctionFormatter中think标签的疑问

[zhangch-ss]

Reminder

• I have read the above rules and searched the existing issues.

System Info

llamafactory-cli train，sft

Reproduction

您好，我对FunctionFormatter.apply中think标签的处理有一些疑问。
function_str = thought.group(1) + function_str，拼接后的str是

think<func_call>function call</func_call> thought.group(1)导致缺少了think标签。

这里用thought.group(0)
<think>think</think>\n\n<func_call>function call</func_call>是不是会好一些？

@dataclass
class FunctionFormatter(StringFormatter):
    def __post_init__(self):
        super().__post_init__()
        self.tool_utils = get_tool_utils(self.tool_format)

    @override
    def apply(self, **kwargs) -> SLOTS:
        content: str = kwargs.pop("content")
        regex = re.compile(r"<think>(.*)</think>", re.DOTALL)
        thought = re.search(regex, content)
        if thought:
            content = content.replace(thought.group(0), "")

        functions: List["FunctionCall"] = []
        try:
            tool_calls = json.loads(content)
            if not isinstance(tool_calls, list):  # parallel function call
                tool_calls = [tool_calls]

            for tool_call in tool_calls:
                functions.append(
                    FunctionCall(tool_call["name"], json.dumps(tool_call["arguments"], ensure_ascii=False))
                )

        except json.JSONDecodeError:
            raise RuntimeError(f"Invalid JSON format in function message: {str([content])}.")  # flat string

        function_str = self.tool_utils.function_formatter(functions)
        if thought:
            function_str = thought.group(1) + function_str

        return super().apply(content=function_str)

Others

No response

[hiyouga]

嗯，我觉得你说得没错，可以提个 PR 上来