Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: streaming reasoning content #3310

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fangzhen
Copy link

This patch re-uses the remove_reasoning model settings.
e.g. with remove_reasoning: reasoning_content, it will stream reaoning_content field of message delta inside
<reasoning_content> </reasoning_content>

This fixes #3086 in a way

My testing:
.aider.model.settings.yml

- name: deepseek/deepseek-reasoner
  remove_reasoning: reasoning_content
- name: openrouter/deepseek/deepseek-r1
  remove_reasoning: reasoning
  extra_params:
    extra_body:
      provider:
        order: ["Nebius"]
        allow_fallbacks: false
      include_reasoning: true
      stream: true

The reasoning content is streamed in tags specified by remove_reasoning field

$ aider -c ~/.aider/conf/general.yml --model openrouter/deepseek/deepseek-r1

Aider v0.74.3.dev37+g62c35449.d20250220
Model: openrouter/deepseek/deepseek-r1 with diff edit format, infinite output
Git repo: .git with 2 files
Repo-map: using 4096 tokens, auto refresh

diff> /ask just say hi.

<reasoning>
Okay, the user asked me to just say hi. Let me check the conversation history to make sure there's no context I'm missing.

Looking back, the user mentioned they're working on a git repo with some files (bar.py and foo.py) but didn't share the contents. They instructed me not to use SEARCH/REPLACE blocks and to describe code changes freely. However, the current request is simply to say hi. 

I need to respond in a friendly and concise manner. Since the user wants the response in the same language they're using, which is English here, I'll keep it straightforward. No need to mention code or files since the user just wants a greeting. 

Let me make sure I'm following all guidelines: no markdown, keep it natural. Just a simple "Hi! 👋" should suffice. No extra information or questions unless the user asks for something else next.

</reasoning>
Hi! 👋
Tokens: 161 sent, 195 received. Cost: $0.00052 message, $0.00052 session.

diff> /clear


diff> /model deepseek/deepseek-reasoner

Aider v0.74.3.dev37+g62c35449.d20250220
Main model: deepseek/deepseek-reasoner with diff edit format, infinite output
Weak model: openrouter/deepseek/deepseek-r1
Git repo: .git with 2 files
Repo-map: using 4096 tokens, auto refresh

diff> /ask just say hi.

<reasoning_content>
Okay, the user told me to just say "hi". Let me check the conversation history to make sure there's no other context I need to consider.

Previous messages show the user setting me up as a code analyst for their git repo, but they mentioned they aren't sharing files yet. Then they asked me to say hi. 

I should respond with a simple greeting. No need for any technical terms or further questions. Keep it friendly and straightforward. Just "Hi!" would work. No extra emojis or anything unless the user's style suggests that. Looking at their past messages, they use a neutral tone, so a plain "Hi!" suffices.
</reasoning_content>
Hi!
Tokens: 161 sent, 144 received. Cost: $0.00040 message, $0.00092 session.

This patch re-uses the `remove_reasoning` model settings.
e.g. with `remove_reasoning: reasoning_content`, it will stream
`reaoning_content` field of message delta inside
<reasoning_content> </reasoning_content>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Suggestion: See what the reasoning models are thinking before they give their output.
2 participants