webui: Add a "Continue" Action for Assistant Message

[allozaur]

Close #16097

Add Continue and Save features for chat messages

What's new

Continue button for assistant messages

• Click the arrow button on any assistant response to keep generating from where it left off
• Useful for getting longer outputs or continuing after you've edited a response
• New content gets appended to the existing message

Save button when editing user messages

• Now you get three options when editing: Cancel, Save, and Send
• Save keeps your edit without regenerating the response (preserves the conversation below)
• Send saves and regenerates like before
• Useful when you just want to fix a typo without losing the assistant's response

Technical notes

• Added continueAssistantMessage() and editUserMessagePreserveResponses() methods to ChatStore
• Continue feature sends a synthetic "continue" prompt to the API (not saved to the database)
• Assistant message edits now preserve trailing whitespace for proper continuation
• Follows existing component architecture patterns

Demos

ggml-org/gpt-oss-20b-GGUF

demo1.mp4

unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF

demo2.mp4

[allozaur]

@ngxson @ggerganov lemme know if you think that this logic works for handling the "Continue" action for assistant messages.

@artyfacialintelagent fell free to test this out and give feedback!

[ggerganov]

Is this supposed to work correctly when pressing Continue after stopping a response while it is generating? I am testing with gpt-oss and after Continue the text does not seem to resume as expected.

[allozaur]

Is this supposed to work correctly when pressing Continue after stopping a response while it is generating? I am testing with gpt-oss and after Continue the text does not seem to resume as expected.

I've tested it for the edited assistant responses so far. I will take a close look at the stopped generation -> continue flow as well

[Iq1pl]

Is this supposed to work correctly when pressing Continue after stopping a response while it is generating? I am testing with gpt-oss and after Continue the text does not seem to resume as expected.

When using gpt-oss in Lm Studio the model generates a new response instead of continuing the previous text, this is because of the Harmony parser, uninstalling it resolves this and the model continues the generation successfully.

[allozaur]

@ggerganov please check the demos i've attached to the PR description and also test this feature on your end. looking forward to your feedback!

[ggerganov]

Continue feature sends a synthetic "continue" prompt to the API (not saved to the database)

Hm, I wonder why do it like this. We already have support on the server to continue the assistant message if it is the last one in the request #13174:

https://github.com/ggml-org/llama.cpp/blob/c7e23c79cf4316c385cee0a9d53b688b0d66686f/tools/server/utils.hpp#L729-L751

The current approach often does not continue properly, as can be seen in the sample videos:

Using the assistant prefill functionality above would make this work correctly in all cases.

[ngxson]

Agree with @ggerganov , it's better to use the prefill assistant message from #13174

Just one thing to note though, I think most templates does not support formatting the reasoning content back to original, so probably that's the only case where it will break

[allozaur]

Thanks guys, I missed that! Will patch it and come back to you.

[allozaur]

@ggerganov @ngxson

I've updated the logic with 859e496 and i have tested with few models and only 1 (Qwen3-VL-32B-Instruct-GGUF) managed to properly continue the assistant message in response to the prefill request. See videos below.

Qwen3-VL-32B-Instruct-GGUF

Qwen3-VL-32B-Instruct-GGUF.mov

ggml-org/gpt-oss-20b-gguf

gpt-oss-20b-gguf.mov

ggml-org/gpt-oss-120b-gguf

gpt-oss-120b-gguf.mov

unsloth/gemma3-12b-it-gguf

demo.mp4

[ggerganov]

For me, both Qwen3 and Gemma3 are able to complete successfully. For example, here is Gemma3 12B IT:

webui-continue-0.mp4

It's strange that it didn't work for you.

Regarding gpt-oss - I think that "Continue" has to also send the reasoning in this case. Currently, it is discarded and I think it confuses the model.

[allozaur]

Regarding gpt-oss - I think that "Continue" has to also send the reasoning in this case. Currently, it is discarded and I think it confuses the model.

Should we then address the thinking models differently for now, at least from the WebUI perspective?

It's strange that it didn't work for you.

I will do some more testing with other instruct models and make sure all is working right.

[ngxson]

It's likely due to chat template, I suspect some chat templates (especially jinja) adds the generation prompt. Can you verify how the chat template looks like with POST /apply-template endpoint? (the request body is the same as /chat/completions)

[ggerganov]

Regarding gpt-oss - I think that "Continue" has to also send the reasoning in this case. Currently, it is discarded and I think it confuses the model.

Should we then address the thinking models differently for now, at least from the WebUI perspective?

If it's not too complicated, I'd say change the logic so that "Continue" includes the reasoning of the last assistant message for all reasoning models.

[ngxson]

If it's not too complicated, I'd say change the logic so that "Continue" includes the reasoning of the last assistant message for all reasoning models.

The main issue is that some chat templates actively suppress the reasoning content from assistant messages, so I'm doubt if it will work cross all model.

Actually I'm thinking about a more generic approach, we can implement a feature in the backend such that both the "raw" generated text (i.e. with <think>, <reasoning>, etc) can be sent along with the already-parsed version.

I would say for now, we can put a warning in the webui to tell user that this feature is experimental and doesn't work cross all models. We can improve it later if it gets more usage.

[allozaur]

@ngxson i guess this doesn't stop us from having that PR reviewed and eventually merged?

Yes the current approach in this PR should be enough. Will give it a try a bit later.

Sure, lemme know!

[ngxson]

Noticed a small bug: if the server can absolute not continue the message, it will response with a stop event right away. This cause the webui to show an error, even when the request technically be success:

[allozaur]

Noticed a small bug: if the server can absolute not continue the message, it will response with a stop event right away. This cause the webui to show an error, even when the request technically be success:

So instead it should inform the user that the response can not be continued or do sth else? Let me know what u think would be the best pattern here.

[ngxson]

I think it should behave like nothing is added, without any error message. IIRC LM Studio has the same behavior.

[ngxson]

the response can not be continued

yeah a message like that can also be a good solution

[allozaur]

I think it should behave like nothing is added, without any error message. IIRC LM Studio has the same behavior.

@ngxson finally decided to go with this, check e68b416 + 9f653dc

[ngxson]

Thanks, it works as expected now

[ServeurpersoCom]

Tested during agentic loop also :)

Stop-And-Start-With-MCP.mp4