server: update deepseek reasoning format (pass reasoning_content as diffs)
#13933
Conversation
…ffs), add legacy option for compat
| // for reasoning model, we split the message into content and thought | ||
| // TODO: implement this as remark/rehype plugin in the future | ||
| const { content, thought, isThinking }: SplitMessage = useMemo(() => { | ||
| if (msg.content === null || msg.role !== 'assistant') { | ||
| return { content: msg.content }; | ||
| } | ||
| let actualContent = ''; | ||
| let thought = ''; | ||
| let isThinking = false; | ||
| let thinkSplit = msg.content.split('<think>', 2); | ||
| actualContent += thinkSplit[0]; |
There was a problem hiding this comment.
If user doesn't explicitly enable deepseek reasoning format (which is the default behavior), this code is there to make sure the thinking is always parsed.
Unless we can control thinking format per-request, I don't think we can remove this code block
There was a problem hiding this comment.
reverted UI changes
| @@ -112,6 +113,7 @@ export interface ViewingChat { | |||
|
|
|||
| export type PendingMessage = Omit<Message, 'content'> & { | |||
| content: string | null; | |||
| reasoningContent: string | null; | |||
There was a problem hiding this comment.
Tbh I'm not very confident adding this, because it is not future-proof.
If you look at how chatgpt structure their message, the content is an array instead of a string, and that is what I wanted to do in near future. It will allow different message parts, like for example a message can contains both reasoning, text response and a tool call
| @@ -2869,6 +2869,7 @@ common_params_context common_params_parser_init(common_params & params, llama_ex | |||
| "(default: deepseek)", | |||
| [](common_params & params, const std::string & value) { | |||
| /**/ if (value == "deepseek") { params.reasoning_format = COMMON_REASONING_FORMAT_DEEPSEEK; } | |||
| else if (value == "deepseek-legacy") { params.reasoning_format = COMMON_REASONING_FORMAT_DEEPSEEK_LEGACY; } | |||
There was a problem hiding this comment.
should we add a help message to explain this mode?
Fixes #13867
(
updated / simplified webui accordinglyedit: reverted UI changes, will let @ngxson update it as follow up)