Skip to content

fix: factor cancellation into `isStreaming` state

Tristan Read requested to merge tr-fix-chat-is-streaming-state into main

What does this MR do?

While working on feat: Add cancelPrompt handlers for DuoChat (gitlab-vscode-extension!1783 - merged) and [DuoChat] Add CancelPrompt handler (gitlab-org/editor-extensions/gitlab-jetbrains-plugin!545 - merged) we encountered an issue:

If a user submits a prompt and cancels while the message is streaming and then goes to submit another prompt the following response will start streaming with the cancel prompt button available. But after a brief moment, the cancel button disappears while the rest of the message streams to completion.

What's the root cause?

The displaySubmitButton state is controlled internally in this component by watchers, which means it depends on the order that the input props are updated. A momentary (even one tick long) setting of both isLoading and isStreaming to false is enough to re-enable the button with no recovery. Therefore, careful management of these two pieces of state is crucial.

Is this a new issue?

In the web version, isLoading is 'controlled' and gets manually set when the set-user-prompt event is emitted.

In the VS Code / JetBrains versions, isLoading is derived, and switches to true only once the network request is triggered.

This was not an issue before cancel was implemented. The Duo Chat component requires that cancellation be implemented by 'turning off the faucet' of chunk updates that stream in. The isStreaming state checks to see if the last message contains a chunkId. If updates are switched off, the isStreaming will remain true after that, until the next user message occurs.

Solution

Ensure that isStreaming switches back to false when an in-progress stream is cancelled.

Detailed breakdown

The play by play of internal variables goes as follows. There are two phases - setup & trigger.

Setup

  1. User inputs a prompt.
  2. Prompt starts streaming back to the UI
    • At this point, isLoading is false, isStreaming is true.
  3. User hits cancel button.
    • This forces displaySubmitButton to be true, re-enabling 'paper airplane mode'.
    • Importantly, because the last message has in-flight chunks, isStreaming remains in the true state until the next prompt is sent by the user.

Triggering the bug

  1. Now the user inputs another prompt and presses submit.
    • The submit click sets displaySubmitButton to false.
  2. After a delay (it awaits the request to ), the user message is added to the messages list.
    • lastMessage is now the user message, which causes isStreaming to switch to false.
    • In VS Code / JetBrains: The last (aka user) message does not have the 'pending' state so isLoading is false. *
    • The isStreaming watcher triggers which sets displaySubmitButton back to true.
  3. After another delay (one tick?) the assistant message appears in the message list.
    • The isLoading state now goes to true.
    • This should update the loading button state, but it does not.

* In web, isLoading is forced to true during step 1 in the send-chat-prompt event handler.

Important notes

This is not a blocker for the JetBrains / VS Code extensions - we can fix the issue there by behaving as the web version does: moving isLoading from derived to controlled state and setting it to true in response to the button being clicked.

However - I think it's still worth fixing as it makes the state more internally consistent, and may help mitigate bugs down the line as we further iterate on these features.

Integration merge requests

Screenshots / Demo

To reproduce:

  1. Check out this commit in the VS Code extension repo.
  2. (optional) Integrate the fix using yalc publish / yalc add locally with this branch.
before after
chat_loading_issue chat_fixed

Does this MR meet the acceptance criteria?

This checklist encourages the authors, reviewers, and maintainers of merge requests (MRs) to confirm changes were analyzed for conformity with the project's guidelines, security and accessibility.

Toggle the acceptance checklist

Conformity

  • Code review guidelines.
  • GitLab UI's contributing guidelines.
  • [n/a] If it changes a Pajamas-compliant component's look & feel, the MR has been reviewed by a UX designer.
  • [n/a] If it changes GitLab UI's documentation guidelines, the MR has been reviewed by a Technical Writer.
  • If the MR changes a component's API, integration MR(s) have been opened (see integration merge requests above).
  • Added the ~"component:*" label(s) if applicable.

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [n/a] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [n/a] Security reports checked/validated by a reviewer from the AppSec team

Accessibility

If this MR adds or modifies a component, take a few moments to review the following:

  • [n/a] All actions and functionality can be done with a keyboard.
  • [n/a] Links, buttons, and controls have a visible focus state.
  • [n/a] All content is presented in text or with a text equivalent. For example, alt text for SVG, or aria-label for icons that have meaning or perform actions.
  • [n/a] Changes in a component’s state are announced by a screen reader. For example, changing aria-expanded="false" to aria-expanded="true" when an accordion is expanded.
  • [n/a] Color combinations have sufficient contrast.
Edited by Tristan Read

Merge request reports