Skip to content

Ensure additional context is added to chat_history

Missy Davies requested to merge md-send-add-ctx-chat-history into master

What does this MR do and why?

For #494897 (closed)

In !164197 (merged) and !166155 (merged), we added functionality to store additional_context in chat_history. However, the additional context provided in one message was not available in subsequent messages.

This MR surfaces additional_context inside of chat_history when building the conversation list of messages so the context is then properly passed to the AIGW and included when assembling the final prompt for Anthropic.

This is a followup to gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!1415 (merged)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

graphql query Duo Chat Rails logs
Screenshot_2024-10-07_at_12.53.56_PM Screenshot_2024-10-07_at_12.53.32_PM Screenshot_2024-10-07_at_12.57.26_PM
Prettified version of the json in the screenshot above

{
  "messages": [
    {
      "role": "user",
      "content": "Here are some important colors to remember. Do you understand? Respond only with yes or no.",
      "additional_context": [
        {
          "id": "favorite_colors.txt",
          "category": "snippet",
          "content": "pink, blue, turquoise"
        }
      ]
    },
    {
      "role": "assistant",
      "content": "Yes.\n\n    I hope this answer meets your expectations. If you have any more questions or need further clarification, please don't hesitate to ask. Your feedback is valuable to us, so please feel free to share your thoughts on our interaction.",
      "additional_context": null
    },
    {
      "role": "user",
      "content": "What are the important colors?",
      "context": null,
      "current_file": null,
      "additional_context": []
    }
  ],
  "model_metadata": null,
  "unavailable_resources": [
    "Pipelines",
    "Vulnerabilities"
  ],
  "options": {
    "agent_scratchpad": {
      "agent_type": "react",
      "steps": []
    }
  }
}

How to set up and validate locally

  1. Run gdk locally, ensure GITLAB_SAAS=0
  2. Enable the following feature flags: v2_chat_agent_integration, duo_additional_context (and maybe also ai_duo_code_suggestions_switch )
  3. Add a print statement on line 96 of step_executor.rb to puts(params.inspect)
  4. In your terminal gdk tail rails
  5. Navigate to the graphql explorer: http://gdk.test:3000/-/graphql-explorer
  6. Run a mutation to send a chat message with additional context:
    mutation {
      aiAction(
        input: {
          chat: {
            content: "Tell me about this random code!", 
            additionalContext: [
              {
       	      category: SNIPPET, 
               id: "cia.rb", 
               content: "puts 'Ciao, world'",
               metadata: { more: "info" }
             }
      ]}}) {
        clientMutationId
        errors
      }
    }
  7. Run another mutation to send a second message without additional context (or use the UI)
  8. Inside the rails logs look for the params.inspect output and note that additional_context is included in the chat history for the second message you sent
  9. Try asking a question about the additional context you sent. Duo should remember it.
old

If it doesn't work properly, try applying this patch: patch.txt

Screenshot_2024-10-04_at_8.01.30_AM

Edited by Missy Davies

Merge request reports

Loading