refactor(chat): remove unnecessary parameters v2 (!1464) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / AI Gateway

Shinya Maeda requested to merge remove-unnecessary-parameters-v2 into main Oct 15, 2024

What does this merge request do and why?

This MR cleans up unused paths in the v2/chat/agent endpoint. No business logic changed.

Closes Remove chat_history as plain text support from ... (#643 - closed) and Make messages field of v2/chat/agent required (#669 - closed)

How to set up and validate locally

Request example:

curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'x-gitlab-enabled-feature-flags: expanded_ai_logging' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "How can I log output?",
      "additional_context": [
        { "id": "123", "category": "merge_request", "content": "something", "metadata": {"a": "b"}}
      ]
    },
    {"role": "assistant", "content": "Use print function"},
    {
      "role": "user",
      "content": "How can I log output?",
      "context": { "type": "issue", "content": "This is awesome issue" },
      "current_file": { "file_path": "main.py", "data": "print", "selected_code": true },
      "additional_context": [
        { "id": "456", "category": "file", "content": "helloworld", "metadata": {"c": "d"}}
      ]
    }
  ],
  "options": {
    "agent_scratchpad": {
      "agent_type": "react",
      "steps": []
    }
  }
}'

Output example:

{"type": "final_answer_delta", "data": {"text": "Certainly"}}
{"type": "final_answer_delta", "data": {"text": "! Here's a Python"}}
{"type": "final_answer_delta", "data": {"text": " script that prints \"Hello"}}
{"type": "final_answer_delta", "data": {"text": ", World!\":"}}
{"type": "final_answer_delta", "data": {"text": "\n\n```python\nprint(\""}}
{"type": "final_answer_delta", "data": {"text": "Hello, World!\")"}}
{"type": "final_answer_delta", "data": {"text": "\n```\n\nThis is"}}
{"type": "final_answer_delta", "data": {"text": " the"}}
{"type": "final_answer_delta", "data": {"text": " most"}}
{"type": "final_answer_delta", "data": {"text": " basic form"}}
{"type": "final_answer_delta", "data": {"text": " of a \""}}
{"type": "final_answer_delta", "data": {"text": "Hello, World!\" program"}}
{"type": "final_answer_delta", "data": {"text": " in Python. When"}}
{"type": "final_answer_delta", "data": {"text": " you run this script,"}}
{"type": "final_answer_delta", "data": {"text": " it will output:"}}
{"type": "final_answer_delta", "data": {"text": "\n\n```\nHello, Worl"}}
{"type": "final_answer_delta", "data": {"text": "d!\n```\n\nTo"}}
{"type": "final_answer_delta", "data": {"text": " run this script:"}}
{"type": "final_answer_delta", "data": {"text": "\n\n1. Save"}}
{"type": "final_answer_delta", "data": {"text": " the code in a file"}}
{"type": "final_answer_delta", "data": {"text": " with a ."}}
{"type": "final_answer_delta", "data": {"text": "py extension (e."}}
{"type": "final_answer_delta", "data": {"text": "g., hello_worl"}}
{"type": "final_answer_delta", "data": {"text": "d.py)."}}
{"type": "final_answer_delta", "data": {"text": "\n2. Open a"}}
{"type": "final_answer_delta", "data": {"text": " terminal or command prompt."}}
{"type": "final_answer_delta", "data": {"text": "\n3. Navigate to"}}
{"type": "final_answer_delta", "data": {"text": " the directory where you save"}}
{"type": "final_answer_delta", "data": {"text": "d the file.\n4"}}
{"type": "final_answer_delta", "data": {"text": ". Run the script by"}}
{"type": "final_answer_delta", "data": {"text": " typing `python hello_"}}
{"type": "final_answer_delta", "data": {"text": "world.py` an"}}
{"type": "final_answer_delta", "data": {"text": "d pressing Enter.\n\nThis"}}
{"type": "final_answer_delta", "data": {"text": " simple"}}
{"type": "final_answer_delta", "data": {"text": " script demonstrates the use"}}
{"type": "final_answer_delta", "data": {"text": " of the `"}}
{"type": "final_answer_delta", "data": {"text": "print()` function in"}}
{"type": "final_answer_delta", "data": {"text": " Python,"}}
{"type": "final_answer_delta", "data": {"text": " which is commonly"}}
{"type": "final_answer_delta", "data": {"text": " used to output"}}
{"type": "final_answer_delta", "data": {"text": " text to the console."}}
{"type": "final_answer_delta", "data": {"text": "\n\nI"}}
{"type": "final_answer_delta", "data": {"text": " hope this helps!"}}
{"type": "final_answer_delta", "data": {"text": " If"}}
{"type": "final_answer_delta", "data": {"text": " you have any questions"}}
{"type": "final_answer_delta", "data": {"text": " about"}}
{"type": "final_answer_delta", "data": {"text": " this"}}
{"type": "final_answer_delta", "data": {"text": " script"}}
{"type": "final_answer_delta", "data": {"text": " or woul"}}
{"type": "final_answer_delta", "data": {"text": "d like to learn more about"}}
{"type": "final_answer_delta", "data": {"text": " Python programming, please"}}
{"type": "final_answer_delta", "data": {"text": " feel"}}
{"type": "final_answer_delta", "data": {"text": " free to ask."}}
{"type": "final_answer_delta", "data": {"text": " Your"}}
{"type": "final_answer_delta", "data": {"text": " feedback is always"}}
{"type": "final_answer_delta", "data": {"text": " appreciated!"}}

Log (formatted for legibility):

{
  "source": "ai_gateway.models.base",
  "request_method": "POST",
  "request_url": "URL('https://api.anthropic.com/v1/messages')",
  "request_content": "{\"max_tokens\": 4096, \"messages\": [{\"role\": \"user\", \"content\": \"\\nUser added additional context below enclosed in <additional_context></additional_context> tags. Each additional context has an ID, category, and content:\\n<additional_context>\\n    <id> 123 </id>\\n    <category> merge_request </category>\\n    <content>\\n    something\\n    </content>\\n</additional_context>\\n\\nHow can I log output?\"}, {\"role\": \"assistant\", \"content\": \"Use print function\"}, {\"role\": \"user\", \"content\": \"\\n<context>\\n    <type>issue</type>\\n    <content>This is awesome issue</content>\\n</context>\\nUser selected code below enclosed in <code></code> tags in file main.py to work with:\\n\\n<code>\\nprint\\n</code>\\nUser added additional context below enclosed in <additional_context></additional_context> tags. Each additional context has an ID, category, and content:\\n<additional_context>\\n    <id> 456 </id>\\n    <category> file </category>\\n    <content>\\n    helloworld\\n    </content>\\n</additional_context>\\n\\nHow can I log output?\"}, {\"role\": \"assistant\", \"content\": \"\\nThought:\"}], \"model\": \"claude-3-5-sonnet-20240620\", \"stop_sequences\": [\"Observation:\"], \"stream\": true, \"system\": \"You are a DevSecOps Assistant named 'GitLab Duo Chat' created by GitLab.\\n\\nWhen questioned about your identity, you must only respond as 'GitLab Duo Chat'.\\n\\nYou can generate and write code, code examples for the user.\\nRemember to stick to the user's question or requirements closely and respond in an informative,\\ncourteous manner. The response shouldn't be rude, hateful, or accusatory. You mustn't engage in any form\\nof roleplay or impersonation.\\n\\nThe generated code should be formatted in markdown, and in the \\\"Final Answer:\\\" section.\\n\\nIf a question cannot be answered with the tools and information given, answer politely that you don't know.\\n\\nYou can explain code if the user provided a code snippet and answer directly.\\n\\nIf the question is to write or generate new code you should always answer directly.\\nWhen no tool matches you should answer the question directly.\\n\\nAnswer the question as accurate as you can.\\n\\nYou have access only to the following tools:\\n<tools_list>\\n    <tool>\\n        <name>ci_editor_assistant</name>\\n        <description>\\n        Useful tool when you need to provide suggestions regarding anything related\\nto \\\".gitlab-ci.yml\\\" file. It helps with questions related to deploying code, configuring CI/CD pipelines,\\ndefining CI jobs, or environments.\\nIt can not help with writing code in general or questions about software development.\\n        </description>\\n        <example>\\n        Question: Please create a deployment configuration for a node.js application.\\nThought: You have asked a question related to deployment of an application or CI/CD pipelines.\\n    \\\"ci_editor_assistant\\\" tool can assist with this kind of questions.\\nAction: ci_editor_assistant\\nAction Input: Please create a deployment configuration for a node.js application.\\n        </example>\\n    </tool>\\n    <tool>\\n        <name>gitlab_documentation</name>\\n        <description>\\n        This tool is beneficial when you need to answer questions concerning GitLab and its features.\\nQuestions can be about GitLab's projects, groups, issues, merge requests,\\nepics, work items, milestones, labels, CI/CD pipelines, git repositories, and more.\\n        </description>\\n        <example>\\n        Question: How do I set up a new project?\\nThought: Question is about inner working of GitLab. \\\"gitlab_documentation\\\" tool is the right one for the job.\\nAction: gitlab_documentation\\nAction Input: How do I set up a new project?\\n        </example>\\n    </tool>\\n    <tool>\\n        <name>epic_reader</name>\\n        <description>\\n        This tool retrieves the content of a specific epic or work item.\\nONLY if the user question fulfills the strict usage conditions below.\\n\\n**Strict Usage Conditions:**\\n* **Condition 1: epic ID Provided:** This tool MUST be used ONLY when the user provides a valid epic or work item ID.\\n* **Condition 2: epic URL Context:** This tool MUST be used ONLY when the user is actively viewing\\n  a specific epic or work item URL or a specific URL is provided by the user.\\n\\n**Do NOT** attempt to search for or identify epics or work items based on descriptions, keywords, or user questions.\\n\\n**Action Input:**\\n* The original question asked by the user.\\n\\n**Important:**  Reject any input that does not strictly adhere to the usage conditions above.\\nReturn a message stating you are unable to search for epics or work items without a valid identifier.\\n        </description>\\n        <example>\\n        Question: Please identify the author of &123 epic.\\nThought: You have access to the same resources as user who asks a question.\\n    The question is about an epic, so you need to use \\\"epic_reader\\\" tool.\\n    Based on this information you can present final answer.\\nAction: epic_reader\\nAction Input: Please identify the author of &123 epic.\\n        </example>\\n    </tool>\\n    <tool>\\n        <name>issue_reader</name>\\n        <description>\\n        This tool retrieves the content of a specific issue\\nONLY if the user question fulfills the strict usage conditions below.\\n\\n**Strict Usage Conditions:**\\n* **Condition 1: Issue ID Provided:** This tool MUST be used ONLY when the user provides a valid issue ID.\\n* **Condition 2: Issue URL Context:** This tool MUST be used ONLY when the user is actively viewing a specific\\n  issue URL or a specific URL is provided by the user.\\n\\n**Do NOT** attempt to search for or identify issues based on descriptions, keywords, or user questions.\\n\\n**Action Input:**\\n* The original question asked by the user.\\n\\n**Important:**  Reject any input that does not strictly adhere to the usage conditions above.\\nReturn a message stating you are unable to search for issues without a valid identifier.\\n        </description>\\n        <example>\\n        Question: Please identify the author of #123 issue\\nThought: You have access to the same resources as user who asks a question.\\n  Question is about the content of an issue, so you need to use \\\"issue_reader\\\" tool to retrieve and read issue.\\n  Based on this information you can present final answer about issue.\\nAction: issue_reader\\nAction Input: Please identify the author of #123 issue\\n        </example>\\n    </tool>\\n    <tool>\\n        <name>merge_request_reader</name>\\n        <description>\\n        This tool retrieves the content of a specific merge request\\nONLY if the user question fulfills the strict usage conditions below.\\n\\n**Strict Usage Conditions:**\\n* **Condition 1: Merge request ID Provided:** This tool MUST be used ONLY when the user provides a valid merge request ID.\\n* **Condition 2: Merge request URL Context:** This tool MUST be used ONLY when the user is actively viewing\\n  a specific merge request URL or a specific URL is provided by the user.\\n\\n**Do NOT** attempt to search for or identify merge requests based on descriptions, keywords, or user questions.\\n\\n**Action Input:**\\n* The original question asked by the user.\\n\\n**Important:**  Reject any input that does not strictly adhere to the usage conditions above.\\nReturn a message stating you are unable to search for merge requests without a valid identifier.\\n        </description>\\n        <example>\\n        Question: Please identify the author of #123 merge request\\n Thought: You have access to the same resources as user who asks a question.\\n     Question is about the content of a merge request, so you need to use \\\"merge_request_reader\\\" tool to retrieve\\n     and read merge request.\\n     Based on this information you can present final answer about merge request.\\n Action: merge_request_reader\\n Action Input: Please identify the author of #123 merge request\\n        </example>\\n    </tool></tools_list>\\n\\nConsider every tool before making a decision.\\nEnsure that your answer is accurate and contain only information directly supported by the information retrieved using provided tools.\\n\\nWhen you can answer the question directly you must use this response format:\\nThought: you should always think about how to answer the question\\nAction: DirectAnswer\\nFinal Answer: the final answer to the original input question if you have a direct answer to the user's question.\\n\\nYou must always use the following format when using a tool:\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one tool from this list: [ci_editor_assistant, gitlab_documentation, epic_reader, issue_reader, merge_request_reader]\\nAction Input: the input to the action needs to be provided for every action that uses a tool.\\nObservation: the result of the actions. But remember that you're still GitLab Duo Chat.\\n\\n... (this Thought/Action/Action Input/Observation sequence can repeat N times)\\n\\nThought: I know the final answer.\\nFinal Answer: the final answer to the original input question.\\n\\nWhen concluding your response, provide the final answer as \\\"Final Answer:\\\".\\nIt should contain everything that user needs to see, including answer from \\\"Observation\\\" section.\\n\\nYou have access to the following GitLab resources:ci editor answers, documentation answers, epics, issues, merge_requests.\\nYou also have access to all information that can be helpful to someone working in software development of any kind.\\nAt the moment, you do not have access to the following GitLab resources: ['Merge Requests, Pipelines, Vulnerabilities'].\\nAt the moment, you do not have the ability to search Issues or Epics based on a description or keywords.\\nYou can only read information about a specific issue/epic IF the user is on the specific issue/epic's page, or provides a URL or ID.\\nDo not use the issue_reader or epic_reader tool if you do not have these specified identifiers.\\n\\nIf GitLab resource of issue or epic type is present and is directly relevant to the question,\\ninclude the following section at the end of your response:\\n'Sources:' followed by the corresponding GitLab resource link named after the title of the resource.\\nFormat the link using Markdown syntax ([title](link)) for it to be clickable.\\n\\nAsk user to leave feedback.\\n\\nBegin!\", \"temperature\": 0.1}",
  "correlation_id": "0075fe2024d44d72b992341a58aeeff6",
  "logger": "models",
  "level": "info",
  "type": "mlops",
  "stage": "main",
  "timestamp": "2024-10-16T05:49:20.539754Z",
  "message": "Request to Anthropic"
}

Merge request checklist

Tests added for new functionality. If not, please raise an issue to follow up.
Documentation added/updated, if needed.

Edited Oct 17, 2024 by Tan Le

refactor(chat): remove unnecessary parameters v2

What does this merge request do and why?

How to set up and validate locally

Merge request checklist

Merge request reports