Sometimes chat gets stuck on picking same tool again and again
Sometimes chat gets stuck on picking same tool again and again
originally reported by @jprovaznik on Feedback from the team when testing
With having folowing chat history
Human: Can you summarize #1?
Assistant: Here is the summary of issue #1 in bullet points:
• Voluptatibus aut quo dolores facilis et et nihil consectetur.
• Difficult to evaluate which provider is better.
• Each provider has pros and cons.
Human: How do I change my password in GitLab
Assistant: {"content":"To change your password in GitLab:\n\n1. On the left sidebar, select your avatar.\n2. Select **Edit profile**. \n3. On the left sidebar, select **Password**.\n4. In the **Current password** text box, enter your current password.\n5. In the **New password** and **Password confirmation** text box, enter your new password.\n6. Select **Save password**.","sources":[{"source_url":"http://127.0.0.1:3000/help/subscriptions/customers_portal","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Purchase","stage":"Fulfillment","title":"The Customers Portal","source":"/doc/subscriptions/customers_portal.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/subscriptions/customers_portal","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Purchase","stage":"Fulfillment","title":"The Customers Portal","source":"/doc/subscriptions/customers_portal.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/profile/index","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","type":"index, howto","group":"Authentication and Authorization","stage":"Manage","title":"User account","source":"/doc/user/profile/index.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/profile/user_passwords","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Authentication and Authorization","stage":"Manage","title":"User passwords","source":"/doc/user/profile/user_passwords.md","source_type":"doc"}]}
Human: How do I fork a project?
Assistant: {"content":"To fork a project:\n\n1. On the project's homepage, in the upper-right corner, select **Fork** (**{fork}**):\n ![Fork this project](img/forking_workflow_fork_button_v13_10.png)\n2. Optional. Edit the **Project name**.\n3. For **Project URL**, select the [namespace](../../namespace/index.md)\n your fork should belong to.\n4. Add a **Project slug**. This value becomes part of the URL to your fork.\n It must be unique in the namespace.\n5. Optional. Add a **Project description**.\n6. Select the **Visibility level** for your fork.","sources":[{"source_url":"http://127.0.0.1:3000/help/user/project/merge_requests/creating_merge_requests","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Code Review","stage":"Create","title":"Creating merge requests","source":"/doc/user/project/merge_requests/creating_merge_requests.md","description":"How to create merge requests in GitLab.","source_type":"doc"},{"source_url":"http://127.0.0.1:3000/help/user/project/repository/forking_workflow","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Source Code","stage":"Create","title":"Project forking workflow","source":"/doc/user/project/repository/forking_workflow.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/project/repository/forking_workflow","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Source Code","stage":"Create","title":"Project forking workflow","source":"/doc/user/project/repository/forking_workflow.md","source_type":"doc"}]}
Then for question 'Can you explain the code ""def hello_world\nputs(""Hello, world!\n"");\nend""?' it keeps picking explain code tool again and again without providing final answer (although reasonable output is added in each iteration as Observation:
to the prompt. After removing Thought:
from zero shot prompt, it doesn't get stuck.
prompt after result of explain code was added 10 times to it
Human: Can you summarize #1?
Assistant: Here is the summary of issue #1 in bullet points:
• Voluptatibus aut quo dolores facilis et et nihil consectetur.
• Difficult to evaluate which provider is better.
• Each provider has pros and cons.
Human: How do I change my password in GitLab
Assistant: {"content":"To change your password in GitLab:\n\n1. On the left sidebar, select your avatar.\n2. Select **Edit profile**. \n3. On the left sidebar, select **Password**.\n4. In the **Current password** text box, enter your current password.\n5. In the **New password** and **Password confirmation** text box, enter your new password.\n6. Select **Save password**.","sources":[{"source_url":"http://127.0.0.1:3000/help/subscriptions/customers_portal","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Purchase","stage":"Fulfillment","title":"The Customers Portal","source":"/doc/subscriptions/customers_portal.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/subscriptions/customers_portal","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Purchase","stage":"Fulfillment","title":"The Customers Portal","source":"/doc/subscriptions/customers_portal.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/profile/index","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","type":"index, howto","group":"Authentication and Authorization","stage":"Manage","title":"User account","source":"/doc/user/profile/index.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/profile/user_passwords","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Authentication and Authorization","stage":"Manage","title":"User passwords","source":"/doc/user/profile/user_passwords.md","source_type":"doc"}]}
Human: How do I fork a project?
Assistant: {"content":"To fork a project:\n\n1. On the project's homepage, in the upper-right corner, select **Fork** (**{fork}**):\n ![Fork this project](img/forking_workflow_fork_button_v13_10.png)\n2. Optional. Edit the **Project name**.\n3. For **Project URL**, select the [namespace](../../namespace/index.md)\n your fork should belong to.\n4. Add a **Project slug**. This value becomes part of the URL to your fork.\n It must be unique in the namespace.\n5. Optional. Add a **Project description**.\n6. Select the **Visibility level** for your fork.","sources":[{"source_url":"http://127.0.0.1:3000/help/user/project/merge_requests/creating_merge_requests","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Code Review","stage":"Create","title":"Creating merge requests","source":"/doc/user/project/merge_requests/creating_merge_requests.md","description":"How to create merge requests in GitLab.","source_type":"doc"},{"source_url":"http://127.0.0.1:3000/help/user/project/repository/forking_workflow","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Source Code","stage":"Create","title":"Project forking workflow","source":"/doc/user/project/repository/forking_workflow.md","source_type":"doc"},{"source_url":"http://192.168.1.8:3000/help/user/project/repository/forking_workflow","info":"To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments","group":"Source Code","stage":"Create","title":"Project forking workflow","source":"/doc/user/project/repository/forking_workflow.md","source_type":"doc"}]}
Human: You are a DevSecOps Assistant named 'GitLab Duo Chat' created by GitLab.
By only engaging in discussions pertinent to DevSecOps, software development, source code,
project management, CI/CD or GitLab, your task is to process this request.
When questioned about your identity, you must only respond as 'GitLab Duo Chat'.
You can generate and write code, code examples for the user.
Remember to stick to the user's question or requirements closely and respond in an informative,
courteous manner. The response shouldn't be rude, hateful, or accusatory. You mustn't engage in any form
of roleplay or impersonation.
The generated code should be formatted in markdown.
If a question cannot be answered with the tools and information given, answer politely that you don’t know.
If the question is to write or generate new code you should always answer directly.
When no tool matches you should answer the question directly.
Answer the question as accurate as you can.
Start with identifying the resource first.
You have access to the following tools:
- ExplainCode: Useful tool to explain code snippets and blocks.
Here is an example of using this tool:
Question: How would you improve the ```def hello_world
puts('Hello, world!\n');
end``` code? Picked tools: "ExplainCode" tool. Reason: The question has a code block that needs improvement. "ExplainCode" tool can process this question.
- IssueIdentifier: Useful tool when you need to identify a specific issue. Do not use this tool if you have already identified the issue.In this context, word `issue` means core building block in GitLab that enable collaboration, discussions, planning and tracking of work.
Here is an example of using this tool:
Question: Please identify the author of http://gitlab.example/ai/test/-/issues/1 issue
Picked tools: First: "IssueIdentifier" tool, second: "Resource Reader" tool.
Reason: You have access to the same resources as user who asks a question.
There is issue identifier in the question, so you need to use "IssueIdentifier" tool.
Once the issue is identified, you should use "Resource Reader" tool to fetch relevant information
about the resource. Based on this information you can present final answer.
- Resource Reader: Useful tool when you need to get information about specific resource that was already identified. Action Input for this tools always starts with: `data`
Here is an example of using this tool:
Question: Who is an author of this issue
Picked tools: First: "IssueIdentifier" tool, second: "Resource Reader" tool.
Reason: You have access to the same resources as user who asks a question.
Once the resource is identified, you should use "Resource Reader" tool to fetch relevant information
about the resource. Based on this information you can present final answer.
- SummarizeComments: This tool is useful when you need to create a summary of all notes, comments or discussions on a given, identified resource.
Here is an example of using this tool:
Question: Please summarize the http://gitlab.example/ai/test/-/issues/1 issue in the bullet points
Picked tools: First: "IssueIdentifier" tool, second: "SummarizeComments" tool.
Reason: There is issue identifier in the question, so you need to use "IssueIdentifier" tool.
Once the issue is identified, you should use "SummarizeComments" tool to summarize the issue.
For the final answer, please rewrite it into the bullet points.
- GitlabDocumentation: This tool is useful when you need to answer questions about GitLab,
like how to perform certain actions or how to use certain features.
Questions can be about GitLab's projects, groups, issues, merge requests,
epics, milestones, labels, CI/CD pipelines, git repositories, and more.
Here is an example of using this tool:
Question: How do I set up a new project?
Picked tools: "GitlabDocumentation" tool.
Reason: Question is about inner working of GitLab. "GitlabDocumentation" tool is the right one for
the job.
Consider every tool before making decision.
Identifying resource mustn't be the last step.
Ensure that your answer is accurate and doesn’t contain any information not directly supported
by the information retrieved using provided tools.
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one from this list: ExplainCode, IssueIdentifier, Resource Reader, SummarizeComments, GitlabDocumentation
Action Input: the input to the action
Observation: the result of the actions
... (this Thought/Action/Action Input/Observation sequence can repeat N times)
Thought: I know the final answer.
Final Answer: the final answer to the original input question.
REMEMBER to ALWAYS start a line with "Final Answer:" to give me the final answer.
Begin!
Thought: This code defines a method named hello_world that prints "Hello, world!" when called.
Action: ExplainCode
Action Input: ```ruby
def hello_world
puts("Hello, world!\n");
end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This line defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This line denotes the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called. Action: ExplainCode Action Input: ```ruby def hello_world puts("Hello, world!\n"); end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This line defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This line denotes the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called.
Action: ExplainCode
Action Input: ```ruby
def hello_world
puts("Hello, world!\n");
end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This line defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This line denotes the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called. Action: ExplainCode Action Input: ```ruby def hello_world puts("Hello, world!\n"); end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!\n")
This line prints Hello, world! followed by a newline \n.
end
This marks the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called.
Action: ExplainCode
Action Input: ```ruby
def hello_world
puts("Hello, world!\n");
end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This line defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This line denotes the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called. Action: ExplainCode Action Input: ```ruby def hello_world puts("Hello, world!\n"); end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This line defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This line denotes the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called.
Action: ExplainCode
Action Input: ```ruby
def hello_world
puts("Hello, world!\n");
end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This marks the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called. Action: ExplainCode Action Input: ```ruby def hello_world puts("Hello, world!\n"); end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!")
This line prints Hello, world! to the console. puts is the Ruby method for printing/outputting to the console.
end
This marks the end of the method definition.
So in short, this Ruby code defines a simple method that prints a greeting when called.
Thought: This code defines a method named hello_world that prints "Hello, world!" when called.
Action: ExplainCode
Action Input: ```ruby
def hello_world
puts("Hello, world!\n");
end
Observation: This is a code snippet in the Ruby programming language. It defines a method called hello_world that prints "Hello, world!" when called.
Specifically:
def hello_world
This defines the start of the method. def is the keyword for defining a method in Ruby.
puts("Hello, world!\n")
This line prints Hello, world! followed by a newline \n.
end
This denotes the end of the method definition.
So in short, this method just prints a greeting when called. You could call the method like this:
hello_world()
And it would print:
Hello, world!
Question: Can you explain the code ""def hello_world\nputs(""Hello, world!\n"");\nend""? Thought:
Assistant:
</details>