Skip to content

Add code suggestions test and env

Mark Lapierre requested to merge ml-add-cs-perf-test into main

What does this MR do?

Adds a test of the Code Suggestions Completions API endpoint. See gitlab-org/gitlab#424424 (closed)

The environment URL http://35.232.161.224 is a temporary one until a new GPT project is ready.

The thresholds will probably need to be tuned, but for now this is intended to be able to run the test and allow us to collect some performance data. I've opened https://gitlab.com/gitlab-org/gitlab/-/issues/429786 to determine proper test data and targets.

FYI I copied a lot of this from @john.mcdonnell's MR: !510 (closed)

Checklist

  • [-] The prompt is the test data. Later work will be needed to quantify the effect of different prompts Ensure test data for the new endpoint is representative. We recommend using large data ("edge cases") in performance tests. The impact of optimizations is more visible on the big numbers and large data may expose more performance bottlenecks.
  • Fill in @endpoint, @example_uri, @description, and @gpt_data_version tags with the information regarding the new test.
    • @gpt_data_version should be generally 1 unless you know the test is for data that's in a specific version.
  • [-] AFAIK we're not applying this target to AI features Create a bug if the endpoint response time is above the performance main target (200 ms).
    • Specify created issue link under @issue tag.
  • Consider specifying the additional @flags:
    • unsafe - required for Unsafe tests. Ensure that data before and after the test is consistent.
    • search - labels Search related tests.
    • dash_url - labels Web tests with the check if the endpoint path has a dash \ redirect.
  • [-] Web tests
Edited by Mark Lapierre

Merge request reports

Loading