Add Code Creation exception to Error Budget
Why is this change being made?
This updates the https://about.gitlab.com/handbook/engineering/error-budgets/#stage-groups-with-different-error-budgets handbook to add Create:Code Creation as a temporary exception.
Description of the problem that is the cause of the budget spend: We are quickly iterating on the interactions with various LLMs to roll out the code suggestion features. These features are new and not yet GA. These language models have various response times that we are working to investigate. The bulk of the error budget is used by runway_ingress
on the ai_gateway
, which is where we call the LLMs.
Relevant resources showing that the work is scoped out:
- Create:Code Creation - Investigate Error Budget (gitlab-org/gitlab#425095 - closed)
- Root Cause Analysis : Latency for Code Suggestion (gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#110 - closed)
- https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34624+
Target date when the exception must be revisited: 2024-01-31
. I chose this time frame because it gives us time to work on getting our features to GA. During that process, we hope to address some of the latency problems we are seeing. It also gives us time after the release to GA to focus on the error budget.
See https://gitlab.com/gitlab-org/error-budget-reports/-/issues/28#note_1596852671
Author and Reviewer Checklist
Please verify the check list and ensure to tick them off before the MR is merged.
-
Provided a concise title for this Merge Request (MR) -
Added a description to this MR explaining the reasons for the proposed change, per say why, not just what - Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
-
Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI) - If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
Maintained by
section on the page being edited - If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
- The when to get approval handbook section explains the workflow in more detail
- If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
-
For transparency, share this MR with the audience that will be impacted. -
Team: For changes that affect your direct team, share in your group Slack channel -
Department: If the update affects your department, share the MR in your department Slack channel -
Company: If the update affects all (or the majority of) GitLab team members, post an update in #whats-happening-at-gitlab linking to this MR - For high-priority company-wide announcements work with the internal communications team to post the update in #company-fyi and align on a plan to circulate in additional channels like the "While You Were Iterating" Newsletter
-