Update Data Team workflow stages and summary

Why is this change being made?

The current workflow has several pain points:

• Issues move directly from Validation (stage 2) to Scheduling (stage 3), but there is not enough rigor in defining the business value/scope/level of effort during the Validation stage. As a result, Scheduling and Scheduled statuses include several hundred issues that do not have enough detail to actually develop against.
• We don't have an effective way to flag issues that have been fully refined (a subset of the backlog that has clear development requirements & estimated level of effort for a developer to pull from if capacity is available)
• We don't have clear guidance on the criteria or process to "Won't Do" an issue (i.e. almost every issue that ever gets opened sits in Validation in perpetuity)

Proposing the following changes:

• Replace the 3 - Scheduling stage with 3 - Refinement.
  • The existing Scheduling stage is low value, because our Validation process does not provide sufficient details required to schedule an issue (example: according to this board we have ~275 issues in Scheduling across the GitLab Data Team project, although we can't reasonably be working on scheduling that many issues at a given point in time), so we end up with issues in this stage that are actually not ready to be scheduled.
  • I suggest leaving the Validation stage as-is, and use it to house the "long tail" of requests (valid requests even if we don't expect them to be worked on in the near future). We can distinguish between the "long tail" vs the ones that are being actively prioritized & scoped because the latter will progress to Refinement.
  • To clarify the difference between Validation vs. Refinement - the exit criteria for Validation is a Do / Won't Do decision, whereas the exit criteria for Refinement is a clear technical solution, scope, and issue weight.
  • Leaving the existing Triage and Validation stages in place will also be least disruptive for Data Team members (minimizing changes).
• Replace the 4 - Scheduled stage with 4 - Ready to Develop.
  • The existing Scheduled stage is low value because we already have a separate field for Milestone, so we can clearly see whether an issue has been scheduled or not. (The above board shows 60 issues in Scheduled, of which 30 don't have a Milestone, so we are not adhering to the definition).
  • The proposed Ready to Develop stage would be used during Milestone Planning. We would filter to this list to ensure that we are only pulling issues into a milestone when refinement has been fully completed. This would help us avoid ambiguity and rework, and in the event that there is slack capacity / a new developer onboards, we would have a clear list of items with clear descriptions & level of efforts to pull from.

Old workflow:

graph LR
A[Triage] --> B[Validation]
B --> C[Scheduling]
C --> D[Scheduled]
D --> E[Development]
E --> F[Review]
F --> G{Close}

Proposed workflow:

graph LR
A[Triage] --> B[Validation]
B --> H{Won't Do & Close}
B --> C[Refinement]
C --> D[Ready to Develop]
D --> E[Development]
E --> F[Review]
F --> G{Complete & Close}

Future Iterations:

• We should update the workflow stages to attach SLAs to them (giving stakeholders a reasonable estimate for the amount of time to expect responses). I expect this will require us to review our triage/project intake process as well
• One major planning painpoint we have is the ability to tag issues by Quarter. Looking at the documentation and handbook pages, it looks like the Milestone field is intended to be used for those longer time periods/project milestones, whereas the Iterations is a more sprint-like biweekly interval. In a future iteration of our workflow statuses I would like to review the usage of both of these fields. We are piloting it on a small scale on the Tableau migration project.

Author Checklist

• Provided a concise title for this Merge Request (MR)
• Added a description to this MR explaining the reasons for the proposed change, per say why, not just what
  • Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
• Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI)
  • If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the Maintained by section on the page being edited
  • If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
  • The when to get approval handbook section explains the workflow in more detail
• If the changes affect team members, or warrant an announcement in another way, please consider posting an update in #whats-happening-at-gitlab linking to this MR
  • If this is a change that directly impacts the majority of global team members, it should be a candidate for #company-fyi. Please work with internal communications and check the handbook for examples.

---