Skip to content

Pipeline Graph Structural Update

Release Notes

description: We have updated our Pipeline Graph to support better performance and maintainability by introducing a new regular pipeline graph layout. You can now easily visualize your pipelines, stay tuned for dependency line updates in a future release to support :needs in your pipelines.

documentation_link: https://docs.gitlab.com/ee/ci/pipelines/#regular-pipeline-graphs

image_url:

Graph behavior: This contains needs links when hovering over jobs which was removed from the final version gif

Overview & Context

Just to remind us all, this work has been undertaken to support our goal of showing DAG views in the main pipeline graph. (&4509 (closed)) This has been a long time coming: the current DAG view was an MVC on making sure we could show the relationships and exploring what users would want to see when they see these.

Part of the reason this addition has been so difficult is that the current pipeline graph is showing its age a bit. (I hope our new version is lucky enough to become this much of a legacy.) It is rather brittle, both in terms of the layout and in terms of data handling. It uses a homegrown mediator–store combination and our homemade polling tech. The link display does not make sense any more given a DAG. This debt needed to be addressed in order to add the new-style layout in a maintainable way.

Plan

Taken from the associated spike (#249094 (comment 439139145))

See detailed steps working list

Updating the data fetching to use GraphQL

The basic version of this code is working (!46353 (closed)), and it needs now to be broken into reviewable MRs with production-quality code. These will likely be

  1. ✅ Add the feature flag and plumbing: !46380 (merged)
  2. ✅ Refactor components to prepare for GraphQL: !47772 (merged)

--------- I GOT HERE BY THE END OF 13.6 ---------------

  1. ✅ Add the GraphQL version (this will likely end up being ~7 MRs) ➡ !48330 (merged), !48339 (merged), !48498 (merged), !48558 (merged), gitlab-ui!1872 (merged), !48779 (merged), !49164 (merged), !49825 (merged) (◀ this is the last one)

Updating the layout to support shared architecture and new link style

  1. ✅ Create production-level template components for the layout and stream wrapper — this involves making the code in the sf-graphql-to-pipeline-graph branch production ready and adding tests and things, and can be done independently of the previous steps (this will probably be rolled into the MRs for the graph itself): ➡ !48498 (merged), !49164 (merged)

--------- I GOT HERE BY THE END OF 13.7 ---------------

  1. ✅ Create production-level template component for the links — this is similar to the above, but involves a bit more work to normalize how we are shaping the link data ➡ !51594 (merged)

  2. ✅ Apply to the pipeline graph ➡ !51594 (merged), !51688 (merged)

  3. ✅ Do UX review to refine interactions and confirm view: ➡ !51688 (merged)

Unexpected Occurrences

  1. ✅ Darkmode CSS Pass — review and be sure we are not breaking darkmode in the new code (spoiler: we probably are). ➡ !51688 (merged)

--------- I GOT HERE BY THE END OF 13.8 ---------------

Performance Bonus

  1. ✅ Add initial query to startup.js for bonus speed. !51899 (merged)

Rolling Out the Changes

  1. ✅ Update tests #293638 (closed)

  2. Add the rollout instrumentation
    a. ✅ API: #281351 (closed), !52820 (merged), !53902 (merged)
    b. ✅ Sentry: #294028 (closed), !54441 (merged)

--------- I GOT HERE BY THE END OF 13.9 ---------------

Unexpected Occurrences, Pt. 2

  1. ✅ ⚠ Resolving the Long Polling Dilemma: #292797 (closed) ⚠ This issue is a blocker for rollout.

    c. Visual Testing: #294019

  2. Create rollout issue: #299112 (closed)

  3. Do the rollout

--------- I EXPECT TO GET HERE BY THE END OF 13.10 ---------------

Updating the layout to show the nodes arranged by the number of previous jobs (the DAG view)

This is the final goal and can be done in 1–2 MRs, once everything else has been taken care of. That's the finish line! &4509 (closed)

Followup MRs for after rollout (not in any order)
  1. Link interaction and perf refinements: #296971 (closed)
  2. Better job references that do not rely on names: https://gitlab.com/gitlab-org/gitlab/-/issues/296982https://gitlab.com/gitlab-org/gitlab/-/issues/296982
  3. Apply LinksLayer to Pipeline Editor Component: #296984 (closed)
  4. Code cleanup after 100% rollout: #291043 (closed)
  5. Keyboard navigation: #267117
  6. Hover on parallel jobs: #267116 (closed)
  7. Unify Apollo Cache with header, so we fetch just once: #299075 (closed), !56697 (merged)

Further details

This is expected to roll over a few milestones. Finding iterative, shippable solutions here is hard since we're essentially replacing a large body of work and don't want any loss of core features.

Documentation

Update the screenshot and adding a > - [Visualization improved](issue/epic link) in GitLab 13.X when it's enabled by default for everyone

Availability & Testing

Tests have been updated as we go to cover the same cases as legacy tests.

  1. ✅ Update tests #293638 (closed)

  2. Add the rollout instrumentation
    a. ✅ API: #281351 (closed), !52820 (merged), !53902 (merged)
    b. ✅ Sentry: #294028 (closed), !54441 (merged)
    c. Visual Testing: #294019

What does success look like, and how can we measure that?

Success is things not breaking. Very simple. This will be measured via Sentry reports, plus the performance testing planned in #281351 (closed).

Is this a cross-stage feature?

This affects grouppipeline authoring and ~"group::continuous integration".

Links / references

Edited by Sam Beckham