Skip to content

Add cronjob to sync finished pipelines to ClickHouse

Pedro Pombeiro requested to merge pedropombeiro/470076/implement-ch-worker into master
Sequence MR
1 Create p_ci_finished_pipeline_ch_sync_events table (gitlab!158060 - merged)
2 Implement service to sync pipelines to ClickHouse (gitlab!158362 - merged)
3 Introduce worker to import finished pipelines t... (gitlab!159083 - merged)
4 you are here !7783 (merged)
5 gitlab-org/charts/gitlab!3839 (merged)

What does this MR do?

This MR follows up on gitlab!159083 (merged) to add the cronjob to sync finished builds to ClickHouse to the omnibus package. For context, this job is only supposed to run on .com for the time being. We're working on a path for self-managed users to leverage a ClickHouse database in their installs.

There's no changelog trailer since this feature is behind a feature flag.

Related issues

Create worker to sync finished pipelines to Cli... (gitlab#470076 - closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes. This is simply adding a new worker to the configuration
  • Documentation created/updated. There is no documentation to update here, AFAIK
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened. I don't think we need to add one, since this is only meant for GitLab.com for now, similar to !7169 (merged)
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Clemens Beck

Merge request reports

Loading