Add Vector to Devkit
Problem
We're going to need Vector to support Clickhouse Cloud in production to get Events from Kafka into the Cluster since Clickhouse's Kafka Engine is not supported there. Only doing this for the clickhouse cloud setup would introduce two different ways on how events can get from Kafka to Clickhouse:
- As a pull system via the Kafka Engine in Clickhouse
- As a push system via Vector
This would create increased maintenance effort and make the production setup very different from the local setup and non-prod clusters as well as create separate table setups via the Configurator.
In https://gitlab.com/gitlab-org/analytics-section/analytics-configurator/-/merge_requests/23#note_1546453516 we've started to discuss standardizing on Vector as part of our Infrastructure.
Desired Outcome
Vector is Setup in the configurator and sends events from Clickhouse to Kafka based on the table setup introduced in https://gitlab.com/gitlab-org/analytics-section/analytics-configurator/-/merge_requests/23
Possible Solution
TBD