Skip to content

Review Patroni MR and determine remaining work

Overview

As outlined in &2588 (closed), we will add support for using Patroni to handle PostgreSQL replication in Omnibus. For the duration of GitLab 13.x, users will have the option of using either repmgr or Patroni. If they are on PostgreSQL 12, they will need to use Patroni.

There has been an MR in progress for almost a year to add Patroni to Omnibus. It has been worked on by a number of different contributors. The first step for adding Patroni is to determine whether we should work from the existing MR.

Proposal

To implement this feature we recommend creating a new MR. Many changes from !3196 (closed) can be reused by adopting and refactoring them based on the current requirements, namely keeping repmgr and opt-in mechanism for Patroni. We can also improve the changes based on recent developments in GitLab.com infrastructure if there is any. As a result, !3196 (closed) can be closed and the work continues in the new MR and the ones that will come after it to satisfy non-functional and usability requirements.

The rationale for this approach can be justified by the following:

  • Rebasing !3196 (closed) with the latest master branch and resolving the conflicts is not straightforward. Starting a new MR, hand-picking and porting changes on the other hand is easier. It also provides an opportunity to refactor/improve/update the changes.
  • In !3196 (closed), the assumption about what controls PostgreSQL configuration was not explicit. We start the new MR assuming that Patroni is in charge of PostgreSQL configuration and management. We leave the possibility of changing this assumption open.
  • The non-functional requirements and usability were not the major concerns of !3196 (closed). These requirements will be addressed not only in the new MR (e.g. the choice between repmgr and Patroni) but also in a series of new MRs which cover documentation, management tools, and workflow verification.

In this new MR, the focus is to implement the feature, testing it in short-cycled feedback loops, and deciding on what controls PostgreSQL configuration.

After that we will continue our work on three different fronts:

  1. Testing and validation against PostgreSQL 11 and eventually PostgreSQL 12 both with and without Geo. Verifying switching from repmgr to Patroni and vice-versa
  2. patroni sub-command for gitlab-ctl which wraps patronictl and provides almost the same experience as gitlab-ctl repmgr.
  3. Documentation

Requirements

  • Review the MR !3196 (closed) to understand the changes it makes
  • In this issue, document what would need to be removed from the MR if we were to use it for adding Patroni support. For example, the MR description states that the MR removes repmgr. We don't want to remove repmgr until GitLab 14.0 (see &2588 (closed))
  • In this issue, document what the MR is missing (what would need to be added to meet the requirements outlined in &2588 (closed))
  • Compare the effort required to modify the existing MR to meet our needs, versus starting a new MR. Provide your best guess of the number of weeks required for each option.

Acceptance criteria

  • A "Proposal" section has been added to the description of this issue. It provides a recommendation for a) working from the existing MR, OR b) starting a new MR. It includes the reason for why that decision was made.
  • If the decision is to start a new MR, close !3196 (closed), leave a comment in the MR explaining why it has been closed, and add a link to the new MR/issue
  • If the decision is to start a new MR, note in the "Proposal" section of this issue whether any of the code from the existing MR can be used.
  • Issues have been created for the next steps. The issues have been added to &2588 (closed)
  • Mention @ljlane on the issues that are created for next steps so they can be prioritized and assigned a milestone
Edited by Hossein Pursultani