Allow only forward incremental deployments
The problem
As part of introducing Incremental Rollouts we hit a bunch of problems (described in https://gitlab.com/gitlab-org/gitlab-ee/issues/1660#note_100311728):
- If a new pipeline is pushed, the already scheduled delayed jobs gonna be executed, even though that they do incremental rollout of older pipelines,
- Currently, this can be avoided only by a user doing a specially crafted script to check that,
- This is a problem as we don't have a concept of fowrad deployments,
- Stop environment can be executed for past deployment on environment, where we should allow only the latest one.
The solution
However, with the introduction of Make deployments stateful we keep track of all future deployments, including their statuses. This makes it possible for us to solve the problem of concurrent deploys:
- When we try to run deployment, we verify if our deployment is latest one,
- We allow the deployment to
run
, thus job to bepending/running
when the last success/failed deployment sha is equal to ours, or our deployment is never than latest success/failed one, - We apply this rule to every CI job doing the action on environment.
The example
class Build
enum :failure_reason {
out_of_date_deployment_action: ...
}
def is_latest_deployment_on_environment?
environment = persisted_environment
return unless environment
last_deployment = environment.last_deployment
return true unless last_deployment
if starts_deployment?
# if our deployment is later than last successful deployment
# it should fullfil the rule of being a forward deployment
# if this is incremental deployment it will always be later in the pipeline
deployment.id >= last_deployment.id
elsif stops_deployment?
# if we want to stop, we require that stop action is for the same SHA
last_deployment.sha == sha
end
end
end
UX proposal
A new setting will be added under CI/CD settings -> General Pipelines called Drop older active deployment
as described below
Edited by Orit Golowinski