Migration to add project and issue permissions to issues in index
What does this MR do?
Related to #273234 (closed)
Part of an overall larger effort to migration issues documents to a new index in Elasticsearch. This MR contains a new migration which will back fill the issues_access_level
and visibility_level
fields on all issues documents within the current index. Added specs for the new migration options class and the migration itself.
Note: Only checking for issues missing visibility_level
to determine if the issue needs to be re-indexed, due to the order that the previous MRs were merged, there could be issues in the index with issues_access_level
that would be incorrect
To support this work, a few changes to the Elastic::MigrationWorker
were required:
- adding
MigrationOptions
class and instance methods to allow setting options used by the migration worker. Currently two options are supported:batched!
andthrottle_delay
. The throttle_delay is defaulted to 5 minutes - support for running batched migrations, the worker will re-enqueue itself after running a migration set to use
batched
mode - updated documentation on how to create migrations for Elasticsearch - the new migration options are described
Max time estimated to complete: 84 hours for GitLab.com
- ~5,000,000 issues in GitLab.com
- run in batches of 5,000 every 5 minutes
- complete 60,000 an hour
Not user facing so I did not add a changelog
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
- [-] Changelog entry
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides - [-] Database guides
-
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Related to #273234 (closed)