Consider running `rspec-all frontend_fixture as-if-foss` when feature flag change is introduced in an MR
Summary
This is a follow-up issue from gitlab-org/quality/engineering-productivity/master-broken-incidents#64 (comment 1162967942).
Quoting comments from Root Cause Analysis of this master FOSS broken incident:
The root cause is that the
cube_api_proxy
feature flag was moved to EE in !103141 (merged), but this feature flag was still referenced in CE codebase.
I think we can catch this incident even if we only run
rspec-all frontend_fixture as-if-foss
instead of allas-if-foss
jobs since the belowInvalidFeatureFlagError
showed up in all failed jobs, such as rspec system pg12 20/28, rspec system pg12 18/28, and rspec-all frontend_fixture 3/5.Failures: 1) Gcp Cluster when third party offers are disabled user does not see the offer Got 0 failures and 2 other errors: 1.1) Failure/Error: raise InvalidFeatureFlagError, "Missing feature definition for `#{key}`" unless type_definition[:optional] Feature::InvalidFeatureFlagError: Missing feature definition for `cube_api_proxy` # ./lib/feature/definition.rb:125:in `valid_usage!' # ./lib/feature.rb:86:in `enabled?' # ./spec/support/helpers/stubbed_feature.rb:33:in `enabled?' # ./app/views/admin/application_settings/general.html.haml:122:in `_app_views_admin_application_settings_general_html_haml___592075237452215480_987520'
Proposal
We should consider running rspec-all frontend_fixture as-if-foss
to catch similar InvalidFeatureFlagError
in MR pipelines instead of FOSS master pipelines later in the future. For example, @godfat-gitlab
suggested gitlab-org/quality/engineering-productivity/master-broken-incidents#64 (comment 1169114905):
Given that we have
if-merge-request-labels-frontend-and-feature-flag
I think we can add those to rules forfrontend_fixture
just like we did for jest. What do you think?