Run `*-as-if-foss` jobs on master only

Opportunity

rspec * as-if-foss jobs cost $15,139.61 for March 2020 (2211.11+1373.70+6052.90+5501.90).

Detailed Analysis

*-as-if-foss jobs are expensive and may have a situation where adding the jobs manually may be more cost effective. Based on my initial analysis it looks like the costs of the rspec as-if-foss jobs for March were $18,525.30. Over the last 30 days there have been 1,638 of 674,062 builds (0.24%) where the associated *-as-if-foss job failed and the non foss job succeeded.

I know the *-as-if-foss jobs safeguard against foss failures but perhaps there's a solution we can consider to reduce the frequency of these jobs.

• Could these be manual for merge requests?
  • Could there be a job in the test stage which "plays" a subset of the foss jobs depending on which files are changed?
• Is there some configuration of changes: that can be used to reduce the frequency of always jobs for foss?

Proposal

• Run rspec *-as-if-foss jobs for gitlab-org/gitlab master and gitlab-org/security/gitlab MRs.rspec * -as-if-foss jobs should be manual in gitlab-org/gitlab MRs

Expected outcomes

• Reduce cost per pipeline KPI to $1.90
• Decrease gitlab-org/gitlab pipeline costs by $10,000 per month
• Decrease in the master pipeline success rate due to foss errors being acceptable master failures

Future iteration

• If foss error rate is high further experiment with using changes or a component to selectively execute some -as-if-foss jobs based on the files that are changed.