Improve usage data tests
What does this MR do?
Improves the efficiency of the test suite by reducing creation of database objects.
Specifically
- Reduce database object creation to minimal for suite.
- Refactor into factories and helpers to help test readability.
- Utilize
shared_examples
to better leverage standard rspec constructs instead of loops.
Factory Doctor Analysis
- Analyzed locally using this command
FDOC=1 be rspec spec/lib/gitlab/usage_data_spec.rb
Potential time improvements
- Time wasted ~11s
- Total spec runtime ~44s
initial output
Total (potentially) bad examples: 22
Total wasted time: 00:11.122
Gitlab::UsageData (./spec/lib/gitlab/usage_data_spec.rb:5) (198 records created, 00:11.122)
is expected to all respond to #totals (./spec/lib/gitlab/usage_data_spec.rb:278) – 9 records created, 00:00.527
does not have any conflicts (./spec/lib/gitlab/usage_data_spec.rb:287) – 9 records created, 00:00.516
is expected to all be a kind of Hash (./spec/lib/gitlab/usage_data_spec.rb:283) – 9 records created, 00:00.495
is expected to all have attributes {:keys => (all be a kind of Symbol), :values => (all be a kind of Integer)} (./spec/lib/gitlab/usage_data_spec.rb:284) – 9 records created, 00:00.396
gathers feature usage data (./spec/lib/gitlab/usage_data_spec.rb:297) – 9 records created, 00:00.508
gathers components usage data (./spec/lib/gitlab/usage_data_spec.rb:314) – 9 records created, 00:00.502
works when queries time out in new (./spec/lib/gitlab/usage_data_spec.rb:330) – 9 records created, 00:00.504
works when queries time out in to_json (./spec/lib/gitlab/usage_data_spec.rb:337) – 9 records created, 00:00.709
gathers variable data (./spec/lib/gitlab/usage_data_spec.rb:348) – 9 records created, 00:00.834
returns the count when counting succeeds (./spec/lib/gitlab/usage_data_spec.rb:378) – 9 records created, 00:00.402
returns the fallback value when counting fails (./spec/lib/gitlab/usage_data_spec.rb:384) – 9 records created, 00:00.496
is expected to all respond to #totals (./spec/lib/gitlab/usage_data_spec.rb:278) – 9 records created, 00:00.494
does not have any conflicts (./spec/lib/gitlab/usage_data_spec.rb:287) – 9 records created, 00:00.395
is expected to all be a kind of Hash (./spec/lib/gitlab/usage_data_spec.rb:283) – 9 records created, 00:00.478
is expected to all have attributes {:keys => (all be a kind of Symbol), :values => (all be a kind of Integer)} (./spec/lib/gitlab/usage_data_spec.rb:284) – 9 records created, 00:00.499
gathers feature usage data (./spec/lib/gitlab/usage_data_spec.rb:297) – 9 records created, 00:00.469
gathers components usage data (./spec/lib/gitlab/usage_data_spec.rb:314) – 9 records created, 00:00.414
works when queries time out in new (./spec/lib/gitlab/usage_data_spec.rb:330) – 9 records created, 00:00.499
works when queries time out in to_json (./spec/lib/gitlab/usage_data_spec.rb:337) – 9 records created, 00:00.488
gathers variable data (./spec/lib/gitlab/usage_data_spec.rb:348) – 9 records created, 00:00.464
returns the count when counting succeeds (./spec/lib/gitlab/usage_data_spec.rb:378) – 9 records created, 00:00.494
returns the fallback value when counting fails (./spec/lib/gitlab/usage_data_spec.rb:384) – 9 records created, 00:00.528
Process
Performance centric
- Move individual tests out of the data creation loop.
- Notice speed improvements by moving those out of data creation loop.
- Move record creation to a factory.
- Move the record creation out of
before
and into alet!
to reduce data creation for each block. - Move
ProjectFeature.first.update_attribute('repository_access_level', 0)
into factory as well to reduce factory recreation on attribute changes. - Ensured Danger rules for ~"group::telemetry" new files are honored https://gitlab.com/gitlab-org/gitlab/-/blob/36a5f50e566594c1ee94efff2b58760a91c871de/danger/telemetry/Dangerfile#L9
Realized greater than 11s time reduction
from initial runtime of ~44s and current runtime of ~31s.
- Factory Doctor output
Finished in 31.18 seconds (files took 35.65 seconds to load)
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by Doug Stull