Update BatchCounter to stop counting at finish value [RUN AS-IF-FOSS]
What does this MR do?
Given 7181
records (with id
between 1
and 7181
) in the issues
table:
[4] pry(main)> Issue.count
(2.2ms) SELECT COUNT(*) FROM "issues"
=> 7181
[5] pry(main)> Issue.first.id
Issue Load (1.0ms) SELECT "issues".* FROM "issues" ORDER BY "issues"."id" ASC LIMIT 1
=> 1
[6] pry(main)> Issue.last.id
Issue Load (0.7ms) SELECT "issues".* FROM "issues" ORDER BY "issues"."id" DESC LIMIT 1
=> 7181
Before
When we ask for a count of records with id
between 1
and 7000
, we get a larger result (7181
). This is because BatchCounter
counts the entire final batch (1333
records up to 7999
), which goes past the finish
value.
[1] pry(main)> Gitlab::Database::BatchCount.batch_count(Issue, batch_size: 1333, start: 1, finish: 7000)
(2.8ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 1 AND "issues"."id" < 1334
(2.2ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 1334 AND "issues"."id" < 2667
(1.7ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 2667 AND "issues"."id" < 4000
(1.4ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 4000 AND "issues"."id" < 5333
(1.4ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 5333 AND "issues"."id" < 6666
(1.3ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 6666 AND "issues"."id" < 7999
=> 7181
After
With the adjustments in this MR, a count of records between 1
and 7000
gives us the correct result (7000
). The final batch is adjusted so that it does not query past the finish
value.
[3] pry(main)> Gitlab::Database::BatchCount.batch_count(Issue, batch_size: 1333, start: 1, finish: 7000)
(0.8ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 1 AND "issues"."id" < 1334
(0.8ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 1334 AND "issues"."id" < 2667
(1.4ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 2667 AND "issues"."id" < 4000
(1.0ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 4000 AND "issues"."id" < 5333
(0.8ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 5333 AND "issues"."id" < 6666
(0.6ms) SELECT COUNT("issues"."id") FROM "issues" WHERE "issues"."id" >= 6666 AND "issues"."id" < 7001
=> 7000
Does this MR meet the acceptance criteria?
Conformity
- [-] Changelog entry
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides - [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Alishan Ladhani