Use author name snapshot on audit log export
What does this MR do?
Preloading of author name on users table via BatchLoader is still causing performance issue on Audit Event export to CSV. Testing on staging with blanket search (~ 15mb) is still timing out (> 60s).
This MR uses author name snapshot on audit_event and stop preloading via Users association.
Profile
Profiling was carried out on local env with ~ 1,000 of audit events.
Before
Allocations 22,355,154
Measure Mode: allocations
Thread ID: 23640
Fiber ID: 86800
Total: 22355154.000000
Sort by: self_time
%self total self wait child calls name location
24.07 7888299.000 5380332.000 314.000 2507653.000 5269992 *Class#new
9.72 2173263.000 2173152.000 111.000 0.000 724384 String#sub
7.33 3752838.000 1638765.000 151.000 2113922.000 95000 Psych::Parser#parse
3.61 806937.000 806937.000 0.000 0.000 285 PG::Result#values
3.24 3087814.000 724477.000 0.000 2363337.000 1448768 Psych::ScalarScanner#tokenize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/scalar_scanner.rb:31
3.22 719812.000 719812.000 0.000 0.000 1137924 CsvBuilder#excel_sanitize /Users/tanle/Code/gitlab/gitlab-development-kit/gitlab/lib/csv_builder.rb:115
3.08 688251.000 688251.000 0.000 0.000 688251 Integer#to_s
2.55 2564466.000 569654.000 0.000 1994812.000 284827 BatchLoader#batch /Users/tanle/.gem/ruby/2.7.2/gems/batch-loader-1.4.0/lib/batch_loader.rb:26
2.55 569654.000 569654.000 0.000 0.000 284827 BatchLoader#__singleton_class /Users/tanle/.gem/ruby/2.7.2/gems/batch-loader-1.4.0/lib/batch_loader.rb:116
2.35 7395519.000 526284.000 0.000 6869235.000 329405 *Array#map
* recursively called methodsTime 194.004699 secs
Measure Mode: wall_time
Thread ID: 23640
Fiber ID: 86800
Total: 194.004699
Sort by: self_time
%self total self wait child calls name location
5.45 26.813 10.575 0.002 16.236 5271753 *Class#new
5.17 22.671 10.024 0.000 12.647 1448768 Psych::ScalarScanner#tokenize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/scalar_scanner.rb:31
3.18 191.409 6.170 0.000 185.239 467162 *Array#each
2.71 5.256 5.254 0.002 0.000 8776800 String#match?
2.59 26.604 5.018 0.000 21.585 95000 Psych::Parser#parse
2.56 47.918 4.976 0.000 42.943 1638768 *Psych::Visitors::Visitor#visit /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/visitors/visitor.rb:15
2.48 61.727 4.818 0.000 56.909 329901 *Array#map
2.29 10.490 4.446 0.001 6.043 1072274 ActiveModel::LazyAttributeHash#assign_default_value /Users/tanle/.gem/ruby/2.7.2/gems/activemodel-6.0.3.3/li
b/active_model/attribute_set/builder.rb:107
2.05 3.972 3.972 0.000 0.000 1448768 Psych::Nodes::Scalar#initialize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/nodes/scalar.rb:58
* recursively called methodsAfter
Allocations 18,766,837
Measure Mode: allocations
Thread ID: 23640
Fiber ID: 86800
Total: 18766837.000000
Sort by: self_time
%self total self wait child calls name location
25.85 6846125.000 4851845.000 175.000 1994105.000 4753511 *Class#new
12.00 2252426.000 2252410.000 6.000 10.000 750806 String#sub
9.05 3885321.000 1697567.000 75.000 2187679.000 98000 Psych::Parser#parse
4.33 812531.000 812531.000 0.000 0.000 100 PG::Result#values
4.00 3199383.000 750862.000 43.000 2448478.000 1501568 Psych::ScalarScanner#tokenize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/scalar_scanner.rb:31
3.77 708012.000 708012.000 0.000 0.000 1166280 CsvBuilder#excel_sanitize /Users/tanle/Code/gitlab/gitlab-development-kit/gitlab/lib/csv_builder.rb:115
3.73 700111.000 700111.000 0.000 0.000 700111 Integer#to_s
2.61 6222681.000 489516.000 14.000 5733151.000 294309 *Array#map
2.07 388940.000 388940.000 0.000 0.000 77788 MatchData#captures
* recursively called methodsTime 180.422471 secs
Measure Mode: wall_time
Thread ID: 23640
Fiber ID: 86800
Total: 180.422471
Sort by: self_time
%self total self wait child calls name location
6.08 25.378 10.979 0.001 14.398 1501568 Psych::ScalarScanner#tokenize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/scalar_scanner.rb:31
4.61 26.942 8.316 0.000 18.627 4752627 *Class#new
4.48 173.031 8.080 0.001 164.949 373267 *Array#each
3.82 6.886 6.886 0.000 0.000 1501568 Psych::Nodes::Scalar#initialize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/nodes/scalar.rb:58
3.26 5.887 5.886 0.001 0.000 9100400 String#match?
3.10 31.874 5.594 0.001 26.279 98000 Psych::Parser#parse
2.95 57.936 5.317 0.001 52.617 294269 Array#map
2.88 54.537 5.197 0.000 49.340 1697568 *Psych::Visitors::Visitor#visit /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/visitors/visitor.rb:15
2.69 11.690 4.860 0.000 6.830 1070710 ActiveModel::LazyAttributeHash#assign_default_value /Users/tanle/.gem/ruby/2.7.2/gems/activemodel-6.0.3.3/lib/active_model/attribute_set/builder.rb:107
2.41 32.315 4.348 0.000 27.966 1501568 Psych::Visitors::ToRuby#deserialize /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/visitors/to_ruby.rb:46
2.02 18.752 3.639 0.000 15.113 1501568 Psych::TreeBuilder#scalar /Users/tanle/.rubies/2.7.2/lib/ruby/2.7.0/psych/tree_builder.rb:96
* recursively called methodsDoes this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers - [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec - [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Relates to #246742 (closed)
Edited by Tan Le