Skip to content

Importer: Investigate and fix failed relations

The import

Run big import again using rake task from 20724, and with enabled feature flag for failing gracefully & record failures.

Failed relations

  • We should check all recorded ImportFailures and investigate how many relations were not imported and why?

Results

Import was successfully imported, it took around 11h to import on local env.

  • Total imported merge_requests: 102319 (106 more then before)

  • ImportFailures: 17

[[1, 146, 76, 2019-11-28 12:03:08 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [2, 226, 76, 2019-11-28 12:07:32 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [3, 1033, 76, 2019-11-28 12:27:20 UTC, "merge_requests", "NoMethodError", nil, "undefined method `line_for_position' for nil:NilClass"],
 [4, 1195, 76, 2019-11-28 12:31:10 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [5, 1267, 76, 2019-11-28 12:32:22 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [6, 1742, 76, 2019-11-28 12:38:44 UTC, "merge_requests", "NoMethodError", nil, "undefined method `line_for_position' for nil:NilClass"],
 [7, 2076, 76, 2019-11-28 12:42:46 UTC, "merge_requests", "NoMethodError", nil, "undefined method `line_for_position' for nil:NilClass"],
 [8, 2896, 76, 2019-11-28 12:53:26 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [9, 3428, 76, 2019-11-28 12:59:56 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [10, 3580, 76, 2019-11-28 13:01:25 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [11, 3863, 76, 2019-11-28 13:04:35 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [12, 4306, 76, 2019-11-28 13:08:59 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [13, 4314, 76, 2019-11-28 13:09:08 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [14, 5111, 76, 2019-11-28 13:19:00 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [15, 5128, 76, 2019-11-28 13:19:10 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [16, 5505, 76, 2019-11-28 13:23:25 UTC, "merge_requests", "NoMethodError", nil, "undefined method `index' for nil:NilClass"],
 [17, 72071, 76, 2019-11-28 18:41:36 UTC, "merge_requests", "NoMethodError", nil, "undefined method `line_for_position' for nil:NilClass"]]

They are all related to creating DiffNote:

undefined method `index' for nil:NilClass

and

undefined method `line_for_position' for nil:NilClass"

  • Why is this happening?
  • Can we fix them?
  • If not, we might ask customer if not creating these relations will be fine with them.

The rake script finished with exception: GRPC::DeadlineExceeded: 4:Deadline Exceeded Looks like that after_import_service failed because of timeout on gitaly, due to sheer size of repo...

Note:

I have increased timeout to 50s in grpc keepalive params in gitaly, and after_import_service passed without error. So we might need to increase this timeout as well in order to import without errors.

Edited by 🤖 GitLab Bot 🤖