GitHub importer: Fallback to DB read when `Gitlab::GithubImport::IssuableFinder` has a cache miss (improve chances of importing all data)
About
When GitHub issues and PRs are imported, we cache the id
of the record that was created using Gitlab::GithubImport::IssuableFinder#cache_database_id
here and here.
The value is later retrieved through #database_id
when we import subrelations of the issue or MR (like notes, label links, etc.).
Problem
If the #database_id
failed to be set for some reason (!133149 (merged)) or the cache key has expired (possible if the import is very large), then all subrelations that rely on that value being retrieved will fail to save.
Proposal
If there is cache miss in #database_id
, we should attempt to retrieve the issuable from PostgreSQL, and then write it to the cache. We can use the values of #cache_key_type
, #cache_key_iid
and the project
in the class to look it up.
When we read from the DB and we find that there is no DB record, we should write a -1 similar to the BitbucketImport::UserFinder
to signal that there is no DB record. When we retrieve a -1 from the cache in #database_id
we would then return a nil
rather than trying to fetch from the DB something we know doesn't exist.