[217925] LFS import in batches

What does this MR do and why?

Bug fix for Mirrored project with LFS files returns Object does not exist on the server or you don't have permission to access it.. There were cases of missed LFS objects during mirroring. This MR does two things:

• Fetches links and downloads object per batches to avoid links expiration.
• Adding retry in case of request timeout.

Hint for reviewers

All the classes inside this MR were modified (not created). From my POV there are SRP issues what leads to such confusion starting with naming. Unfortunately there is no chance to perform so big refactoring in the scope of this task.

flowchart TB

LfsImportService#execute --> LfsObjectDownloadListService#each_list_item
LfsObjectDownloadListService#each_list_item --> LfsDownloadLinkListService#each_link
LfsDownloadLinkListService#each_link -->|lfs object link| LfsObjectDownloadListService#each_list_item
LfsObjectDownloadListService#each_list_item -->|lfs object link| LfsDownloadService#execute

Importer	How LFS are imported
GitHub	Uses Projects::LfsPointers::LfsObjectDownloadListService, which was modified by this MR
BitbucketServer	Uses Projects::LfsPointers::LfsImportService#execute, which the downstream code was modified by this MR
GitlabProject	Imports LFS objects contained in the imported .tar.gz using a different method
Projects::ImportService	Uses Projects::LfsPointers::LfsImportService#execute, which the downstream code was modified by this MR

Screenshots or screen recordings

Click to expand

Before:

After:

How to set up and validate locally

1. Mirror https://github.com/Unity-Technologies/Graphics
2. Clone it on to your local machine
3. Run git lfs fetch --all

Warning: Repository is huge. My PC needed about 4-5 hours to make this E2E test.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.