Add internal API to retrieve LFS objects
This commit is needed to support the inclusion of LFS files inside a project archive download. Gitaly will access this internal API call to download the file via a custom LFS smudge filter (gitaly!2577 (merged)).
To offload the transfer from the Web application server, we send local files via the Rack Sendfile middleware.
For files stored on object storage, we rely on the Workhorse Send-Data header to make Workhorse retrieve the pre-signed URL.
Part of #15079 (closed)
Data flow:
sequenceDiagram
Client->>+Workhorse: GET /group/project/-/archive/master.zip
Workhorse->>+Rails: GET /group/project/-/archive/master.zip
Rails->>+Workhorse: Gitlab-Workhorse-Send-Data git-archive
Workhorse->>Gitaly: SendArchiveRequest
Gitaly->>Git: git archive master
Git->>Smudge: OID 12345
Smudge->>+Workhorse: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
Workhorse->>+Rails: GET /internal/api/v4/lfs?oid=12345&gl_repository=project-1234
Rails->>+Workhorse: Gitlab-Workhorse-Send-Data send-url
Workhorse->>Smudge: <LFS data>
Smudge->>Git: <LFS data>
Git->>Gitaly: <streamed data>
Gitaly->>Workhorse: <streamed data>
Workhorse->>Client: master.zip
Edited by Stan Hu