GitHub & Gitea importer filtering
What does this MR do?
This MR adds filtering functionality to GitHub & Gitea importers. Please see the gif below.
Note on filtering being done on our side: Octokit client does not provide an easy way of filtering all user's repositories (see https://octokit.github.io/octokit.rb/Octokit/Client/Repositories.html#repositories-instance_method & https://developer.github.com/v3/repos/#list-your-repositories).
Instead, GitHub provides Search API and has search_repositories
functionality (see https://octokit.github.io/octokit.rb/Octokit/Client/Search.html#search_repositories-instance_method). However, this is not exactly what is needed, since search is performed on the entire GitHub. It is possible to scope down this search to user/organisation, but I encountered 2 problems with it:
- We need to know if provided access token belongs to user or organisation. I could not find this functionality available in Octokit client (this is probably the biggest showstopper from using search API)
- When scoping by user I was getting unexpected results. Results did not contain repositories I was a collaborator in, which is different to what we currently return (
client.repos
returns both your repos and all repos you are collaborator in)
Since currently we retrieve all repositories from GitHub anyway, I decided to go with least traction solution of doing filtering after fetching all repos from GitHub.
This MR originally only targeted GitHub, but frontend and backend code is shared between GitHub and Gitea, so I added a few changes to support both importers.
EE Port: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/16185
Screenshots
Before
After
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry for user-facing changes, or community contribution. Check the link for other scenarios. -
Documentation created/updated or follow-up review issue created -
Code review guidelines - [-] Merge request performance guidelines
-
Style guides - [-] Database guides
- [-] Separation of EE specific content