[meta] SubGit Integration
Current State
After talking to @pmachle it seems that with the new changes to our Enterprise pricing adding SubGit as a reseller would be a little too much and will doom the engagement to fail. As a result, we're going to sideline the reseller engagement and refer enterprise SVN clients to SubGit to purchase directly from them and utilize their GitLab integration. If we'll see good pick up from new clients through this path we'll likely get back to it at a later stage were the sales and marketing teams will have additional bandwidth to support reselling the integration.
Goal
SubGit will integrate using our API for us to resell their product and service to our Enterprise clients. This is an easier first step to gauge demand for this integration and make closing deals smoother. If there's a positive pick up for this integration we will later look into shipping SubGit with EE. For this we will need to sign a reseller agreement.
License purchasing process
- Customer approaches GitLab sales team in order to purchase SubGit as an addition to their GitLab installation.
- GitLab sales team sends a request to SubGit license server with details of the customer inquiry: company name, license expiration date and GitLab users limit. This license server can be implemented as a REST server or, alternatively, GitLab sales team can contact our sales via email.
- Our license server sends SubGit license key file in response.
- As soon as customer sends the payment, GitLab sales team sends the license key back to customer along with instructions on how to register GitLab repositories with SubGit license key.
- Finally, customer uploads the license key to the server and registers all Git repositories with this key by running the following command: subgit register --key subgit.key /path/to/repo.git
License upgrading process
- When GitLab customer upgrades GitLab license (increases the user limit for GitLab application), SubGit warns customer to upgrade SubGit license as well.
- Customer approaches GitLab sales team in order to purchase SubGit license upgrade and the process goes on just as in the case of initial purchase.
SubGit license check
- When one runs subgit register --key subgit.key /path/to/repo.git, SubGit does the following:
- First, it discovers the port GitLab is working on: SubGit parses the output of netstat -atnup in order to find the GitLab process name. This only works when running on behalf of 'root' or 'git' user (netstat does not display processes names otherwise). However, this is not a major issue because SubGit has to be run on behalf of 'git' user anyway.
- Then SubGit looks into GitLab database by issuing the command below: gitlab-psql -d gitlabhq_production "select authentication_token from users"
- This way SubGit gets a list of private tokens. This command requires root access, which means subgit register ... has to be run on behalf of root.
- Having a list of private tokens, subgit register command caches one of them to be used later on without root privileges. Current implementation stores that private token in a plain text. We may consider encrypting that private token with some secret key in further versions.
- Finally, subgit register sends REST request in order to fetch the license details of GitLab application: curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://localhost/api/v3/license" And so SubGit is able to check whether the license key is valid for a given installation of GitLab.
- Periodically SubGit has to check whether the license key is still valid:
- Discover GitLab port by parsing netstat -atnup.
- Use cached private token in order to send REST request to GitLab server: curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://localhost/api/v3/license"
- Compare received users limit with the one specified in SubGit license key and raise the warning if GitLab license was upgraded or expired.
Original Issue
We'd like to integrate SubGit tool into GitLab to provide users with a smooth SVN to Git migration path.
This issue will describe changes to GitLab this integration will require.
Integration tasks overview:
- Deploy SubGit binaries (requires JRE) along with GitLab:
- Download as part of container set up, along with JRE dependency
- Or download from within GitLab instance
- Or ask user to install SubGit along with JRE
- Run "subgit sync" on any ref update (push, pr merge, branch/tag manipulation) after all the validation (authorization, etc) has been performed by GitLab, but before actual ref update (pre-receive hook equivalent)
- Run "subgit fetch" on mirror repositories periodically
- Per-repository configuration page/wizard that let user edit mirror configuration, run "subgit install" and see that command results and mirror status.
Integration Parts
-
SubGit binaries and JRE availability to GitLab process
- Question: would it be possible to modify GitLab Dockerfile to include SubGit binaries and JRE?
-
Hooks interception: support for "sync" hook to be ran after all pre-receive hooks succeeded on ref change.
- Question: how to intercept all refs updates (both those from user's push (git-shell) and those from modification via GitLab UI, like merge)? What is the best place to do that?
-
Mirror tasks queue and status tracker:
- Reports current mirror status (per repository), status of a running or completed task
- Schedules user task (configure, enable, disable mirror)
- Schedules periodic task (sync)
- Question: should we use Sidekiq and workers?
- Question: how to add periodic task at start up time?
-
Mirror status page
- Add another tab (SVN Mirror) to the project tabs:
- Add rails view and controller (svn)
- Include "svn" view into app/controllers/projects_controller.rb
- Javascript code goes to app/assets/javascripts
- Question: what is the GitLab approach to display progress bar with percentage for a long running background tasks?
- Question: is there standard support for dynamically updated pages (with a background persistent connection or periodic polling)?
- Add another tab (SVN Mirror) to the project tabs:
-
SubGit License: * Add global setting page where user could enter SubGit registration key (a few lines of text) * Question: where to add global settings page for that purpose? * Question: how to read GitLab license (edition, number of users)? Is it possible to do that from external process?