Skip to content

The 'Test' button for System hooks sends the X-GitLab-Event Header with the value of the event, not "System Hook".

Summary

The 'Test' button for System hooks sends the X-GitLab-Event header with the value of the event, e.g., like a Webhook, not System Hook as documented.

Steps to reproduce

Observed during integration with the https://github.com/jenkinsci/gitlab-branch-source-plugin

  1. Install GitLab Branch Source Plugin
  2. Create a System Hook in GitLab with the appropriate URL and secret token to trigger the plugin.
  3. Hit "Test" for the System hook

I suspect this doesn't reproduce very often because System hook consumers probably don't check the X-GitLab-Event header, since it has only one possible value.

In the case of the GitLab Jenkins Plugin, it appears that X-GitLab-Event is trusted as to whether it's a project hook or a system hook, so the 'Test' button will be hitting a Webhook path in the plugin, not the System hook path. It ends up in the same place, so probably no one noticed.

I was surprised that there wasn't already a bug report for this in the database, but I haven't traced back when the bug was introduced, as this code has been through a few refactors over the last few years.

Example Project

N/A, it's a system behaviour, not a per-project behaviour. I also assume that such hooks are only available in self-hosted GitLab, not GitLab.com, as they are system-admin managed.

What is the current bug behavior?

The test will receive a '500' error, and report Trigger as "Push Hook", "Repository Update Hook", etc.

What is the expected correct behavior?

The test receives a '200' success, and reports Trigger as "System Hook", the same as real event trigger.

Relevant logs and/or screenshots

See https://github.com/jenkinsci/gitlab-branch-source-plugin/issues/116#issuecomment-777436264 for a screenshot showing both tests (500's) and and real trigger events (200's).

The logs from the Jenkins side of the failed test:

Feb 11, 2021 4:22:35 PM WARNING hudson.init.impl.InstallUncaughtExceptionHandler handleException
Caught unhandled exception with ID e67b3619-7ea7-426d-91d2-f90af26d7bfc
org.gitlab4j.api.GitLabApiException: Unsupported X-Gitlab-Event, event Name=Push Hook
	at org.gitlab4j.api.systemhooks.SystemHookManager.handleRequest(SystemHookManager.java:107)
	at org.gitlab4j.api.systemhooks.SystemHookManager.handleEvent(SystemHookManager.java:76)
	at io.jenkins.plugins.gitlabbranchsource.GitLabSystemHookAction.doPost(GitLabSystemHookAction.java:81)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:536)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
Caused: javax.servlet.ServletException

The logs from a successful triggering of the system hook (i.e. not the Test button)

Feb 11, 2021 4:23:41 PM INFO org.gitlab4j.api.systemhooks.SystemHookManager handleRequest
handleEvent: X-Gitlab-Event=System Hook

Output of checks

I'm not the user who hit this, that person provided the details below from their installation, where the above logs came from.

Results of GitLab environment info

Expand for output related to GitLab environment info 

[root@gitlab ~]# gitlab-rake gitlab:env:info

System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   2.7.2p137
Gem Version:    3.1.4
Bundler Version:2.1.4
Rake Version:   13.0.3
Redis Version:  5.0.9
Git Version:    2.29.0
Sidekiq Version:5.2.9
Go Version:     unknown

GitLab information
Version:        13.8.3
Revision:       02529530a3d
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     12.4
URL:            https://gitlab.___
HTTP Clone URL: https://gitlab.___/some-group/some-project.git
SSH Clone URL:  git@gitlab.___:some-group/some-project.git
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version:        13.15.0
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
[root@gitlab ~]# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 13.15.0 ? ... OK (13.15.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 100 users of 100 limit.


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
4/3 ... yes
35/10 ... yes
35/11 ... yes
35/12 ... yes
35/13 ... yes
35/14 ... yes
35/15 ... yes
35/16 ... yes
48/18 ... yes
35/19 ... yes
50/24 ... yes
4/26 ... yes
48/30 ... yes
48/31 ... yes
35/32 ... yes
48/33 ... yes
48/34 ... yes
50/35 ... yes
35/36 ... yes
51/38 ... yes
51/39 ... yes
4/40 ... yes
48/42 ... yes
51/43 ... yes
51/44 ... yes
35/45 ... yes
48/46 ... yes
48/48 ... yes
48/49 ... yes
48/56 ... yes
48/57 ... yes
51/58 ... yes
48/59 ... yes
48/60 ... yes
48/61 ... yes
48/62 ... yes
48/63 ... yes
48/65 ... yes
48/66 ... yes
48/67 ... yes
27/69 ... yes
27/70 ... yes
27/71 ... yes
24/72 ... yes
27/73 ... yes
27/74 ... yes
48/75 ... yes
50/78 ... yes
27/79 ... yes
27/80 ... yes
27/81 ... yes
34/83 ... yes
5/84 ... yes
27/86 ... yes
50/87 ... yes
48/88 ... yes
27/90 ... yes
50/98 ... yes
50/99 ... yes
50/100 ... yes
27/101 ... yes
31/102 ... yes
27/105 ... yes
27/108 ... yes
48/110 ... yes
48/111 ... yes
48/113 ... yes
48/114 ... yes
50/115 ... yes
27/116 ... yes
48/117 ... yes
27/119 ... yes
48/120 ... yes
27/121 ... yes
50/122 ... yes
50/123 ... yes
50/124 ... yes
27/127 ... yes
48/128 ... yes
48/129 ... yes
27/130 ... yes
27/131 ... yes
48/132 ... yes
48/133 ... yes
27/135 ... yes
48/136 ... yes
4/137 ... yes
20/138 ... yes
48/142 ... yes
27/143 ... yes
27/144 ... yes
59/145 ... yes
62/146 ... yes
62/147 ... yes
27/150 ... yes
43/151 ... yes
25/152 ... yes
27/154 ... yes
62/155 ... yes
48/156 ... yes
48/157 ... yes
27/158 ... yes
62/159 ... yes
61/160 ... yes
59/161 ... yes
19/162 ... yes
71/163 ... yes
71/164 ... yes
71/165 ... yes
71/166 ... yes
71/167 ... yes
70/168 ... yes
76/169 ... yes
48/170 ... yes
78/172 ... yes
78/173 ... yes
78/174 ... yes
76/175 ... yes
76/176 ... yes
81/177 ... yes
81/178 ... yes
27/179 ... yes
71/180 ... yes
71/183 ... yes
71/184 ... yes
16/185 ... yes
48/186 ... yes
71/187 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.2)
Git version >= 2.29.0 ? ... yes (2.29.0)
Git user has default SSH configuration? ... yes
Active users: ... 50
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

I believe the bug is in TestHooks::SystemService, as it subclasses TestHooks::BaseService which calls

hook.execute(data, trigger_key)

and that second parameter (one of AVAILABLE_TRIGGERS's keys) ends up being title-cased and singularised into the X-GitLab-Event header.

The system hook trigger used for real events, i.e. not the test button, calls

hook.async_execute(data, 'system_hooks')

and hence always passes X-GitLab-Event: System Hook, as documented.

I would guess that the fix is one of:

  • override execute in TestHooks::SystemService;
  • have TestHooks::BaseService check if the hook is_a?(SystemHook) (which has precedent)
  • have TestHooks::BaseService down-call a new API to get the trigger_key value, making the hook.class.triggers.key(trigger.to_sym) logic specifically overridable by TestHooks::SystemService as a constant 'system_hooks'

This also highlights that the the tests are wrong, e.g.,

expect(hook).to receive(:execute).with(Gitlab::DataBuilder::Push::SAMPLE_DATA, trigger_key).and_return(success_result)

should be

expect(hook).to receive(:execute).with(Gitlab::DataBuilder::Push::SAMPLE_DATA, 'system_hooks').and_return(success_result)

for all tests in this file.

An alternative solution is to accept #206954 (closed) and change the behaviour of System hooks to populate X-GitLab-Event the same way Webhooks do. Then the Test button is correct, and the actual system hook triggering code needs to change from hard-coding system_hooks to providing the relevant trigger key.

I don't support that change, but am including it for completeness.

Edited by Paul "TBBle" Hampson