Support to detect git clone/pull audit event
What does this MR do and why?
Related issue: #384960
Support to detect git clone/pull audit event since now
GitHub support to detect both audit events since 2020.10, technically it is doable. Gitlab needs it too.
We can pass the want/have
info from Gitaly to Rails to
generate the streaming audit for clone/pull/push event.
By the workflow in #384960 (comment 1386091357):
sequenceDiagram
User->>+Workhorse: git clone or pull (HTTPS)
Workhorse->>Rails: request /git-upload-pack
Rails->>Workhorse: authentication info (need an audit?)
Workhorse->>Gitaly: PostUploadPackWithSidechannel
Gitaly->>Workhorse: PostUploadPackWithSidechannelResponse(stats={Wants=9, Haves=0, ...})
Workhorse-->>Rails: Report audit event if needed
Workhorse->>-User: Return
sequenceDiagram
User->>+GitLabShell: git clone or pull (SSH)
GitLabShell->>Rails: request /allowed
Rails->>GitLabShell: authentication info (need an audit?)
GitLabShell->>Gitaly: UploadPackWithSidechannel
Gitaly->>GitLabShell: UploadPackWithSidechannelResponse(stats={Wants=9, Haves=0, ...})
GitLabShell-->>Rails: Report audit event if needed
GitLabShell->>-User: Return
and the discussion from #384960 (comment 1431604700),
This MR creates a new separate internal API file lib/api/internal/shellhorse.rb
to deal with all the API needs to AUTH both the Gitlab-Workhorse-Api-Request
and Gitlab-Shell-Api-Request
header. The new API path is api/v4/internal/shellhorse/git_audit_event
Screenshots or screen recordings
Just add a new field in the streaming audit event JSON: details --> custom_message --> verb
The new verb
field's value should be these: clone/pull/push
{
"id":"4795387c-4ac6-48a4-9ceb-40d9ff4d160b",
"author_id":1,
"entity_id":25,
"entity_type":"Project",
"details":{
"author_name":"Administrator",
"author_class":"User",
"target_id":25,
"target_type":"Project",
"target_details":"pipeline-demo",
"custom_message":{
"protocol":"ssh",
"action":"git-upload-pack",
"verb":"clone"
},
"ip_address":"127.0.0.1",
"entity_path":"tragedxy/pipeline-demo"
},
"ip_address":"127.0.0.1",
"author_name":"Administrator",
"entity_path":"tragedxy/pipeline-demo",
"target_details":"pipeline-demo",
"created_at":"2023-06-08T10:47:07.535Z",
"target_type":"Project",
"target_id":25,
"event_type":"repository_git_operation"
}
How to set up and validate locally
- Register a free http API test account in https://pipedream.com, create a RequestBin workflow
- Add the RequestBin workflow as the streaming destination in Gitlab, just follow the Gitlab doc reference steps
- Enable the feature flag
log_git_streaming_audit_events
- Use the Curl command to request the new internal API like below:
curl --location 'http://gdk.test:3000/api/v4/internal/git_audit_event' \
--header 'Content-Type: application/json' \
--header 'Cookie: BetterErrors-2.10.0-CSRF-Token=4849bd3f-c113-439e-b625-a25468586a29' \
--data '{
"protocol": "ssh",
"action": "git-upload-pack",
"username": "root",
"gl_repository": "project-27",
"packfile_stats": {
"wants": 0,
"haves": 0,
"payload_size": 0,
"packets": 0,
"caps": ["", ""],
"shallows": 0,
"deepen": "",
"filter": ""
}
}'
- Keeping monitor the pipedream.com RequestBin workflow page, the streaming audit event with the new
verb
field should be sent to it if all correct above, like below screenshot:
Related to #384960
/cc @chaomao