Draft: Do not merge, ONLY FOR POC
What does this MR do and why?
This MR is only for testing purposes of clickhouse as database for audit events. This MR should not be merged.
How to set up and validate locally
- Install ClickHouse: https://clickhouse.com/docs/en/install#quick-install
- Create audit events table in ClickHouse server
CREATE TABLE audit_events
(
author_id Int32,
author_name Nullable(String),
created_at DateTime,
details Nullable(String),
entity_id Int32,
entity_path Nullable(String),
entity_type String,
id Int64,
ip_address Nullable(IPv4),
target_details Nullable(String),
target_id Nullable(Int64),
target_type Nullable(String),
) ENGINE = MergeTree()
ORDER BY (entity_type, entity_id, author_id, created_at, id);
- Create a csv of postgres audit events using rails console
require 'csv'
filename = 'audit_events.csv'
CSV.open(filename, "w") do |csv|
AuditEvent.find_each(batch_size: 1000) do |event|
csv << [
event.author_id,
event.author_name,
event.created_at.utc.strftime('%Y-%m-%d %H:%M:%S'),
event.details.to_json,
event.entity_id,
event.entity_path,
event.entity_type,
event.id,
event.ip_address.present? ? event.ip_address.to_s : "\\N",
event.target_details,
event.target_id,
event.target_type
]
end
end
- Import csv to ClickHouse
cat audit_events.csv | clickhouse-client --query="INSERT INTO audit_events FORMAT CSV
- create
config/click_house.yml
inside gitlab directory.
development:
main:
database: default
url: 'http://localhost:8123'
username: default
password: password
- Run GDK
- Visit audit event pages for a group or project.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Harsimar Sandhu