Skip to content

Enable instance level audit event streaming to AWS S3

Huzaifa Iftikhar requested to merge 423236_send_instance_leveL_audit_events_to_aws_s3 into master

What does this MR do and why?

  1. Enable audit event streaming to AWS S3 feature flag allow_streaming_instance_audit_events_to_amazon_s3.
  2. Create base class ee/lib/audit_events/strategies/base_amazon_s3_destination_strategy.rb and move the common logic of streaming both group level and instance level audit events streaming to AWS S3 inside this class.
  3. Created shared specs ee/spec/support/shared_examples/lib/audit_events/strategies/amazon_s3_destination_strategy_shared_examples.rb and used it for both instance and group level audit events streaming.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Ensure you have ultimate license on your GDK.
  2. Enable feature flag by running Feature.enable('allow_streaming_instance_audit_events_to_amazon_s3') in rails console.
  3. Create an AWS bucket and AWS access key id and secret access. You can watch the initial part of this video to understand how to create those.
  4. Once the AWS setup is complete, run the following the rails console (we don't have the GraphQL API or UI yet) updating the values of bucket name, region and others with the ones that you have.
AuditEvents::Instance::AmazonS3Configuration.create({access_key_xid: "MYACCESSKEY", secret_access_key: "MYSECRETACCESSKEY", name: "AWS S3 Sandbox Instance", bucket_name: "audit-logs-gitlab-instance", aws_region: "ap-southeast-2"})
  1. Navigate to gitlab admin application settings (https://gdk.test:3000/admin/application_settings/general) and update any setting. This should create a audit event in the database.
  2. Go to AWS S3 bucket on S3 console and verify that a new JSON file for the above audit event is created.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Closes #423236 (closed)

Edited by Huzaifa Iftikhar

Merge request reports