Add docs for SAST in offline environments (no DinD)
What does this MR do?
Adds instructions on how to get SAST working offline to our documentation for users in offline environments, restricted networks, and air gaps, and without Docker in Docker.
For #11520 (closed)
SAST offline (without DinD) is technically possible but we don't have instructions on how to configure it to work without internet access.
Docs for SAST offline with DinD are not available this time. I can't get it working in a way that doesn't require hacking the SAST CI job template for every project and/or disabling HTTPS for container registry traffic. For SAST using DinD, see: !22713 (closed)
Adding these instructions should allow users to use GitLab's SAST analyzers in their offline environment.
Background and steps for testing offline SAST functionality:
From: !22713 (comment 285846966)
"Offline" SAST Test Environment Instructions
Requirements:
- Linux system or VM with 4+ CPU threads and 12GB+ RAM
To emulate an "on-prem" setup, I suggest using a local machine running Ubuntu as the host OS.
If on-prem
setup is not an option, you can use a VM . To emulate an "offline" environment with a VM:
- block external traffic and requests (for realism, block everything; for testing, block
registry.gitlab.com
andhub.docker.com
at the very least) - Use
tcpdump
(orngrep
) to verify that there are no external resources being fetched
Setup Process:
-
Install Docker
curl -sSL https://get.docker.com/ | sudo bash
-
Install & Register GitLab Runner using Docker Executor
sudo docker run -d --name gitlab-runner --restart always \ -v /srv/gitlab-runner/config:/etc/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:latest
-
Install and start Docker Registry
sudo docker run -d -p 5000:5000 --name registry registry:2
-
Push/import default SAST analyzers to Local Docker Registry. (see
analyzerator.sh
below) -
Modify
/etc/gitlab-runner/config.toml
[[runners]] executor = "docker" [runners.docker] pull_policy = "if-not-present"
-
Upload license for GitLab Ultimate
-
Import or create a project for testing SAST functionality
-
Disable Internet access at system and/or network level to simulate an "air gap".
-
Customize SAST template
.gitlab-ci.yml
: Disable Docker in Docker and set SAST_ANALYZER_IMAGE_PREFIX to point to your local registryinclude: - template: SAST.gitlab-ci.yml variables: SAST_ANALYZER_IMAGE_PREFIX: "localhost:5000/analyzers" SAST_DISABLE_DIND: "true"
-
If all was successful, the SAST analyzers will scan and create reports without requiring Internet access.
🎉
Scripts created to help set up and test this:
setup.sh
- script to automate Steps 1 - 4 (on Ubuntu)
#!/bin/bash
sudo apt-get update
## sudo apt-get upgrade -y
# install system dependencies
sudo apt-get install -y curl ca-certificates openssh-server
# add gitlab sources
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash
# install gitlab: https://about.gitlab.com/install/
## offline local:
sudo EXTERNAL_URL="http://127.0.0.1" apt-get install gitlab-ee
# install docker: https://docs.gitlab.com/runner/install/docker.html#docker-image-installation
curl -sSL https://get.docker.com/ | sudo bash
sudo usermod -aG docker $whoami
# registry: https://docs.docker.com/registry/#basic-commands
docker run -d -p 5000:5000 --name registry registry:2
# gitlab-runner: https://docs.gitlab.com/runner/install/docker.html
docker run -d --name gitlab-runner --restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
analyzerator.sh
- script to automate pull/push of all SAST analyzers (this would be docker export/import in air gap, but same idea here)
#!/bin/bash
analyzers=(bandit brakeman eslint flawfinder gosec kubesec nodejs-scan phpcs-security-audit pmd-apex secrets security-code-scan sobelow spotbugs tslint)
gitlab=registry.gitlab.com/gitlab-org/security-products/analyzers/
registry=localhost:5000
for i in "${analyzers[@]}"
do
echo pulling $gitlab$i:2
docker pull $gitlab$i:2
docker tag $(sudo docker images | grep $i | awk '{print $3}') $registry/analyzers/$i:2;
docker push $registry/analyzers/$i;
done
Does this MR meet the acceptance criteria?
I think so... but if not, I'll fix it!
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
@NicoleSchwartz @twoodham @theoretick @dsearles @david @stkerr @tmccaslin @marcia @marcel.amirault @eread @axil