Allow Agent QA tests to be runnable Production

The problem

For the gitlab-qa tests to be runnable without local hacks against staging and production:

• We should remove this requirement of setting "allow local connections" as admin. I guess this could be done several ways. But mainly, I think we need a way to wrap this line and this line in a conditional that depends on an environment variable or something else, like a specific test scenario, so that these lines are skipped if the test running against staging or production. I believe these lines are only required for dockerized images inside the CI to talk to k3s containers.

Proposal

@nagyv-gitlab implemented a POC of a pipeline that:

• spins up a k3s cluster
• deploys an agent to it
• register the agent
• checks for pull based deployments and CI Tunnel access

What needs to be done

• Clean up https://gitlab.com/nagyv-gitlab/configure-product-walkthrough/
• Move it under gitlab-org/configure
• Add schedule job to run the tests every 3 hours with the version from the gitlab repo
• Take the agentk version from https://gitlab.com/gitlab-org/gitlab/-/blob/master/GITLAB_KAS_VERSION.
  • If we could take it from this file but allow for a floating patch version, it would be even better.

Follow-up next steps

• Send message to slack on failure: gitlab-com/business-technology/change-management#335 and gitlab-org/configure/k8s-agent-qa#3
• Make the code environment friendly so we can run it on staging by creating a mirror: gitlab-org/configure/k8s-agent-qa#4