Skip to content

Spike: Investigate possibility to run X-Ray binary from Rails or other approaches

Investigate how and when we can run the X-Ray (Go) binary from the Rails application. There is an example where we run the GitLab Elasticsearch Indexer from Rails. We can try a similar approach.

Tasks

  • Determine how we can run a Go binary from the monolith.
  • Figure out how are we going to update the binary to new versions.
  • Figure out out when should we run the application.
  • Figure out how we can monitor the application (for debugging and identifying issues).
  • Define how users will be able to check the state of the scan.
  • Create a Proof of Concept MR.

Links

Updated Proposal

Since there are challenges with distributing and running the X-Ray as a binary, we decided to investigate an alternative route that involves running the X-Ray job in a standalone pipeline that is exempt from CI minutes. See #467357 (comment 1997678846) for details.

A POC MR was created: Draft: POC - Run X-Ray automatically in standal... (!159818 - closed)

The approach was verified and we are proceeding in a follow up issue: Implement Repository X-Ray CI pipelines that ar... (#473260 - closed)

Update [2024-07-24]

Unfortunately due to complications with CI pipelines, it appears that an automated X-Ray pipeline is not a viable interim approach. Refer to !159818 (comment 2014396807):

  1. Compliance pipelines. If the project has these enabled, they always run with each new pipeline. We should not make an exception from Compliance for X-Ray pipelines--at least, not without further collaboration with ~"Verify". Even if it's technically simple, it introduces a different pipeline workflow which would need much more involvement and Product decision oversight from the CI teams.

  2. This is a lesser point but compounds the first. The X-Ray job requires that the GitLab instance has at least one runner using a Docker executor and can pull the X-Ray image from a public registry. Now this is something that we require anyway with the current approach. However, in an automated pipeline system we would have to integrate conditions to check if these abilities are present. Again, while these could be technically simple changes, it does point further to the idea that we should not run X-Ray as a job due to these limitations.

Given the above, I believe it's not worth pursuing this POC.

We should further investigate integrating with groupcomposition analysis dependency scanning per #467357 (comment 2004198368). This will be done in a separate issue: Spike: Investigate replacing or integrating Rep... (#474306 - closed).

Edited by Leaminn Ma