Skip to content

Maven VReg: disable check with upstream when cache validity period is 0

Context

In the maven virtual registry, we have this concept of: is a cached response valid or not? Better said: is the cached response fresh enough or not.

To answer that, users can set a validity period in hours at the registry level. The default value is 1, meaning that every hour, when accessing a cached response that is outside the validity period, we will check with upstream if we can still use it.

The problem is that some package registries, in particular the official public ones (in this case, maven central), do not allow packages to be deleted or overwritten. This means that once a package is published, its files will not move at all. For virtual registries, this also means that we don't need to check with maven central ever.

Thus, we need a value representing that the upstream check is disabled. That value can be 0.

What does this MR do and why?

  • Update VirtualRegistries::Packages::Maven::CachedResponse#stale? method to return false if the registry's cache_validity_hours is zero.
  • Add a spec for the change.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

N/A

How to set up and validate locally

  1. Enable the feature flag : Feature.enable(:virtual_registry_maven).

  2. Have a PAT and a root group (any visibility) ready.

  3. For the virtual registry settings, we don't have a UI or API (yet), we thus need to create them in a rails console:

    root_group = Group.all.detect(&:root?)
    r = ::VirtualRegistries::Packages::Maven::Registry.create!(group: <root_group>, cache_validity_hours: 0)
    u = ::VirtualRegistries::Packages::Maven::Upstream.create!(group: <root_group>, url: 'https://repo1.maven.org/maven2')
    VirtualRegistries::Packages::Maven::RegistryUpstream.create!(group: <root_group>, registry: r, upstream: u)
  4. Pull a package file:

    $ curl --header "Private-Token: <PAT>" "http://gdk.test:3000/api/v4/virtual_registries/packages/maven/<r.id>/org/springframework/spring-web/6.1.12/spring-web-6.1.12.pom"
    • Check that you get the .pom file back.
  5. In the rails console:

    cr = ::VirtualRegistries::Packages::Maven::CachedResponse.last
    => #<VirtualRegistries::Packages::Maven::CachedResponse:0x00000003f7d13210
       id: 109,
       group_id: 24,
       upstream_id: 23,
       upstream_checked_at: Mon, 21 Oct 2024 17:01:26.265514000 UTC +00:00,
       downloaded_at: Mon, 21 Oct 2024 17:03:43.910358000 UTC +00:00,
       created_at: Mon, 21 Oct 2024 17:01:26.302464000 UTC +00:00,
       updated_at: Mon, 21 Oct 2024 17:01:26.302464000 UTC +00:00,
       file_store: 2,
       size: 2398,
       downloads_count: 1,
       relative_path: "/org/springframework/spring-web/6.1.12/spring-web-6.1.12.pom",
       file: "spring-web-6.1.12.pom",
       upstream_etag: "\"54ce07f4124259b2ea58548e9d620004\"",
       content_type: "[FILTERED]",
       file_final_path: "5f/9c/5f9c4ab08cac7457e9111a30e4664920607ea2c115a1433d7be98e97e64244ca/@final/f7/ea/31a351c53f3a950a1ae4c4b4f8053a5e9c213ec12981814a8be52cab2504",
       status: "default",
       object_storage_key: "[FILTERED]",
       file_md5: "54ce07f4124259b2ea58548e9d620004",
       file_sha1: "bbde7c9fb6d74f9a2393bb36b0d4ac7e72c227ee">

    Cache entry created ! 🎉

  6. Downloading the file again will not hit the upstream:

    $ curl --header "Private-Token: <PAT>" "http://gdk.test:3000/api/v4/virtual_registries/packages/maven/<r.id>/org/springframework/spring-web/6.1.12/spring-web-6.1.12.pom"
  7. Notice that the upstream_checked_at hasn't been updated, which means that downloading the file again didn't hit the upstream

    cr.reload.upstream_checked_at

Related to #486494 (closed)

Edited by Moaz Khalifa

Merge request reports

Loading