Geo::FileRegistryRemovalService is failing if uploads is configured with an object storage.

Summary

We have a customer where Sidekiq frequently fails with the following error in their Secondary Geo Node:

2022-07-12T12:01:45.045Z pid=23 tid=626fvn class=Geo::FileRegistryRemovalWorker jid=8fe553602485d080a7bdccc8 INFO: start
2022-07-12T12:01:45.046Z pid=23 tid=626fvn class=Geo::FileRegistryRemovalWorker jid=8fe553602485d080a7bdccc8 INFO: arguments: ["upload",15962]
2022-07-12T12:01:45.067Z pid=23 tid=626fvn class=Geo::FileRegistryRemovalWorker jid=8fe553602485d080a7bdccc8 elapsed=0.022 INFO: fail
...
2022-07-12T12:01:45.068Z pid=23 tid=626fvn WARN: NoMethodError: undefined method `model' for #<CarrierWave::Storage::Fog::File:0x00007f6f67da28d8>
2022-07-12T12:01:45.068Z pid=23 tid=626fvn WARN: /srv/gitlab/app/uploaders/file_uploader.rb:42:in `relative_path'
/srv/gitlab/app/uploaders/file_uploader.rb:36:in `absolute_path'
/srv/gitlab/ee/app/services/geo/file_registry_removal_service.rb:94:in `block in file_path'
/srv/gitlab/lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'
/srv/gitlab/ee/app/services/geo/file_registry_removal_service.rb:87:in `file_path'
/srv/gitlab/ee/app/services/geo/file_registry_removal_service.rb:64:in `destroy_file'
/srv/gitlab/ee/app/services/geo/file_registry_removal_service.rb:39:in `block in execute'
/srv/gitlab/app/services/concerns/exclusive_lease_guard.rb:29:in `try_obtain_lease'
/srv/gitlab/ee/app/services/geo/file_registry_removal_service.rb:31:in `execute'
/srv/gitlab/ee/app/workers/geo/file_registry_removal_worker.rb:20:in `perform'

I was able to reproduce this on my end by manually triggering Geo::FileRegistryRemovalService:

irb(main):084:0> Geo::FileRegistryRemovalService.new("upload", 9).execute
Traceback (most recent call last):
       10: from (irb):83
        9: from ee/app/services/geo/file_registry_removal_service.rb:31:in `execute'
        8: from app/services/concerns/exclusive_lease_guard.rb:29:in `try_obtain_lease'
        7: from ee/app/services/geo/file_registry_removal_service.rb:39:in `block in execute'
        6: from ee/app/services/geo/file_registry_removal_service.rb:64:in `destroy_file'
        5: from ee/app/services/geo/file_registry_removal_service.rb:87:in `file_path'
        4: from lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'
        3: from ee/app/services/geo/file_registry_removal_service.rb:94:in `block in file_path'
        2: from app/uploaders/file_uploader.rb:36:in `absolute_path'
        1: from app/uploaders/file_uploader.rb:42:in `relative_path'
NoMethodError (undefined method `model' for #<CarrierWave::Storage::Fog::File:0x0000ffff6eab9be0>)
irb(main):085:0>

This happens if you configure Uploads to use an object storage.

Looking at the code, https://gitlab.com/gitlab-org/gitlab/-/blob/v14.10.4-ee/ee/app/services/geo/file_registry_removal_service.rb#L94

The error happens in file_uploader.class.absolute_path(file_uploader.file).

Steps to reproduce

Configure GitLab Geo.
Make sure that uploads is configured with an object storage.
Run Geo::FileRegistryRemovalService.new("upload", 9).execute. Replace 9 with the actual id of the upload.

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info


(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)