Geo: Hashed storage migration failure does not log to Sentry or geo.log well
I saw https://sentry.gitlap.com/gitlab/geo1/issues/117887/, but was curious why the migration on the testbed failed. This was the corresponding geo.log
entry:
{"severity":"ERROR","time":"2018-01-10T23:55:51.110Z","class":"Geo::HashedStorageMigrationService","message":"Repository could not be migrated to Hashed Storage","error":"{:project_id=\u003e13774, :source=\u003e\"dealmore/dealmore-html\", :target=\u003e\"@hashed/42/33/423359e0a0ad21de32b01ac85bf4f11c95e3d1f3b05f254e0a80f333c0d07fcf\"}"}
I had to search production.log
to find the real reason:
root@sidekiq-asap-01.sv.geo2.gitlab.com:/var/log/gitlab/gitlab-rails# grep dealmore production.log
Moving repository from </var/opt/gitlab/git-data-file08/repositories/dealmore/dealmore-html.git> to </var/opt/gitlab/git-data-file08/repositories/@hashed/42/33/423359e0a0ad21de32b01ac85bf4f11c95e3d1f3b05f254e0a80f333c0d07fcf.git>.
mv-project failed: source path </var/opt/gitlab/git-data-file08/repositories/dealmore/dealmore-html.wiki.git> does not exist.
mv-project failed: source path </var/opt/gitlab/git-data-file08/repositories/dealmore/dealmore-html.git> does not exist.
mv-project failed: source path </var/opt/gitlab/git-data-file08/repositories/dealmore/dealmore-html.git> does not exist.
mv-project failed: source path </var/opt/gitlab/git-data-file08/repositories/dealmore/dealmore-html.git> does not exist.
I think the mv-project
error message comes from gitlab-shell, but perhaps we can bubble up the error better.