Skip to content

Backup task: Do not generate too long file names for uploads

Summary

There was already the same ticket created number #254215 (closed). Seems like it was fixed, but we're still having the same exact issue. Basically it's the same exact description of the ticket, but I will add new error messages.

Problem: <class 'OSError: [Errno 36] File name too long: '/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/.s3cmd.9lj_ex2h.tmp' -> b'/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/url_sa_i_url_https_3A_2F_2Fwww.latintimes.com_2Fburn-notice-series-finale-spoilers-how-did-it-all-end-and-which-characters-survived-explosive-finale_psig_AOvVaw2d2_5UBy4dGKayGeDAmyhT_ust_1622228720698000_source_images_cd_vfe_ved_0CAIQjRxqFwoTCODh84vH6vACFQAAAAAdAAAAABAD'

Steps to reproduce Not sure, probably paste file from OneNote like described in the closed ticket linked above.

What is the current bug behavior? File name generated for a file uploaded via Gitlab UI is too long and backups script can't proceed with backup.

Relevant logs and/or screenshots

`!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ)
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /usr/local/bin/s3cmd --stop-on-error --delete-removed sync s3://dkb-gitlab-gitlab-uploads-eu-central-1/ /srv/gitlab/tmp/uploads/
Problem: <class 'OSError: [Errno 36] File name too long: '/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/.s3cmd.9lj_ex2h.tmp' -> b'/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/url_sa_i_url_https_3A_2F_2Fwww.latintimes.com_2Fburn-notice-series-finale-spoilers-how-did-it-all-end-and-which-characters-survived-explosive-finale_psig_AOvVaw2d2_5UBy4dGKayGeDAmyhT_ust_1622228720698000_source_images_cd_vfe_ved_0CAIQjRxqFwoTCODh84vH6vACFQAAAAAdAAAAABAD'
S3cmd:   2.2.0
python:   3.8.9 (default, Apr  6 2021, 18:45:51) 
[GCC 8.3.0]
environment LANG=C.UTF-8

Traceback (most recent call last):
  File "/usr/local/bin/s3cmd", line 3209, in <module>
    rc = main()
  File "/usr/local/bin/s3cmd", line 3106, in main
    rc = cmd_func(args)
  File "/usr/local/bin/s3cmd", line 1967, in cmd_sync
    return cmd_sync_remote2local(args)
  File "/usr/local/bin/s3cmd", line 1533, in cmd_sync_remote2local
    ret, seq, size_transferred = _download(remote_list, seq, remote_count + update_count, size_transferred, dir_cache)
  File "/usr/local/bin/s3cmd", line 1387, in _download
    os.rename(chkptfname_b, deunicodise(dst_file))
OSError: [Errno 36] File name too long: '/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/.s3cmd.9lj_ex2h.tmp' -> b'/srv/gitlab/tmp/uploads/@hashed/71/ee/71ee45a3c0db9a9865f7313dd3372cf60dca6479d46261f3542eb9346e4a04d6/028acbe35bbe90de09dc21d3d4159916/url_sa_i_url_https_3A_2F_2Fwww.latintimes.com_2Fburn-notice-series-finale-spoilers-how-did-it-all-end-and-which-characters-survived-explosive-finale_psig_AOvVaw2d2_5UBy4dGKayGeDAmyhT_ust_1622228720698000_source_images_cd_vfe_ved_0CAIQjRxqFwoTCODh84vH6vACFQAAAAAdAAAAABAD'

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ)
  If the error persists, please report the
  above lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net`

Git Info

`System information
System:
Proxy:          GITLAB_GITLAB_PAGES_SERVICE_PORT_HTTP_GITLAB_PAGES_PROXY: 8090
Current User:   git
Using RVM:      no
Ruby Version:   2.7.5p203
Gem Version:    3.2.19
Bundler Version:2.2.19
Rake Version:   13.0.6
Redis Version:  unknown
Git Version:    unknown
Sidekiq Version:6.3.1
Go Version:     unknown

GitLab information
Version:        14.7.2-ee
Revision:       39a169b2f25
Directory:      /srv/gitlab
DB Adapter:     PostgreSQL
DB Version:     12.4
URL:            https://gitlab.dev.codefactory.sh
HTTP Clone URL: https://gitlab.dev.codefactory.sh/some-group/some-project.git
SSH Clone URL:  git@gitlab.dev.codefactory.sh:some-group/some-project.git
Elasticsearch:  yes
Geo:            no
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers: saml, saml_2

GitLab Shell
Version:        13.22.2
Repository storage paths:
- default:      /var/opt/gitlab/repo
GitLab Shell path:              /home/git/gitlab-shell
Git:            /usr/bin/git
git@gitlab-toolbox-backup-manual-z9fu3-vnt4x:/$ `
Edited by Michael Kozono