Enable pages_serve_with_zip_file_protocol by default
GitLab Pages supports serving content from Zip archives located on disk gitlab-pages#485 (closed). This flag enables the internal API to return a
zip source with file://
protocol that Pages understands.
What does this MR do?
Enable pages_serve_with_zip_file_protocol
by default for self-managed instances.
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
Tested manually on Omnibus nightly package gitlab-ee_13.9.0+rnightly.186879.66b31650-0_amd64.deb
.
> Feature.enable(:pages_serve_from_migrated_zip)
Pages logs:
{"level":"info","msg":"GitLab internal pages status API connected successfully","time":"2021-02-19T00:58:49Z"}
{"host":"gitlab.jaimem.me:443","level":"trace","msg":"httptrace.ClientTrace.GetConn","time":"2021-02-19T01:07:48Z"}
{"connection_resumed":false,"error":null,"level":"trace","msg":"httptrace.ClientTrace.TLSHandshakeDone","time":"2021-02-19T01:07:48Z","version":772}
{"idle_time_ms":0,"level":"trace","msg":"httptrace.ClientTrace.GotConn","reused":false,"time":"2021-02-19T01:07:48Z","was_idle":false}
{"cache-control":"max-age=0, private, must-revalidate","client_name":"gitlab_internal_api","connection":"keep-alive","content-type":"application/json","date":"Fri, 19 Feb 2021 01:07:49 GMT","etag":"W/\"b0024b4718a918f251cd567782b0a41a\"","level":"trace","msg":"response","referrer-policy":"strict-origin-when-cross-origin","req_url":"https://gitlab.jaimem.me/api/v4/internal/pages?host=root.pages.jaimem.me","res_status_code":200,"server":"nginx","strict-transport-security":"max-age=31536000","time":"2021-02-19T01:07:49Z","vary":"Accept-Encoding;Origin","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","x-gitlab-feature-category":"pages","x-request-id":"01EYVX88HTVNW1RC5WNQJMGC0V","x-runtime":"0.290369"}
{"level":"debug","msg":"retrieval response sent","time":"2021-02-19T01:07:49Z"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"1","content-range":"bytes 0-0/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"1024","content-range":"bytes 56-1079/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"299","content-range":"bytes 781-1079/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"error":null,"level":"trace","msg":"Root call","path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":".","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"/index.html","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"/index.html","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":"file does not exist","level":"trace","msg":"Lstat call","name":"/index.html.br","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":"file does not exist","level":"trace","msg":"Lstat call","name":"/index.html.gz","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"30","content-range":"bytes 70-99/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"error":null,"level":"trace","msg":"Open call","name":"/index.html","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"/index.html","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"326","content-range":"bytes 150-475/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"","duration_ms":31,"host":"root.pages.jaimem.me","level":"info","method":"GET","msg":"access","pages_host":"root.pages.jaimem.me","pages_https":true,"pages_project_id":2,"pages_project_prefix":"/pages-html/","pages_project_serving_type":"zip","proto":"HTTP/1.0","referrer":"","remote_addr":"118.127.122.131","remote_ip":"118.127.122.131","status":200,"system":"http","time":"2021-02-19T01:07:49Z","ttfb_ms":31,"uri":"/pages-html/","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36","written_bytes":632}
{"error":null,"level":"trace","msg":"Root call","path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"style.css","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"style.css","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":"file does not exist","level":"trace","msg":"Lstat call","name":"style.css.br","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":"file does not exist","level":"trace","msg":"Lstat call","name":"style.css.gz","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"30","content-range":"bytes 492-521/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"error":null,"level":"trace","msg":"Open call","name":"style.css","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"error":null,"level":"trace","msg":"Lstat call","name":"style.css","root-path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-19T01:07:49Z","vfs":"zip"}
{"accept-ranges":"bytes","client_name":"zip_vfs","content-length":"194","content-range":"bytes 571-764/1080","content-type":"application/zip","last-modified":"Thu, 18 Feb 2021 07:54:15 GMT","level":"trace","msg":"response","req_url":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","res_status_code":206,"time":"2021-02-19T01:07:49Z"}
{"content_type":"text/css; charset=utf-8","correlation_id":"","duration_ms":0,"host":"root.pages.jaimem.me","level":"info","method":"GET","msg":"access","pages_host":"root.pages.jaimem.me","pages_https":true,"pages_project_id":2,"pages_project_prefix":"/pages-html/","pages_project_serving_type":"zip","proto":"HTTP/1.0","referrer":"https://root.pages.jaimem.me/pages-html/","remote_addr":"118.127.122.131","remote_ip":"118.127.122.131","status":200,"system":"http","time":"2021-02-19T01:07:49Z","ttfb_ms":0,"uri":"/pages-html/style.css","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36","written_bytes":313}
{"content_type":"text/html; charset=utf-8","correlation_id":"","duration_ms":0,"error":"domain does not exist","host":"root.pages.jaimem.me","level":"info","method":"GET","msg":"access","pages_host":"root.pages.jaimem.me","pages_https":true,"proto":"HTTP/1.0","referrer":"https://root.pages.jaimem.me/pages-html/","remote_addr":"118.127.122.131","remote_ip":"118.127.122.131","status":404,"system":"http","time":"2021-02-19T01:07:49Z","ttfb_ms":0,"uri":"/favicon.ico","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36","written_bytes":3067}
Error unsupported protocol scheme
If GitLab Pages serves a 500 error page, and the logs show an error unsupported protocol scheme \"file\""
{"error":"Get \"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip\": unsupported protocol scheme \"file\"","level":"trace","msg":"Root call","path":"file:///var/opt/gitlab/gitlab-rails/shared/pages/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/pages_deployments/1/artifacts.zip","time":"2021-02-17T07:09:16Z","vfs":"zip"}
Users can disable this feature flag from the rails console and report an issue
> Feature.disable(:pages_serve_from_migrated_zip)
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Related to #321677 (closed)