Skip to content

Job log viewer only shows last 500Kb

Current Status 2024-05-20 Update

See #295654 (comment 1898896880) for the experimental work done to view full logs.

This issue is blocked until a new renderer has been implemented to make this performant and ensure platform stability.

Summary

When viewing the logs of a particularly verbose CI job, only the last 500Kb is shown. The only way to see the whole log is a "Complete Raw" link, which is not useful if the output contains ANSI control codes for things such as colour.

This means that, to view long logs with ANSI codes, the user must:

  1. download the complete log
  2. use less -R or, if they want to keep the scrollbar, a third-party viewer program
  3. remember to delete the log afterwards

Long logs are often generated when trying to debug CI problems, and having to go through these extra steps is a poor UX.

Steps to reproduce

  • Run a CI job which produces over 500Kb of output
  • Attempt to read the output in the job viewer

Example Project

This is running on a private instance of GitLab.

What is the current bug behaviour?

The only way to view the whole log (or, indeed, anything that happens before a "clear screen" control code) is to view the raw file. There is no way to view the whole log in-browser with ANSI codes processed.

What is the expected correct behavior?

As well as the "Complete Raw" link, there should be a "View Complete Log" button, which causes the whole log to be shown in the log viewer, with colour codes. If this would cause memory issues, the log should be paginated — but the whole log should be viewable in the standard viewer.

Ideally, "clear screen" control codes should cause a line saying something like «screen cleared» to be displayed rather than everything before that point being lost — or, if pagination is used, it could simply start a new page.

Design proposal

image

Allow users to either load the next 5MiB of log or download complete raw.

Relevant logs and/or screenshots

Screenshot_2020-12-23_at_12.54.43

You can see that the line shown as line 1 is clearly not the start of the actual log.

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information System: Current User: git Using RVM: no Ruby Version: 2.7.2p137 Gem Version: 3.1.4 Bundler Version:2.1.4 Rake Version: 13.0.1 Redis Version: 5.0.9 Git Version: 2.29.0 Sidekiq Version:5.2.9 Go Version: unknown GitLab information Version: 13.7.0 Revision: 91da1dcc7a1 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.9 URL: https://git.omgponies.org.uk HTTP Clone URL: https://git.omgponies.org.uk/some-group/some-project.git SSH Clone URL: git@git.omgponies.org.uk:some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 13.14.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 

Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version \>= 13.14.0 ? ... OK (13.14.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 3/1 ... yes 2/2 ... yes 2/3 ... yes 3/4 ... yes 2/5 ... yes 3/7 ... yes 2/8 ... yes 2/9 ... yes 2/10 ... yes 2/11 ... yes 2/12 ... yes 2/13 ... yes 3/14 ... yes 3/15 ... yes 2/16 ... yes 2/17 ... yes 3/18 ... yes 2/20 ... yes 2/21 ... yes Redis version \>= 4.0.0? ... yes Ruby version \>= 2.7.2 ? ... yes (2.7.2) Git version \>= 2.29.0 ? ... yes (2.29.0) Git user has default SSH configuration? ... yes Active users: ... 4 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes

Checking GitLab App ... Finished

Checking GitLab subtasks ... Finished

Possible fixes

Edited by Rutvik Shah