pg-upgrade keeps failing with Session terminated
I am currently using an old version of Gitlab (8.17.8). After looking at all the past issues and recommendations I am following the path of 8.17.8 -> 9.5.10 ---> 10.8.7 to upgrade. Before the upgrade I am upgrading postgresql with this command: sudo gitlab-ctl pg-upgrade
It keeps failing with the following error. I am unable to find the reason for this. This is the output after running that command. (I can skip the upgrading part for version 9 , but for 10 I will have to upgrade it anyways)
sudo gitlab-ctl pg-upgrade
Checking for an omnibus managed postgresql: OK
Checking if we already upgraded: NOT OK
Checking for a newer version of PostgreSQL to install: OK
Upgrading PostgreSQL to 9.6.1
Checking if PostgreSQL bin files are symlinked to the expected location: OK
Toggling deploy page:cp /opt/gitlab/embedded/service/gitlab-rails/public/deploy.html /opt/gitlab/embedded/service/gitlab-rails/public/index.html
Toggling deploy page: OK
Toggling services:ok: down: logrotate: 0s, normally up
ok: down: sidekiq: 0s, normally up
Toggling services: OK
Stopping the database:ok: down: postgresql: 1s, normally up
Stopping the database: OK
Update the symlinks: OK
Creating temporary data directory: OK
Initializing the new database: OK
Upgrading the data:/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:124:in `run_command': Command timed out after 600s: (Mixlib::ShellOut::CommandTimeout)
Command exceeded allowed execution time, process terminated
---- Begin output of su - gitlab-psql -c "/opt/gitlab/embedded/bin/pg_upgrade -b /opt/gitlab/embedded/postgresql/9.2.18/bin -d /var/opt/gitlab/postgresql/data -D /var/opt/gitlab/postgresql/data.9.6.1 -B /opt/gitlab/embedded/bin" ----
STDOUT: Performing Consistency Checks
-----------------------------
Checking cluster versions ok
Checking database user is the install user ok
Checking database connection settings ok
Checking for prepared transactions ok
Checking for reg* system OID user data types ok
Checking for contrib/isn with bigint-passing mismatch ok
Checking for roles starting with 'pg_' ok
Checking for invalid "line" user columns ok
Creating dump of global objects ok
Creating dump of database schemas
gitlabhq_production
postgres
template1
ok
Checking for presence of required libraries ok
Checking database user is the install user ok
Checking for prepared transactions ok
If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting files from new pg_clog ok
Copying old pg_clog to new server ok
Setting next transaction ID and epoch for new cluster ok
Deleting files from new pg_multixact/offsets ok
Setting oldest multixact ID on new cluster ok
Resetting WAL archives ok
Setting frozenxid and minmxid counters in new cluster ok
Restoring global objects in the new cluster ok
Restoring database schemas in the new cluster
gitlabhq_production
postgres
template1
ok
Setting minmxid counter in new cluster ok
Copying user relation files
/var/opt/gitlab/postgresql/data/base/16385/11965
/var/opt/gitlab/postgresql/data/base/16385/11967
/var/opt/gitlab/postgresql/data/base/16385/11831
/var/opt/gitlab/postgresql/data/base/16385/11833
/var/opt/gitlab/postgresql/data/base/16385/61475
/var/opt/gitlab/postgresql/data/base/16385/61478
/var/opt/gitlab/postgresql/data/base/16385/61480
/var/opt/gitlab/postgresql/data/base/16385/61481
/var/opt/gitlab/postgresql/data/base/16385/439451
/var/opt/gitlab/postgresql/data/base/16385/61495
/var/opt/gitlab/postgresql/data/base/16385/61497
/var/opt/gitlab/postgresql/data/base/16385/61497_fsm
/var/opt/gitlab/postgresql/data/base/16385/61500
/var/opt/gitlab/postgresql/data/base/16385/61502
/var/opt/gitlab/postgresql/data/base/16385/61503
/var/opt/gitlab/postgresql/data/base/16385/61505
/var/opt/gitlab/postgresql/data/base/16385/61508
/var/opt/gitlab/postgresql/data/base/16385/61510
/var/opt/gitlab/postgresql/data/base/16385/61511
/var/opt/gitlab/postgresql/data/base/16385/61513
/var/opt/gitlab/postgresql/data/base/16385/61516
/var/opt/gitlab/postgresql/data/base/16385/61518
/var/opt/gitlab/postgresql/data/base/16385/61521
/var/opt/gitlab/postgresql/data/base/16385/61523
/var/opt/gitlab/postgresql/data/base/16385/61523.1
/var/opt/gitlab/postgresql/data/base/16385/61523.2
/var/opt/gitlab/postgresql/data/base/16385/61523.3
/var/opt/gitlab/postgresql/data/base/16385/61523_fsm
/var/opt/gitlab/postgresql/data/base/16385/61523_vm
/var/opt/gitlab/postgresql/data/base/16385/61526
/var/opt/gitlab/postgresql/data/base/16385/61526.1
/var/opt/gitlab/postgresql/data/base/16385/61526_fsm
/var/opt/gitlab/postgresql/data/base/16385/61526_vm
/var/opt/gitlab/postgresql/data/base/16385/61528
/var/opt/gitlab/postgresql/data/base/16385/61528_fsm
/var/opt/gitlab/postgresql/data/base/16385/61529
/var/opt/gitlab/postgresql/data/base/16385/61531
/var/opt/gitlab/postgresql/data/base/16385/61531_fsm
/var/opt/gitlab/postgresql/data/base/16385/61531_vm
/var/opt/gitlab/postgresql/data/base/16385/61534
/var/opt/gitlab/postgresql/data/base/16385/61536
/var/opt/gitlab/postgresql/data/base/16385/61539
/var/opt/gitlab/postgresql/data/base/16385/61541
/var/opt/gitlab/postgresql/data/base/16385/61542
/var/opt/gitlab/postgresql/data/base/16385/438626
/var/opt/gitlab/postgresql/data/base/16385/438626_fsm
/var/opt/gitlab/postgresql/data/base/16385/438626_vm
/var/opt/gitlab/postgresql/data/base/16385/61551
/var/opt/gitlab/postgresql/data/base/16385/61553
/var/opt/gitlab/postgresql/data/base/16385/61553_fsm
/var/opt/gitlab/postgresql/data/base/16385/61553_vm
/var/opt/gitlab/postgresql/data/base/16385/61557
/var/opt/gitlab/postgresql/data/base/16385/61559
/var/opt/gitlab/postgresql/data/base/16385/61560
/var/opt/gitlab/postgresql/data/base/16385/61562
/var/opt/gitlab/postgresql/data/base/16385/61562_fsm
/var/opt/gitlab/postgresql/data/base/16385/61562_vm
/var/opt/gitlab/postgresql/data/base/16385/61565
/var/opt/gitlab/postgresql/data/base/16385/418173
/var/opt/gitlab/postgresql/data/base/16385/418173_fsm
/var/opt/gitlab/postgresql/data/base/16385/418173_vm
/var/opt/gitlab/postgresql/data/base/16385/61573
/var/opt/gitlab/postgresql/data/base/16385/61575
/var/opt/gitlab/postgresql/data/base/16385/61575_fsm
/var/opt/gitlab/postgresql/data/base/16385/61575_vm
/var/opt/gitlab/postgresql/data/base/16385/61578
/var/opt/gitlab/postgresql/data/base/16385/61580
/var/opt/gitlab/postgresql/data/base/16385/61581
/var/opt/gitlab/postgresql/data/base/16385/61583
/var/opt/gitlab/postgresql/data/base/16385/61583_fsm
/var/opt/gitlab/postgresql/data/base/16385/61583_vm
/var/opt/gitlab/postgresql/data/base/16385/61586
/var/opt/gitlab/postgresql/data/base/16385/61586.1
/var/opt/gitlab/postgresql/data/base/16385/61586.2
/var/opt/gitlab/postgresql/data/base/16385/61586.3
/var/opt/gitlab/postgresql/data/base/16385/61586.4
/var/opt/gitlab/postgresql/data/base/16385/61586.5
/var/opt/gitlab/postgresql/data/base/16385/61586.6
/var/opt/gitlab/postgresql/data/base/16385/61586.7
/var/opt/gitlab/postgresql/data/base/16385/61586.8
/var/opt/gitlab/postgresql/data/base/16385/61586.9
/var/opt/gitlab/postgresql/data/base/16385/61586.10
/var/opt/gitlab/postgresql/data/base/16385/61586.11
/var/opt/gitlab/postgresql/data/base/16385/61586.12
STDERR: Session terminated, terminating shell... ...terminated.
---- End output of su - gitlab-psql -c "/opt/gitlab/embedded/bin/pg_upgrade -b /opt/gitlab/embedded/postgresql/9.2.18/bin -d /var/opt/gitlab/postgresql/data -D /var/opt/gitlab/postgresql/data.9.6.1 -B /opt/gitlab/embedded/bin" ----
Ran su - gitlab-psql -c "/opt/gitlab/embedded/bin/pg_upgrade -b /opt/gitlab/embedded/postgresql/9.2.18/bin -d /var/opt/gitlab/postgresql/data -D /var/opt/gitlab/postgresql/data.9.6.1 -B /opt/gitlab/embedded/bin" returned
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
from /opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/util.rb:8:in `get_command_output'
from /opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/pg_upgrade.rb:31:in `run_pg_command'
from (eval):182:in `block (2 levels) in load_file'
from (eval):294:in `progress_message'
from (eval):180:in `block in load_file'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.3.6/lib/omnibus-ctl.rb:181:in `block in add_command_under_category'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.3.6/lib/omnibus-ctl.rb:575:in `run'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.3.6/bin/omnibus-ctl:31:in `<top (required)>'
from /opt/gitlab/embedded/bin/omnibus-ctl:22:in `load'
from /opt/gitlab/embedded/bin/omnibus-ctl:22:in `<main>'
Ensuring PostgreSQL is updated: NOT OK
Error ensuring PostgreSQL is updated. Please check the logs
dpkg: error processing package gitlab-ce (--configure):
subprocess installed post-installation script returned error exit status 1
E: Sub-process /usr/bin/dpkg returned an error code (1)Edited by Arthos