Skip to content

Gitlab-CE running in docker - temporary unavailable - lot of 502 errors

Summary

Running Gitlab 10.5.1 docker at Amazon ECS ElasticLoadBalancer for http/s NetworkLoadBalancer for SSH at the moment only one docker container - no active/active config (we had this before, but to analyse issues we went to single container) Using Elasticache Redis ReplicationGroup Using AWS RDS Postgres

Gitlab is showing 502, Gitlab is taking to much time to respond or some 404 errors. At the same time i can see lots of 502 on the AWS Loadbalancer gl2 and negative spikes in the container memory consumption on ecs cloudwatch metricsgl1

Steps to reproduce

really dont know ... at the moment i don't know how to reproduce

What is the current bug behavior?

For some reaseon gitlab shows sometimes 502 or 404 errors

What is the expected correct behavior?

Works without 502 and 404 errors

Relevant logs

A lot of gitlab.socket failures half an hour before 502 errors 
13:02:21
2018-03-01_13:02:21.71007 2018/03/01 13:02:21 error: GET "/": badgateway: failed after 0s: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: resource temporarily unavailable

13:02:22
2018-03-01_13:02:22.07196 2018/03/01 13:02:22 error: GET "/": badgateway: failed after 0s: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: resource temporarily unavailable

no other relevant log entrys found.

Details of package version

Provide the package version installation details 
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                                  Version                         Architecture                    Description
+++-=====================================================-===============================-===============================-===============================================================================================================
ii  gitlab-ce                                             10.5.1-ce.0                     amd64                           GitLab Community Edition (including NGINX, Postgres, Redis)

Environment details

  • Operating System: gitlab/gitlab-ce:10.5.1-ce.0
  • Installation Target:
    • VM: AWS
    • Other: Using ECS Service, running docker version of gitlab-ce
  • Installation Type, remove incorrect values:
    • Other: upgrading to gitlab:latest about each month
  • Is there any other software running on the machine: AWS ECS / Docker
  • At the moment single node
  • Resources
    • CPU: m4.xlarge - 4 core
    • Memory total: memory 13 Gig, memory_reservation 12 Gig KiB Mem : 16434008 total, 1510628 free, 3130040 used, 11793340 buff/cache

Configuration details

dont know whats relevant, will provide it if needed...

Edited by Markus Siebert