two factor registration broken in gitlab 14.3.1
Summary
Two factor registration process fails after hitting "Register with two-factor app". The screen refreshes and no changes are made.
Steps to reproduce
Install a fresh copy or from scratch of gitlab-ce 14.3.1 omnibus. Configure LDAP setup (to active directory in our case) Login with new user account go to user settings > account Scan the barcode or enter the key in your MFA app (tried Duo Mobile, Google Auth and andOTP) Enter the generated pin code and your current password. Hit "Register with two-factor app" The page will refresh and a message appears at the top saying "you must provie a valid current password". The registration does not complete and users are not prompted for 2fa when logging in.
We've had several people try this, all with the same result, so we're fairly certain this is not a password issue, as our passwords worked when logging in.
What is the current bug behavior?
Registration page for two factor app does not work.
What is the expected correct behavior?
The two factor app setup should be registered with the users and account, it should indicate this on their profile and they should be prompted with 2fa login upon login.
Relevant logs and/or screenshots
The following is a tail from the server at the time the "register with two-factor app" button was hit.
gitlab-ctl tail
==> /var/log/gitlab/nginx/error.log <==
==> /var/log/gitlab/nginx/gitlab_error.log <==
2021/10/13 16:46:35 [crit] 100450#0: *2786 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 10.228.66.11, server: gitlabqa.it.ucf.edu, request: "GET /", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/"
==> /var/log/gitlab/nginx/access.log <==
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.228.66.11 - - [13/Oct/2021:17:15:49 -0400] "GET / -" 301 162 "" "-" -
10.228.66.11 - - [13/Oct/2021:17:15:50 -0400] "GET / -" 302 107 "" "-" -
10.228.66.10 - - [13/Oct/2021:17:15:52 -0400] "GET / -" 301 162 "" "-" -
10.228.66.10 - - [13/Oct/2021:17:15:53 -0400] "GET / -" 302 107 "" "-" -
10.228.66.11 - - [13/Oct/2021:17:15:54 -0400] "GET / -" 301 162 "" "-" -
10.228.66.11 - - [13/Oct/2021:17:15:55 -0400] "GET / -" 302 107 "" "-" -
10.228.66.10 - - [13/Oct/2021:17:15:57 -0400] "GET / -" 301 162 "" "-" -
10.228.66.10 - - [13/Oct/2021:17:15:58 -0400] "GET / -" 302 107 "" "-" -
10.228.66.11 - - [13/Oct/2021:17:15:59 -0400] "GET / -" 301 162 "" "-" -
10.228.66.11 - - [13/Oct/2021:17:16:00 -0400] "GET / -" 302 107 "" "-" -
==> /var/log/gitlab/nginx/current <==
==> /var/log/gitlab/grafana/current <==
2021-10-13_20:48:45.07932 t=2021-10-13T16:48:45-0400 lvl=info msg="Path Plugins" logger=settings path=/var/opt/gitlab/grafana/data/plugins
2021-10-13_20:48:45.07932 t=2021-10-13T16:48:45-0400 lvl=info msg="Path Provisioning" logger=settings path=/var/opt/gitlab/grafana/provisioning
2021-10-13_20:48:45.07933 t=2021-10-13T16:48:45-0400 lvl=info msg="App mode production" logger=settings
2021-10-13_20:48:45.09929 t=2021-10-13T16:48:45-0400 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3
2021-10-13_20:48:45.18798 t=2021-10-13T16:48:45-0400 lvl=info msg="Starting DB migrations" logger=migrator
2021-10-13_20:48:45.19097 t=2021-10-13T16:48:45-0400 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=330 duration=812.591µs
2021-10-13_20:48:45.20399 t=2021-10-13T16:48:45-0400 lvl=info msg="Starting plugin search" logger=plugins
2021-10-13_20:48:46.27041 t=2021-10-13T16:48:46-0400 lvl=eror msg="Failed to read plugin provisioning files from directory" logger=provisioning.plugins path=/var/opt/gitlab/grafana/provisioning/plugins error="open /var/opt/gitlab/grafana/provisioning/plugins: no such file or directory"
2021-10-13_20:48:46.27517 t=2021-10-13T16:48:46-0400 lvl=info msg="Live Push Gateway initialization" logger=live.push_http
2021-10-13_20:48:46.33821 t=2021-10-13T16:48:46-0400 lvl=info msg="HTTP Server Listen" logger=http.server address=127.0.0.1:3000 protocol=http subUrl=/-/grafana socket=
==> /var/log/gitlab/gitlab-exporter/current <==
2021-10-13_21:15:35.65884 ::1 - - [13/Oct/2021:17:15:35 EDT] "GET /database HTTP/1.1" 200 1114
2021-10-13_21:15:35.65886 - -> /database
2021-10-13_21:15:36.29933 ::1 - - [13/Oct/2021:17:15:36 EDT] "GET /sidekiq HTTP/1.1" 200 65682
2021-10-13_21:15:36.29934 - -> /sidekiq
2021-10-13_21:15:46.55007 ::1 - - [13/Oct/2021:17:15:46 EDT] "GET /ruby HTTP/1.1" 200 998
2021-10-13_21:15:46.55010 - -> /ruby
2021-10-13_21:15:50.66046 ::1 - - [13/Oct/2021:17:15:50 EDT] "GET /database HTTP/1.1" 200 1114
2021-10-13_21:15:50.66048 - -> /database
2021-10-13_21:15:51.32611 ::1 - - [13/Oct/2021:17:15:51 EDT] "GET /sidekiq HTTP/1.1" 200 65682
2021-10-13_21:15:51.32612 - -> /sidekiq
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5F7ZBCKD4F202435ES34","duration_ms":34,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:38-04:00","ttfb_ms":33,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5HA07VVK5FQCY7PP4NQ8","duration_ms":18,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:40-04:00","ttfb_ms":18,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5M353VHTWE68ENW76S07","duration_ms":31,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:43-04:00","ttfb_ms":31,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5P6DXPHWZ5GV4Q8CRTGP","duration_ms":20,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:45-04:00","ttfb_ms":20,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5S0H759VYS1XTJEZXBY6","duration_ms":18,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:48-04:00","ttfb_ms":18,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5V49GY7X9S71TKP7YP55","duration_ms":16,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:50-04:00","ttfb_ms":16,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR5XVRKVC6CXMM150B7A8S","duration_ms":17,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:53-04:00","ttfb_ms":17,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR600XW0GDA99S0M6RYCG5","duration_ms":22,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:55-04:00","ttfb_ms":22,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR62R4DCY3FRKVWMHY2RVW","duration_ms":25,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:15:58-04:00","ttfb_ms":25,"uri":"/","user_agent":"","written_bytes":107}
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR64VGJGAJWERT7WSJN7MW","duration_ms":46,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:16:00-04:00","ttfb_ms":45,"uri":"/","user_agent":"","written_bytes":107}
==> /var/log/gitlab/sidekiq/current <==
{"severity":"INFO","time":"2021-10-13T21:15:19.618Z","queue":"cronjob:schedule_merge_request_cleanup_refs","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"90fd77c5b1addcaf0f01cd40","created_at":"2021-10-13T21:15:19.606Z","meta.caller_id":"Cronjob","meta.feature_category":"code_review","correlation_id":"3e92a7169ac2aefc2e00f1edb5735d64","idempotency_key":"resque:gitlab:duplicate:cronjob:schedule_merge_request_cleanup_refs:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","enqueued_at":"2021-10-13T21:15:19.611Z","job_size_bytes":2,"pid":108530,"message":"ScheduleMergeRequestCleanupRefsWorker JID-90fd77c5b1addcaf0f01cd40: done: 0.003363 sec","job_status":"done","scheduling_latency_s":0.003973,"redis_calls":2,"redis_duration_s":0.00027899999999999995,"redis_read_bytes":213,"redis_write_bytes":282,"redis_cache_calls":1,"redis_cache_duration_s":0.000149,"redis_cache_read_bytes":203,"redis_cache_write_bytes":61,"redis_queues_calls":1,"redis_queues_duration_s":0.00013,"redis_queues_read_bytes":10,"redis_queues_write_bytes":221,"db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.001675,"mem_objects":980,"mem_bytes":155272,"mem_mallocs":220,"mem_total_bytes":194472,"duration_s":0.003363,"completed_at":"2021-10-13T21:15:19.618Z","db_duration_s":0.0}
{"severity":"INFO","time":"2021-10-13T21:15:19.627Z","queue":"cronjob:pages_domain_verification_cron","args":[],"class":"PagesDomainVerificationCronWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"015f4a878ee0ec3b462aee93","created_at":"2021-10-13T21:15:19.562Z","meta.caller_id":"Cronjob","meta.feature_category":"pages","correlation_id":"011af8b7b9d51899324310da9bdaf2a5","idempotency_key":"resque:gitlab:duplicate:cronjob:pages_domain_verification_cron:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","enqueued_at":"2021-10-13T21:15:19.580Z","job_size_bytes":2,"pid":108530,"message":"PagesDomainVerificationCronWorker JID-015f4a878ee0ec3b462aee93: done: 0.04573 sec","job_status":"done","scheduling_latency_s":0.000991,"redis_calls":2,"redis_duration_s":0.008067000000000001,"redis_read_bytes":213,"redis_write_bytes":384,"redis_cache_calls":1,"redis_cache_duration_s":0.007361,"redis_cache_read_bytes":203,"redis_cache_write_bytes":168,"redis_queues_calls":1,"redis_queues_duration_s":0.000706,"redis_queues_read_bytes":10,"redis_queues_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"cpu_s":0.004042,"mem_objects":1346,"mem_bytes":170512,"mem_mallocs":278,"mem_total_bytes":224352,"duration_s":0.04573,"completed_at":"2021-10-13T21:15:19.627Z","db_duration_s":0.011445}
{"severity":"INFO","time":"2021-10-13T21:15:19.634Z","queue":"cronjob:schedule_migrate_external_diffs","args":[],"class":"ScheduleMigrateExternalDiffsWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"6b057edd1817710c9674303b","created_at":"2021-10-13T21:15:19.626Z","meta.caller_id":"Cronjob","meta.feature_category":"code_review","correlation_id":"3b86f709c85d0db7b545fa251881a0fe","idempotency_key":"resque:gitlab:duplicate:cronjob:schedule_migrate_external_diffs:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","enqueued_at":"2021-10-13T21:15:19.633Z","job_size_bytes":2,"pid":108530,"message":"ScheduleMigrateExternalDiffsWorker JID-6b057edd1817710c9674303b: start","job_status":"start","scheduling_latency_s":0.001597}
{"severity":"INFO","time":"2021-10-13T21:15:19.669Z","queue":"cronjob:schedule_migrate_external_diffs","args":[],"class":"ScheduleMigrateExternalDiffsWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"6b057edd1817710c9674303b","created_at":"2021-10-13T21:15:19.626Z","meta.caller_id":"Cronjob","meta.feature_category":"code_review","correlation_id":"3b86f709c85d0db7b545fa251881a0fe","idempotency_key":"resque:gitlab:duplicate:cronjob:schedule_migrate_external_diffs:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","enqueued_at":"2021-10-13T21:15:19.633Z","job_size_bytes":2,"pid":108530,"message":"ScheduleMigrateExternalDiffsWorker JID-6b057edd1817710c9674303b: done: 0.034851 sec","job_status":"done","scheduling_latency_s":0.001597,"redis_calls":3,"redis_duration_s":0.006168,"redis_read_bytes":12,"redis_write_bytes":530,"redis_queues_calls":1,"redis_queues_duration_s":0.00059,"redis_queues_read_bytes":10,"redis_queues_write_bytes":217,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005578,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":313,"db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.001889,"mem_objects":984,"mem_bytes":169288,"mem_mallocs":179,"mem_total_bytes":208648,"duration_s":0.034851,"completed_at":"2021-10-13T21:15:19.669Z","db_duration_s":0.0}
{"severity":"INFO","time":"2021-10-13T21:15:19.670Z","queue":"cronjob:import_stuck_project_import_jobs","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"3ea77e475a5edf23673a7f9f","created_at":"2021-10-13T21:15:19.655Z","meta.caller_id":"Cronjob","meta.feature_category":"importers","correlation_id":"095eb4842bb7ec738b8e037da60d8f05","idempotency_key":"resque:gitlab:duplicate:cronjob:import_stuck_project_import_jobs:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","enqueued_at":"2021-10-13T21:15:19.662Z","job_size_bytes":2,"pid":108530,"message":"Gitlab::Import::StuckProjectImportJobsWorker JID-3ea77e475a5edf23673a7f9f: start","job_status":"start","scheduling_latency_s":0.007747}
{"severity":"INFO","time":"2021-10-13T21:15:19.709Z","queue":"cronjob:import_stuck_project_import_jobs","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"3ea77e475a5edf23673a7f9f","created_at":"2021-10-13T21:15:19.655Z","meta.caller_id":"Cronjob","meta.feature_category":"importers","correlation_id":"095eb4842bb7ec738b8e037da60d8f05","idempotency_key":"resque:gitlab:duplicate:cronjob:import_stuck_project_import_jobs:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","enqueued_at":"2021-10-13T21:15:19.662Z","job_size_bytes":2,"pid":108530,"message":"Gitlab::Import::StuckProjectImportJobsWorker JID-3ea77e475a5edf23673a7f9f: done: 0.03886 sec","job_status":"done","scheduling_latency_s":0.007747,"redis_calls":1,"redis_duration_s":0.00013,"redis_read_bytes":10,"redis_write_bytes":218,"redis_queues_calls":1,"redis_queues_duration_s":0.00013,"redis_queues_read_bytes":10,"redis_queues_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"cpu_s":0.004818,"mem_objects":1914,"mem_bytes":190936,"mem_mallocs":482,"mem_total_bytes":267496,"duration_s":0.03886,"completed_at":"2021-10-13T21:15:19.709Z","db_duration_s":0.025768}
{"severity":"INFO","time":"2021-10-13T21:15:19.713Z","queue":"cronjob:database_batched_background_migration","args":[],"class":"Database::BatchedBackgroundMigrationWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"2987098e3eefc944f2e98434","created_at":"2021-10-13T21:15:19.701Z","meta.caller_id":"Cronjob","meta.feature_category":"database","correlation_id":"b86ab17e72f86e1400b5adcbf569c2d5","idempotency_key":"resque:gitlab:duplicate:cronjob:database_batched_background_migration:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2021-10-13T21:15:19.712Z","job_size_bytes":2,"pid":108530,"message":"Database::BatchedBackgroundMigrationWorker JID-2987098e3eefc944f2e98434: start","job_status":"start","scheduling_latency_s":0.000986}
{"severity":"INFO","time":"2021-10-13T21:15:19.735Z","queue":"cronjob:user_status_cleanup_batch","args":[],"class":"UserStatusCleanup::BatchWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"dd6ae8d6bd5fdb7f211ea177","created_at":"2021-10-13T21:15:19.727Z","meta.caller_id":"Cronjob","meta.feature_category":"users","correlation_id":"8978bfa68a1fc741542b0f59084a02ff","idempotency_key":"resque:gitlab:duplicate:cronjob:user_status_cleanup_batch:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","enqueued_at":"2021-10-13T21:15:19.734Z","job_size_bytes":2,"pid":108530,"message":"UserStatusCleanup::BatchWorker JID-dd6ae8d6bd5fdb7f211ea177: start","job_status":"start","scheduling_latency_s":0.001164}
{"severity":"INFO","time":"2021-10-13T21:15:19.737Z","queue":"cronjob:database_batched_background_migration","args":[],"class":"Database::BatchedBackgroundMigrationWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"2987098e3eefc944f2e98434","created_at":"2021-10-13T21:15:19.701Z","meta.caller_id":"Cronjob","meta.feature_category":"database","correlation_id":"b86ab17e72f86e1400b5adcbf569c2d5","idempotency_key":"resque:gitlab:duplicate:cronjob:database_batched_background_migration:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2021-10-13T21:15:19.712Z","job_size_bytes":2,"pid":108530,"message":"Database::BatchedBackgroundMigrationWorker JID-2987098e3eefc944f2e98434: done: 0.024622 sec","job_status":"done","scheduling_latency_s":0.000986,"redis_calls":2,"redis_duration_s":0.000557,"redis_read_bytes":132,"redis_write_bytes":258,"redis_cache_calls":1,"redis_cache_duration_s":0.000355,"redis_cache_read_bytes":122,"redis_cache_write_bytes":35,"redis_queues_calls":1,"redis_queues_duration_s":0.000202,"redis_queues_read_bytes":10,"redis_queues_write_bytes":223,"db_count":1,"db_write_count":0,"db_cached_count":0,"cpu_s":0.003167,"mem_objects":1213,"mem_bytes":133888,"mem_mallocs":243,"mem_total_bytes":182408,"duration_s":0.024622,"completed_at":"2021-10-13T21:15:19.737Z","db_duration_s":0.006488}
{"severity":"INFO","time":"2021-10-13T21:15:19.746Z","queue":"cronjob:user_status_cleanup_batch","args":[],"class":"UserStatusCleanup::BatchWorker","retry":0,"version":0,"queue_namespace":"cronjob","jid":"dd6ae8d6bd5fdb7f211ea177","created_at":"2021-10-13T21:15:19.727Z","meta.caller_id":"Cronjob","meta.feature_category":"users","correlation_id":"8978bfa68a1fc741542b0f59084a02ff","idempotency_key":"resque:gitlab:duplicate:cronjob:user_status_cleanup_batch:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","enqueued_at":"2021-10-13T21:15:19.734Z","job_size_bytes":2,"pid":108530,"message":"UserStatusCleanup::BatchWorker JID-dd6ae8d6bd5fdb7f211ea177: done: 0.010176 sec","job_status":"done","scheduling_latency_s":0.001164,"redis_calls":1,"redis_duration_s":0.000146,"redis_read_bytes":10,"redis_write_bytes":211,"redis_queues_calls":1,"redis_queues_duration_s":0.000146,"redis_queues_read_bytes":10,"redis_queues_write_bytes":211,"db_count":1,"db_write_count":0,"db_cached_count":0,"cpu_s":0.002149,"mem_objects":928,"mem_bytes":100864,"mem_mallocs":156,"mem_total_bytes":137984,"duration_s":0.010176,"completed_at":"2021-10-13T21:15:19.745Z","db_duration_s":0.00134}
==> /var/log/gitlab/puma/puma_stdout.log <==
{"timestamp":"2021-10-13T21:12:59.018Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2056.02734375 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:13:19.019Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2054.5546875 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:13:39.020Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2056.23828125 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:13:59.021Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2056.5546875 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:14:19.022Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2054.671875 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:14:39.022Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2056.9609375 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:14:59.023Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2059.20703125 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:15:19.024Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2056.00390625 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:15:39.025Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2057.51953125 mb with master and 2 workers."}
{"timestamp":"2021-10-13T21:15:59.026Z","pid":108501,"message":"PumaWorkerKiller: Consuming 2060.1171875 mb with master and 2 workers."}
==> /var/log/gitlab/puma/current <==
2021-10-13_20:48:50.18948 {"timestamp":"2021-10-13T20:48:50.187Z","pid":108501,"message":"* Environment: production"}
2021-10-13_20:48:50.18948 {"timestamp":"2021-10-13T20:48:50.187Z","pid":108501,"message":"* Master PID: 108501"}
2021-10-13_20:48:50.18949 {"timestamp":"2021-10-13T20:48:50.187Z","pid":108501,"message":"* Workers: 2"}
2021-10-13_20:48:50.18949 {"timestamp":"2021-10-13T20:48:50.187Z","pid":108501,"message":"* Restarts: (✔) hot (✖) phased"}
2021-10-13_20:48:50.18949 {"timestamp":"2021-10-13T20:48:50.188Z","pid":108501,"message":"* Preloading application"}
2021-10-13_20:49:58.58900 {"timestamp":"2021-10-13T20:49:58.588Z","pid":108501,"message":"* Listening on unix:///var/opt/gitlab/gitlab-rails/sockets/gitlab.socket"}
2021-10-13_20:49:58.58927 {"timestamp":"2021-10-13T20:49:58.589Z","pid":108501,"message":"* Listening on http://127.0.0.1:8080"}
2021-10-13_20:49:58.59052 {"timestamp":"2021-10-13T20:49:58.589Z","pid":108501,"message":"! WARNING: Detected 1 Thread(s) started in app boot:"}
2021-10-13_20:49:58.59053 {"timestamp":"2021-10-13T20:49:58.589Z","pid":108501,"message":"! #\u003cThread:0x00007f8da5418b08 /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/support/scheduler.rb:73 sleep\u003e - /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rack-timeout-0.5.2/lib/rack/timeout/support/scheduler.rb:91:in `sleep'"}
2021-10-13_20:49:58.59054 {"timestamp":"2021-10-13T20:49:58.589Z","pid":108501,"message":"Use Ctrl-C to stop"}
==> /var/log/gitlab/puma/puma_stderr.log <==
=== puma startup: 2021-10-13 16:04:18 -0400 ===
=== puma startup: 2021-10-13 16:10:12 -0400 ===
=== puma startup: 2021-10-13 16:49:58 -0400 ===
unknown OID 28: failed to recognize type of 'relfrozenxid'. It will be treated as String.
unknown OID 1034: failed to recognize type of 'relacl'. It will be treated as String.
unknown OID 194: failed to recognize type of 'relpartbound'. It will be treated as String.
unknown OID 28: failed to recognize type of 'relfrozenxid'. It will be treated as String.
unknown OID 1034: failed to recognize type of 'relacl'. It will be treated as String.
unknown OID 194: failed to recognize type of 'relpartbound'. It will be treated as String.
==> /var/log/gitlab/logrotate/current <==
2021-10-13_20:46:36.01963 Received TERM from runit, sending to process group (-PID)
==> /var/log/gitlab/gitaly/gitaly_ruby_json.log <==
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:14:49Z","grpc.time_ms":0.18,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"abf6913ab8dae10c5931cda305d0f012","time":"2021-10-13T21:14:49.054Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:14:49Z","grpc.time_ms":0.228,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"e98ba936c2c6dcf228a6fee8716db854","time":"2021-10-13T21:14:49.057Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:04Z","grpc.time_ms":0.196,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"1ad0765b040f467f621287fbab3f353f","time":"2021-10-13T21:15:04.056Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:04Z","grpc.time_ms":0.214,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"fe3ec9a9a896601452a0a438df39af77","time":"2021-10-13T21:15:04.059Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:19Z","grpc.time_ms":0.243,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"36a45f62aa0ba12d36aa5ebcac91af64","time":"2021-10-13T21:15:19.058Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:19Z","grpc.time_ms":0.161,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"6ce65264c7857a325e9a17039b40123e","time":"2021-10-13T21:15:19.061Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:34Z","grpc.time_ms":0.283,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"12bacb872459f4b0edd8cba739050d91","time":"2021-10-13T21:15:34.062Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:34Z","grpc.time_ms":0.343,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"f7d0101ac1cfc24bd057f88b093a160d","time":"2021-10-13T21:15:34.062Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:49Z","grpc.time_ms":0.181,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"daf4fb68df2ce1cb52eef4a7a269e928","time":"2021-10-13T21:15:49.064Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:15:49Z","grpc.time_ms":0.166,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"aa1bd8bceb5dca35abf43c29dcb93122","time":"2021-10-13T21:15:49.065Z"}
==> /var/log/gitlab/gitaly/current <==
{"level":"info","msg":"Gitaly stopped","time":"2021-10-13T16:48:48.741Z"}
{"gitaly":100443,"level":"error","msg":"wrapper for gitaly shutting down","time":"2021-10-13T16:48:49.270Z","wrapper":100759}
{"level":"info","msg":"Wrapper started","time":"2021-10-13T16:48:49.365Z","wrapper":108533}
{"level":"info","msg":"finding gitaly","pid_file":"/var/opt/gitlab/gitaly/gitaly.pid","time":"2021-10-13T16:48:49.366Z","wrapper":108533}
{"level":"info","msg":"adopting a process","time":"2021-10-13T16:48:49.434Z","wrapper":108533}
{"gitaly":108425,"level":"info","msg":"monitoring gitaly","time":"2021-10-13T16:48:49.434Z","wrapper":108533}
{"level":"info","msg":"PID 108522 BUNDLE_GEMFILE=/opt/gitlab/embedded/service/gitaly-ruby/Gemfile","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","108425","/var/opt/gitlab/gitaly/internal_sockets/ruby.0"],"supervisor.name":"gitaly-ruby.0","time":"2021-10-13T16:48:49.731Z"}
{"level":"info","msg":"PID 108524 BUNDLE_GEMFILE=/opt/gitlab/embedded/service/gitaly-ruby/Gemfile","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","108425","/var/opt/gitlab/gitaly/internal_sockets/ruby.1"],"supervisor.name":"gitaly-ruby.1","time":"2021-10-13T16:48:50.391Z"}
{"correlation_id":"01FHXQ12Z4SYY25B2Y91HZTZFJ","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"unary","grpc.method":"RepositoryExists","grpc.request.deadline":"2021-10-13T16:55:56.011Z","grpc.request.fullMethod":"/gitaly.RepositoryService/RepositoryExists","grpc.request.glProjectPath":"gitlab-instance-ceccd68b/Monitoring","grpc.request.glRepository":"project-1","grpc.request.repoPath":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git","grpc.request.repoStorage":"default","grpc.service":"gitaly.RepositoryService","grpc.start_time":"2021-10-13T16:55:46.400Z","grpc.time_ms":0.329,"level":"info","msg":"finished unary call with code OK","peer.address":"@","pid":108425,"remote_ip":"10.228.66.12","span.kind":"server","system":"grpc","time":"2021-10-13T16:55:46.400Z","username":"root"}
{"correlation_id":"01FHXQ12Z4SYY25B2Y91HZTZFJ","error":"rpc error: code = NotFound desc = GetRepoPath: not a git repository: \"/data/gitlab/git-data/repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git\"","grpc.code":"NotFound","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"unary","grpc.method":"FindDefaultBranchName","grpc.request.deadline":"2021-10-13T16:55:56.008Z","grpc.request.fullMethod":"/gitaly.RefService/FindDefaultBranchName","grpc.request.glProjectPath":"gitlab-instance-ceccd68b/Monitoring","grpc.request.glRepository":"project-1","grpc.request.repoPath":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git","grpc.request.repoStorage":"default","grpc.service":"gitaly.RefService","grpc.start_time":"2021-10-13T16:55:46.439Z","grpc.time_ms":2.499,"level":"info","msg":"finished unary call with code NotFound","peer.address":"@","pid":108425,"remote_ip":"10.228.66.12","span.kind":"server","system":"grpc","time":"2021-10-13T16:55:46.441Z","username":"root"}
==> /var/log/gitlab/redis/current <==
2021-10-13_21:10:48.06802 95962:M 13 Oct 2021 17:10:48.067 * 10 changes in 300 seconds. Saving...
2021-10-13_21:10:48.06930 95962:M 13 Oct 2021 17:10:48.068 * Background saving started by pid 112735
2021-10-13_21:10:48.09500 112735:C 13 Oct 2021 17:10:48.094 * DB saved on disk
2021-10-13_21:10:48.09502 112735:C 13 Oct 2021 17:10:48.094 * RDB: 0 MB of memory used by copy-on-write
2021-10-13_21:10:48.16863 95962:M 13 Oct 2021 17:10:48.168 * Background saving terminated with success
2021-10-13_21:15:49.09984 95962:M 13 Oct 2021 17:15:49.099 * 10 changes in 300 seconds. Saving...
2021-10-13_21:15:49.09986 95962:M 13 Oct 2021 17:15:49.099 * Background saving started by pid 113596
2021-10-13_21:15:49.11272 113596:C 13 Oct 2021 17:15:49.110 * DB saved on disk
2021-10-13_21:15:49.11273 113596:C 13 Oct 2021 17:15:49.110 * RDB: 0 MB of memory used by copy-on-write
2021-10-13_21:15:49.20108 95962:M 13 Oct 2021 17:15:49.200 * Background saving terminated with success
==> /var/log/gitlab/postgres-exporter/current <==
2021-10-13_20:03:45.09920 time="2021-10-13T16:03:45-04:00" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1850"
2021-10-13_20:03:45.42771 time="2021-10-13T16:03:45-04:00" level=info msg="Established new database connection to \"/var/opt/gitlab/postgresql:5432\"." source="postgres_exporter.go:983"
2021-10-13_20:03:45.44846 time="2021-10-13T16:03:45-04:00" level=info msg="Semantic Version Changed on \"/var/opt/gitlab/postgresql:5432\": 0.0.0 -> 12.7.0" source="postgres_exporter.go:1552"
2021-10-13_20:07:20.56638 time="2021-10-13T16:07:20-04:00" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1850"
2021-10-13_20:07:30.43131 time="2021-10-13T16:07:30-04:00" level=info msg="Established new database connection to \"/var/opt/gitlab/postgresql:5432\"." source="postgres_exporter.go:983"
2021-10-13_20:07:30.44344 time="2021-10-13T16:07:30-04:00" level=info msg="Semantic Version Changed on \"/var/opt/gitlab/postgresql:5432\": 0.0.0 -> 12.7.0" source="postgres_exporter.go:1552"
2021-10-13_20:48:46.52097 time="2021-10-13T16:48:46-04:00" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1850"
2021-10-13_20:49:00.42527 time="2021-10-13T16:49:00-04:00" level=info msg="Established new database connection to \"/var/opt/gitlab/postgresql:5432\"." source="postgres_exporter.go:983"
2021-10-13_20:49:00.44494 time="2021-10-13T16:49:00-04:00" level=info msg="Semantic Version Changed on \"/var/opt/gitlab/postgresql:5432\": 0.0.0 -> 12.7.0" source="postgres_exporter.go:1552"
==> /var/log/gitlab/node-exporter/current <==
2021-10-13_20:48:46.18820 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=thermal_zone
2021-10-13_20:48:46.18820 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=time
2021-10-13_20:48:46.18821 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=timex
2021-10-13_20:48:46.18821 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=udp_queues
2021-10-13_20:48:46.18821 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=uname
2021-10-13_20:48:46.18822 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=vmstat
2021-10-13_20:48:46.18822 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=xfs
2021-10-13_20:48:46.18823 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:113 collector=zfs
2021-10-13_20:48:46.18824 level=info ts=2021-10-13T20:48:46.183Z caller=node_exporter.go:195 msg="Listening on" address=localhost:9100
2021-10-13_20:48:46.21072 level=info ts=2021-10-13T20:48:46.207Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
==> /var/log/gitlab/prometheus/current <==
2021-10-13_20:48:48.27086 level=info ts=2021-10-13T20:48:48.270Z caller=main.go:802 msg="TSDB started"
2021-10-13_20:48:48.27087 level=info ts=2021-10-13T20:48:48.270Z caller=main.go:928 msg="Loading configuration file" filename=/var/opt/gitlab/prometheus/prometheus.yml
2021-10-13_20:48:48.28108 level=error ts=2021-10-13T20:48:48.279Z caller=manager.go:314 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=kubernetes
2021-10-13_20:48:48.28111 level=error ts=2021-10-13T20:48:48.279Z caller=manager.go:314 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=kubernetes
2021-10-13_20:48:48.28111 level=error ts=2021-10-13T20:48:48.279Z caller=manager.go:314 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=kubernetes
2021-10-13_20:48:48.30626 level=info ts=2021-10-13T20:48:48.306Z caller=main.go:959 msg="Completed loading of configuration file" filename=/var/opt/gitlab/prometheus/prometheus.yml totalDuration=35.360008ms remote_storage=2.21µs web_handler=556ns query_engine=1.09µs scrape=4.725098ms scrape_sd=512.219µs notify=38.875µs notify_sd=17.07µs rules=26.417307ms
2021-10-13_20:48:48.30627 level=info ts=2021-10-13T20:48:48.306Z caller=main.go:751 msg="Server is ready to receive web requests."
2021-10-13_20:48:53.30710 level=error ts=2021-10-13T20:48:53.306Z caller=manager.go:188 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-cadvisor
2021-10-13_20:48:53.30712 level=error ts=2021-10-13T20:48:53.306Z caller=manager.go:188 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-nodes
2021-10-13_20:48:53.30712 level=error ts=2021-10-13T20:48:53.306Z caller=manager.go:188 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-pods
==> /var/log/gitlab/postgresql/current <==
2021-10-13_20:08:22.57231 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
2021-10-13_20:47:13.93917 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2021-10-13_20:47:13.93966 FATAL: Peer authentication failed for user "gitlab"
2021-10-13_20:47:13.93967 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
2021-10-13_20:47:14.11236 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2021-10-13_20:47:14.11237 FATAL: Peer authentication failed for user "gitlab"
2021-10-13_20:47:14.11238 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
2021-10-13_20:47:14.28172 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2021-10-13_20:47:14.28174 FATAL: Peer authentication failed for user "gitlab"
2021-10-13_20:47:14.28174 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2021-10-13-16-47-15.log <==
-- indexes(:users)
-> 0.0109s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- remove_index(:users, {:algorithm=>:concurrently, :name=>"idx_users_on_user_type_project_bots_batched"})
-> 0.0083s
-- execute("RESET statement_timeout")
-> 0.0003s
== 20210914095310 CleanupOrphanProjectAccessTokens: migrated (0.1470s) ========
==> /var/log/gitlab/gitlab-rails/grpc.log <==
# Logfile created on 2021-10-13 16:01:31 -0400 by logger.rb/v1.4.2
==> /var/log/gitlab/gitlab-rails/application_json.log <==
{"severity":"INFO","time":"2021-10-13T21:02:11.285Z","correlation_id":"01FHXQCV6Q5RM1BPGH63M8SHWH","message":"Successful Login: username=root ip=10.228.66.12 method=standard admin=true"}
{"severity":"INFO","time":"2021-10-13T21:02:20.036Z","correlation_id":"01FHXQD3M9CPYRGWKJBMHBKR0S","message":"Emails disabled! Interceptor prevented sending mail Two-factor authentication disabled"}
{"severity":"INFO","time":"2021-10-13T21:02:27.262Z","correlation_id":"01FHXQDAWYNT865B6QCHGRPJ6Q","message":"User Logout: username=su324177 ip=10.228.66.12"}
{"severity":"INFO","time":"2021-10-13T21:02:34.324Z","correlation_id":"01FHXQDHT4HSP5HKNFWAZ6RCV7","message":"(ldapmain) Callback phase initiated."}
{"severity":"INFO","time":"2021-10-13T21:02:34.411Z","correlation_id":"01FHXQDHT4HSP5HKNFWAZ6RCV7","message":"(LDAP) saving user Sunghyeon.Kim@ucf.edu from login with admin =\u003e false, extern_uid =\u003e cn=su324177,ou=people,dc=net,dc=ucf,dc=edu"}
{"severity":"DEBUG","time":"2021-10-13T21:02:34.614Z","correlation_id":"01FHXQDHT4HSP5HKNFWAZ6RCV7","message":"Instantiating Gitlab::Auth::Ldap::Person with LDIF:\ndn: CN=su324177,OU=People,DC=net,DC=ucf,DC=edu\ncn: su324177\nmail: Sunghyeon.Kim@ucf.edu\nsamaccountname: su324177\nuserprincipalname: su324177@ucf.edu\n"}
{"severity":"INFO","time":"2021-10-13T21:08:54.021Z","correlation_id":"01FHXQS4JT818Y22GR451RDW8T","message":"User Logout: username=root ip=10.228.66.12"}
{"severity":"INFO","time":"2021-10-13T21:08:58.943Z","correlation_id":"01FHXQS9D5TV4ASJAKPWTSRZK5","message":"(ldapmain) Callback phase initiated."}
{"severity":"INFO","time":"2021-10-13T21:08:59.511Z","correlation_id":"01FHXQS9D5TV4ASJAKPWTSRZK5","message":"(LDAP) saving user Jason.Gilfoil@ucf.edu from login with admin =\u003e false, extern_uid =\u003e cn=jgilfoil,ou=people,dc=net,dc=ucf,dc=edu"}
{"severity":"DEBUG","time":"2021-10-13T21:08:59.535Z","correlation_id":"01FHXQS9D5TV4ASJAKPWTSRZK5","message":"Instantiating Gitlab::Auth::Ldap::Person with LDIF:\ndn: CN=jgilfoil,OU=People,DC=net,DC=ucf,DC=edu\ncn: jgilfoil\nmail: Jason.Gilfoil@ucf.edu\nsamaccountname: jgilfoil\nuserprincipalname: jgilfoil@ucf.edu\n"}
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:40.242Z","params":[],"remote_ip":"10.228.66.11","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5HA07VVK5FQCY7PP4NQ8","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.11","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.11","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.01378,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108812,"queue_duration_s":0.00443,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00687}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:43.107Z","params":[],"remote_ip":"10.228.66.10","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5M353VHTWE68ENW76S07","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.10","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.10","redis_calls":5,"redis_duration_s":0.001323,"redis_read_bytes":933,"redis_write_bytes":310,"redis_cache_calls":5,"redis_cache_duration_s":0.001323,"redis_cache_read_bytes":933,"redis_cache_write_bytes":310,"db_count":1,"db_write_count":0,"db_cached_count":0,"cpu_s":0.026876,"mem_objects":12076,"mem_bytes":1878499,"mem_mallocs":2966,"mem_total_bytes":2361539,"pid":108814,"queue_duration_s":0.016758,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.0094}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:45.249Z","params":[],"remote_ip":"10.228.66.11","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5P6DXPHWZ5GV4Q8CRTGP","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.11","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.11","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.016901,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108812,"queue_duration_s":0.00606,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00719}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:48.130Z","params":[],"remote_ip":"10.228.66.10","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5S0H759VYS1XTJEZXBY6","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.10","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.10","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.015553,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108814,"queue_duration_s":0.004619,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00709}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2021-10-13T21:15:49.931Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.012123,"mem_objects":1608,"mem_bytes":253896,"mem_mallocs":1705,"mem_total_bytes":318216,"pid":108814,"correlation_id":"55c585b0-9b4c-45e8-8075-927d36741daf","db_duration_s":0.0,"view_duration_s":0.0006,"duration_s":0.01077}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:50.296Z","params":[],"remote_ip":"10.228.66.11","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5V49GY7X9S71TKP7YP55","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.11","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.11","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.014166,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108812,"queue_duration_s":0.00381,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00664}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:53.097Z","params":[],"remote_ip":"10.228.66.10","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR5XVRKVC6CXMM150B7A8S","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.10","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.10","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.014185,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108812,"queue_duration_s":0.00442,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00675}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:55.314Z","params":[],"remote_ip":"10.228.66.11","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR600XW0GDA99S0M6RYCG5","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.11","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.11","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.015133,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108814,"queue_duration_s":0.004511,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00736}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:15:58.108Z","params":[],"remote_ip":"10.228.66.10","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR62R4DCY3FRKVWMHY2RVW","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.10","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.10","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.01394,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108814,"queue_duration_s":0.012813,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00658}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:16:00.284Z","params":[],"remote_ip":"10.228.66.11","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR64VGJGAJWERT7WSJN7MW","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.11","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.11","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.020894,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108814,"queue_duration_s":0.02773,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.01085}
==> /var/log/gitlab/gitlab-rails/graphql_json.log <==
# Logfile created on 2021-10-13 16:55:55 -0400 by logger.rb/v1.4.2
{"severity":"INFO","time":"2021-10-13T20:55:55.085Z","correlation_id":"01FHXQ1BXG52WRGN3VS08A0TNS","query_string":"query getUsersGroupCounts($usernames: [String!]) {\n users(usernames: $usernames) {\n nodes {\n id\n groupCount\n __typename\n }\n __typename\n }\n}\n","variables":"{\"usernames\"=\u003e[\"su324177\", \"jgilfoil\", \"root\"]}","duration_s":0.0,"operation_name":"getUsersGroupCounts","depth":3,"complexity":14,"used_fields":["UserCore.id","UserCore.groupCount","UserCore.__typename","UserCoreConnection.nodes","UserCoreConnection.__typename","Query.users"],"used_deprecated_fields":[]}
{"severity":"INFO","time":"2021-10-13T21:02:16.570Z","correlation_id":"01FHXQD0EPZW2JJPHWWV1WWWY9","query_string":"query getUsersGroupCounts($usernames: [String!]) {\n users(usernames: $usernames) {\n nodes {\n id\n groupCount\n __typename\n }\n __typename\n }\n}\n","variables":"{\"usernames\"=\u003e[\"su324177\", \"jgilfoil\", \"root\"]}","duration_s":0.0,"operation_name":"getUsersGroupCounts","depth":3,"complexity":14,"used_fields":["UserCore.id","UserCore.groupCount","UserCore.__typename","UserCoreConnection.nodes","UserCoreConnection.__typename","Query.users"],"used_deprecated_fields":[]}
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2021-10-13-16-00-56.log <==
OK
== Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/010_settings.rb
Saved CI JWT signing key
== Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/998_gitlab_instance_administration_project.rb
Successfully created self monitoring project.
== Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/999_common_metrics.rb
==> /var/log/gitlab/gitlab-rails/auth.log <==
{"severity":"INFO","time":"2021-10-13T20:07:05.889Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:09:20.259Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:09:20.265Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:09:20.270Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:41:43.006Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:42:36.356Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:47:54.758Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:48:37.034Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:49:15.512Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2021-10-13T20:49:17.421Z","correlation_id":null,"gitlab_throttle_user_allowlist":[]}
==> /var/log/gitlab/gitlab-rails/application.log <==
2021-10-13T21:08:54.013Z: User Logout: username=root ip=10.228.66.12
2021-10-13T21:08:58.938Z: (ldapmain) Callback phase initiated.
2021-10-13T21:08:59.511Z: (LDAP) saving user Jason.Gilfoil@ucf.edu from login with admin => false, extern_uid => cn=jgilfoil,ou=people,dc=net,dc=ucf,dc=edu
2021-10-13T21:08:59.534Z: Instantiating Gitlab::Auth::Ldap::Person with LDIF:
dn: CN=jgilfoil,OU=People,DC=net,DC=ucf,DC=edu
cn: jgilfoil
mail: Jason.Gilfoil@ucf.edu
samaccountname: jgilfoil
userprincipalname: jgilfoil@ucf.edu
==> /var/log/gitlab/gitlab-rails/backup_json.log <==
{"severity":"INFO","time":"2021-10-13T20:42:44.806Z","correlation_id":null,"message":"Dumping builds ... "}
{"severity":"INFO","time":"2021-10-13T20:42:44.827Z","correlation_id":null,"message":"[SKIPPED]"}
{"severity":"INFO","time":"2021-10-13T20:42:44.835Z","correlation_id":null,"message":"Dumping artifacts ... "}
{"severity":"INFO","time":"2021-10-13T20:42:44.860Z","correlation_id":null,"message":"[SKIPPED]"}
{"severity":"INFO","time":"2021-10-13T20:42:44.875Z","correlation_id":null,"message":"Dumping pages ... "}
{"severity":"INFO","time":"2021-10-13T20:42:44.904Z","correlation_id":null,"message":"[SKIPPED]"}
{"severity":"INFO","time":"2021-10-13T20:42:44.913Z","correlation_id":null,"message":"Dumping lfs objects ... "}
{"severity":"INFO","time":"2021-10-13T20:42:44.938Z","correlation_id":null,"message":"[SKIPPED]"}
{"severity":"INFO","time":"2021-10-13T20:42:44.958Z","correlation_id":null,"message":"Dumping container registry images ... "}
{"severity":"INFO","time":"2021-10-13T20:42:44.984Z","correlation_id":null,"message":"[DISABLED]"}
==> /var/log/gitlab/gitlab-rails/audit_json.log <==
# Logfile created on 2021-10-13 16:33:55 -0400 by logger.rb/v1.4.2
{"severity":"INFO","time":"2021-10-13T20:33:55.655Z","correlation_id":"01FHXNS2P13D4K19B8STX3AR09","author_id":2,"author_name":"jgilfoil","entity_id":2,"entity_type":"User","with":"ldapmain","target_id":2,"target_type":"User","target_details":"jgilfoil"}
{"severity":"INFO","time":"2021-10-13T20:33:57.721Z","correlation_id":"01FHXNS55PQFN7B0S057Q5E5JK","author_id":3,"author_name":"su324177","entity_id":3,"entity_type":"User","with":"ldapmain","target_id":3,"target_type":"User","target_details":"su324177"}
{"severity":"INFO","time":"2021-10-13T20:55:45.808Z","correlation_id":"01FHXQ12QEFNDR2JPYS9045BGH","author_id":1,"author_name":"Administrator","entity_id":1,"entity_type":"User","with":"standard","target_id":1,"target_type":"User","target_details":"Administrator"}
{"severity":"INFO","time":"2021-10-13T20:56:17.984Z","correlation_id":"01FHXQ21R9K947PS3597FR96ZF","author_id":2,"author_name":"jgilfoil","entity_id":2,"entity_type":"User","with":"ldapmain","target_id":2,"target_type":"User","target_details":"jgilfoil"}
{"severity":"INFO","time":"2021-10-13T21:02:11.287Z","correlation_id":"01FHXQCV6Q5RM1BPGH63M8SHWH","author_id":1,"author_name":"Administrator","entity_id":1,"entity_type":"User","with":"standard","target_id":1,"target_type":"User","target_details":"Administrator"}
{"severity":"INFO","time":"2021-10-13T21:02:34.654Z","correlation_id":"01FHXQDHT4HSP5HKNFWAZ6RCV7","author_id":3,"author_name":"su324177","entity_id":3,"entity_type":"User","with":"ldapmain","target_id":3,"target_type":"User","target_details":"su324177"}
{"severity":"INFO","time":"2021-10-13T21:08:59.549Z","correlation_id":"01FHXQS9D5TV4ASJAKPWTSRZK5","author_id":2,"author_name":"jgilfoil","entity_id":2,"entity_type":"User","with":"ldapmain","target_id":2,"target_type":"User","target_details":"jgilfoil"}
==> /var/log/gitlab/gitlab-rails/exceptions_json.log <==
# Logfile created on 2021-10-13 16:02:08 -0400 by logger.rb/v1.4.2
{"severity":"ERROR","time":"2021-10-13T20:02:08.696Z","correlation_id":"d9330aee4df761d927329628ffb0048c","exception.class":"Sidekiq::Worker::EnqueueFromTransactionError","exception.message":"`AuthorizedProjectUpdate::ProjectCreateWorker.perform_async` cannot be called inside a transaction as this can lead to\nrace conditions when the worker runs before the transaction is committed and\ntries to access a model that has not been saved yet.\n\nUse an `after_commit` hook, or include `AfterCommitQueue` and use a `run_after_commit` block instead.\n","exception.backtrace":["config/initializers/forbid_sidekiq_in_transactions.rb:25:in `block (2 levels) in \u003cmodule:NoEnqueueingFromTransactions\u003e'","app/workers/concerns/application_worker.rb:94:in `perform_async'","app/services/projects/create_service.rb:137:in `setup_authorizations'","app/services/projects/create_service.rb:114:in `after_create_actions'","app/services/projects/create_service.rb:70:in `block in execute'","lib/gitlab/application_context.rb:74:in `block in use'","lib/gitlab/application_context.rb:74:in `use'","lib/gitlab/application_context.rb:27:in `with_context'","app/services/projects/create_service.rb:69:in `execute'","app/services/concerns/measurable.rb:35:in `execute'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:59:in `create_project'","app/models/concerns/stepable.rb:14:in `call'","app/models/concerns/stepable.rb:14:in `block in execute_steps'","app/models/concerns/stepable.rb:13:in `each'","app/models/concerns/stepable.rb:13:in `inject'","app/models/concerns/stepable.rb:13:in `execute_steps'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:27:in `execute'","(eval):3:in `block (2 levels) in run_file'","lib/gitlab/database.rb:260:in `block in transaction'","lib/gitlab/database.rb:259:in `transaction'","lib/tasks/gitlab/db.rake:66:in `block (3 levels) in \u003ctop (required)\u003e'"],"user.username":null,"tags.program":"web","tags.locale":"en","tags.feature_category":null,"tags.correlation_id":"d9330aee4df761d927329628ffb0048c"}
{"severity":"ERROR","time":"2021-10-13T20:02:08.749Z","correlation_id":"d9330aee4df761d927329628ffb0048c","exception.class":"Sidekiq::Worker::EnqueueFromTransactionError","exception.message":"`Projects::PostCreationWorker.perform_async` cannot be called inside a transaction as this can lead to\nrace conditions when the worker runs before the transaction is committed and\ntries to access a model that has not been saved yet.\n\nUse an `after_commit` hook, or include `AfterCommitQueue` and use a `run_after_commit` block instead.\n","exception.backtrace":["config/initializers/forbid_sidekiq_in_transactions.rb:25:in `block (2 levels) in \u003cmodule:NoEnqueueingFromTransactions\u003e'","app/workers/concerns/application_worker.rb:94:in `perform_async'","app/services/projects/create_service.rb:118:in `after_create_actions'","app/services/projects/create_service.rb:70:in `block in execute'","lib/gitlab/application_context.rb:74:in `block in use'","lib/gitlab/application_context.rb:74:in `use'","lib/gitlab/application_context.rb:27:in `with_context'","app/services/projects/create_service.rb:69:in `execute'","app/services/concerns/measurable.rb:35:in `execute'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:59:in `create_project'","app/models/concerns/stepable.rb:14:in `call'","app/models/concerns/stepable.rb:14:in `block in execute_steps'","app/models/concerns/stepable.rb:13:in `each'","app/models/concerns/stepable.rb:13:in `inject'","app/models/concerns/stepable.rb:13:in `execute_steps'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:27:in `execute'","(eval):3:in `block (2 levels) in run_file'","lib/gitlab/database.rb:260:in `block in transaction'","lib/gitlab/database.rb:259:in `transaction'","lib/tasks/gitlab/db.rake:66:in `block (3 levels) in \u003ctop (required)\u003e'"],"user.username":null,"tags.program":"web","tags.locale":"en","tags.feature_category":null,"tags.correlation_id":"d9330aee4df761d927329628ffb0048c"}
{"severity":"ERROR","time":"2021-10-13T20:02:09.274Z","correlation_id":"d9330aee4df761d927329628ffb0048c","exception.class":"Gitlab::Git::PreReceiveError","exception.message":"Internal API unreachable","exception.backtrace":["lib/gitlab/gitaly_client/operation_service.rb:351:in `user_commit_files'","lib/gitlab/git/repository.rb:895:in `block in multi_action'","lib/gitlab/git/wraps_gitaly_errors.rb:7:in `wrapped_gitaly_errors'","lib/gitlab/git/repository.rb:894:in `multi_action'","app/models/repository.rb:827:in `block in multi_action'","app/models/repository.rb:810:in `with_cache_hooks'","app/models/repository.rb:827:in `multi_action'","app/models/repository.rb:791:in `create_file'","app/services/files/create_service.rb:16:in `create_transformed_commit'","app/services/files/create_service.rb:10:in `create_commit!'","app/services/commits/create_service.rb:30:in `execute'","app/services/projects/create_service.rb:160:in `create_readme'","app/services/projects/create_service.rb:120:in `after_create_actions'","app/services/projects/create_service.rb:70:in `block in execute'","lib/gitlab/application_context.rb:74:in `block in use'","lib/gitlab/application_context.rb:74:in `use'","lib/gitlab/application_context.rb:27:in `with_context'","app/services/projects/create_service.rb:69:in `execute'","app/services/concerns/measurable.rb:35:in `execute'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:59:in `create_project'","app/models/concerns/stepable.rb:14:in `call'","app/models/concerns/stepable.rb:14:in `block in execute_steps'","app/models/concerns/stepable.rb:13:in `each'","app/models/concerns/stepable.rb:13:in `inject'","app/models/concerns/stepable.rb:13:in `execute_steps'","lib/gitlab/database_importers/self_monitoring/project/create_service.rb:27:in `execute'","(eval):3:in `block (2 levels) in run_file'","lib/gitlab/database.rb:260:in `block in transaction'","lib/gitlab/database.rb:259:in `transaction'","lib/tasks/gitlab/db.rake:66:in `block (3 levels) in \u003ctop (required)\u003e'"],"user.username":null,"tags.program":"web","tags.locale":"en","tags.feature_category":null,"tags.correlation_id":"d9330aee4df761d927329628ffb0048c"}
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/" for 10.228.66.10 at 2021-10-13 17:15:58 -0400
Processing by RootController#index as HTML
Redirected to https://gitlabqa.it.ucf.edu/users/sign_in
Filter chain halted as :redirect_unlogged_user rendered or redirected
Completed 302 Found in 6ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 3149)
Started GET "/" for 10.228.66.11 at 2021-10-13 17:16:00 -0400
Processing by RootController#index as HTML
Redirected to https://gitlabqa.it.ucf.edu/users/sign_in
Filter chain halted as :redirect_unlogged_user rendered or redirected
Completed 302 Found in 11ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 3149)
==> /var/log/gitlab/gitlab-rails/service_measurement.log <==
# Logfile created on 2021-10-13 16:01:31 -0400 by logger.rb/v1.4.2
==> /var/log/gitlab/gitlab-rails/api_json.log <==
# Logfile created on 2021-10-13 16:01:32 -0400 by logger.rb/v1.4.2
{"time":"2021-10-13T20:55:50.976Z","severity":"INFO","duration_s":0.12357,"db_duration_s":0.06469,"view_duration_s":0.05888,"status":200,"method":"GET","path":"/api/v4/application/statistics","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/application/statistics","user_id":1,"username":"root","queue_duration_s":0.038507,"redis_calls":1,"redis_duration_s":0.000284,"redis_read_bytes":181,"redis_write_bytes":746,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":746,"db_count":20,"db_write_count":4,"db_cached_count":0,"cpu_s":0.064061,"mem_objects":18385,"mem_bytes":3526928,"mem_mallocs":6301,"mem_total_bytes":4262328,"pid":108814,"correlation_id":"01FHXQ17RYH4M4XHGQAJY8Y9FQ","meta.user":"root","meta.caller_id":"GET /api/:version/application/statistics","meta.remote_ip":"10.228.66.12","meta.feature_category":"devops_reports","meta.client_id":"user/1"}
{"time":"2021-10-13T20:55:58.740Z","severity":"INFO","duration_s":0.02993,"db_duration_s":0.0019,"view_duration_s":0.02803,"status":200,"method":"GET","path":"/api/v4/users/3/status","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:user_id/status","user_id":1,"username":"root","queue_duration_s":0.035019,"redis_calls":1,"redis_duration_s":0.000654,"redis_read_bytes":181,"redis_write_bytes":746,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":746,"db_count":4,"db_write_count":0,"db_cached_count":0,"cpu_s":0.045097,"mem_objects":11415,"mem_bytes":3337744,"mem_mallocs":5476,"mem_total_bytes":3794344,"pid":108812,"correlation_id":"01FHXQ1FEJ77PH300X8CNN5JJA","meta.user":"root","meta.caller_id":"GET /api/:version/users/:user_id/status","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T20:55:58.754Z","severity":"INFO","duration_s":0.07145,"db_duration_s":0.04214,"view_duration_s":0.02931,"status":200,"method":"GET","path":"/api/v4/users/3","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:id","user_id":1,"username":"root","queue_duration_s":0.009367,"redis_calls":2,"redis_duration_s":0.000811,"redis_read_bytes":182,"redis_write_bytes":795,"redis_cache_calls":1,"redis_cache_duration_s":0.000289,"redis_cache_read_bytes":1,"redis_cache_write_bytes":47,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":748,"db_count":8,"db_write_count":0,"db_cached_count":0,"cpu_s":0.038073,"mem_objects":17062,"mem_bytes":2355192,"mem_mallocs":9102,"mem_total_bytes":3037672,"pid":108814,"correlation_id":"01FHXQ1FEHW8J5N9EC2KYQ30JK","meta.user":"root","meta.caller_id":"GET /api/:version/users/:id","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T20:55:58.778Z","severity":"INFO","duration_s":0.01211,"db_duration_s":0.00192,"view_duration_s":0.01019,"status":200,"method":"GET","path":"/api/v4/users/2/status","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:user_id/status","user_id":1,"username":"root","queue_duration_s":0.009125,"redis_calls":1,"redis_duration_s":0.000299,"redis_read_bytes":181,"redis_write_bytes":748,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":748,"db_count":4,"db_write_count":0,"db_cached_count":0,"cpu_s":0.018426,"mem_objects":9542,"mem_bytes":584744,"mem_mallocs":2084,"mem_total_bytes":966424,"pid":108814,"correlation_id":"01FHXQ1FH43685T298DYEYBB21","meta.user":"root","meta.caller_id":"GET /api/:version/users/:user_id/status","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T20:55:58.843Z","severity":"INFO","duration_s":0.06305,"db_duration_s":0.03475,"view_duration_s":0.0283,"status":200,"method":"GET","path":"/api/v4/users/2","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:id","user_id":1,"username":"root","queue_duration_s":0.024388,"redis_calls":2,"redis_duration_s":0.000499,"redis_read_bytes":182,"redis_write_bytes":795,"redis_cache_calls":1,"redis_cache_duration_s":0.000306,"redis_cache_read_bytes":1,"redis_cache_write_bytes":47,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000193,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":748,"db_count":8,"db_write_count":0,"db_cached_count":0,"cpu_s":0.0374,"mem_objects":17038,"mem_bytes":2348632,"mem_mallocs":9075,"mem_total_bytes":3030152,"pid":108812,"correlation_id":"01FHXQ1FH3RG09RV0BF2C66AWB","meta.user":"root","meta.caller_id":"GET /api/:version/users/:id","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T20:55:59.575Z","severity":"INFO","duration_s":0.01145,"db_duration_s":0.00186,"view_duration_s":0.00959,"status":200,"method":"GET","path":"/api/v4/users/2/status","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:user_id/status","user_id":1,"username":"root","queue_duration_s":0.006628,"redis_calls":1,"redis_duration_s":0.000236,"redis_read_bytes":181,"redis_write_bytes":744,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":181,"redis_shared_state_write_bytes":744,"db_count":4,"db_write_count":0,"db_cached_count":0,"cpu_s":0.015542,"mem_objects":6308,"mem_bytes":457432,"mem_mallocs":1377,"mem_total_bytes":709752,"pid":108814,"correlation_id":"01FHXQ1GA4FS2MKBBE3CR4NFM6","meta.user":"root","meta.caller_id":"GET /api/:version/users/:user_id/status","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T21:02:14.483Z","severity":"INFO","duration_s":0.04289,"db_duration_s":0.02422,"view_duration_s":0.01867,"status":200,"method":"GET","path":"/api/v4/application/statistics","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/application/statistics","user_id":1,"username":"root","queue_duration_s":0.006869,"redis_calls":1,"redis_duration_s":0.000359,"redis_read_bytes":211,"redis_write_bytes":748,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":211,"redis_shared_state_write_bytes":748,"db_count":20,"db_write_count":4,"db_cached_count":0,"cpu_s":0.026699,"mem_objects":9290,"mem_bytes":518872,"mem_mallocs":1856,"mem_total_bytes":890472,"pid":108814,"correlation_id":"01FHXQCYD1MEBNSSGJNT25AZCD","meta.user":"root","meta.caller_id":"GET /api/:version/application/statistics","meta.remote_ip":"10.228.66.12","meta.feature_category":"devops_reports","meta.client_id":"user/1"}
{"time":"2021-10-13T21:02:17.499Z","severity":"INFO","duration_s":0.01143,"db_duration_s":0.00189,"view_duration_s":0.00954,"status":200,"method":"GET","path":"/api/v4/users/3/status","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:user_id/status","user_id":1,"username":"root","queue_duration_s":0.006809,"redis_calls":1,"redis_duration_s":0.000232,"redis_read_bytes":211,"redis_write_bytes":748,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":211,"redis_shared_state_write_bytes":748,"db_count":4,"db_write_count":0,"db_cached_count":0,"cpu_s":0.015208,"mem_objects":6318,"mem_bytes":458296,"mem_mallocs":1381,"mem_total_bytes":711016,"pid":108812,"correlation_id":"01FHXQD1C8HGHFT86M2RK9WGT2","meta.user":"root","meta.caller_id":"GET /api/:version/users/:user_id/status","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
{"time":"2021-10-13T21:02:17.507Z","severity":"INFO","duration_s":0.02928,"db_duration_s":0.00471,"view_duration_s":0.02457,"status":200,"method":"GET","path":"/api/v4/users/3","params":[],"host":"gitlabqa.it.ucf.edu","remote_ip":"10.228.66.12, 127.0.0.1","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","route":"/api/:version/users/:id","user_id":1,"username":"root","queue_duration_s":0.006567,"redis_calls":2,"redis_duration_s":0.000462,"redis_read_bytes":212,"redis_write_bytes":795,"redis_cache_calls":1,"redis_cache_duration_s":0.000192,"redis_cache_read_bytes":1,"redis_cache_write_bytes":47,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":211,"redis_shared_state_write_bytes":748,"db_count":8,"db_write_count":0,"db_cached_count":0,"cpu_s":0.023542,"mem_objects":8780,"mem_bytes":584184,"mem_mallocs":1785,"mem_total_bytes":935384,"pid":108814,"correlation_id":"01FHXQD1BYGMFZ84GXR9XYQ5ZX","meta.user":"root","meta.caller_id":"GET /api/:version/users/:id","meta.remote_ip":"10.228.66.12","meta.feature_category":"users","meta.client_id":"user/1"}
==> /var/log/gitlab/gitlab-rails/sidekiq_client.log <==
# Logfile created on 2021-10-13 16:01:31 -0400 by logger.rb/v1.4.2
==> /var/log/gitlab/redis-exporter/current <==
2021-10-13_20:03:21.87998 time="2021-10-13T16:03:21-04:00" level=info msg="Redis Metrics Exporter 1.17.0 build date: sha1: Go: go1.16.4 GOOS: linux GOARCH: amd64"
2021-10-13_20:03:21.88210 time="2021-10-13T16:03:21-04:00" level=info msg="Providing metrics at localhost:9121/metrics"
2021-10-13_20:48:48.03501 time="2021-10-13T16:48:48-04:00" level=info msg="Redis Metrics Exporter 1.17.0 build date: sha1: Go: go1.16.4 GOOS: linux GOARCH: amd64"
2021-10-13_20:48:48.04200 time="2021-10-13T16:48:48-04:00" level=info msg="Providing metrics at localhost:9121/metrics"
==> /var/log/gitlab/alertmanager/current <==
2021-10-13_20:48:43.88393 level=info ts=2021-10-13T20:48:43.881Z caller=main.go:221 msg="Starting Alertmanager" version="(version=0.22.2, branch=master, revision=)"
2021-10-13_20:48:43.88405 level=info ts=2021-10-13T20:48:43.881Z caller=main.go:222 build_context="(go=go1.16.4, user=GitLab-Omnibus, date=)"
2021-10-13_20:48:43.88406 level=info ts=2021-10-13T20:48:43.882Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=10.228.66.16 port=9094
2021-10-13_20:48:43.91030 level=info ts=2021-10-13T20:48:43.910Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s
2021-10-13_20:48:43.97498 level=info ts=2021-10-13T20:48:43.973Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2021-10-13_20:48:43.97576 level=info ts=2021-10-13T20:48:43.975Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2021-10-13_20:48:43.98353 level=info ts=2021-10-13T20:48:43.979Z caller=main.go:514 msg=Listening address=localhost:9093
2021-10-13_20:48:43.98354 level=info ts=2021-10-13T20:48:43.980Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
2021-10-13_20:48:45.91404 level=info ts=2021-10-13T20:48:45.912Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.002001068s
2021-10-13_20:48:53.91849 level=info ts=2021-10-13T20:48:53.918Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.008080969s
==> /var/log/gitlab/gitlab-exporter/current <==
2021-10-13_21:16:01.54476 ::1 - - [13/Oct/2021:17:16:01 EDT] "GET /ruby HTTP/1.1" 200 998
2021-10-13_21:16:01.54478 - -> /ruby
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/-/profile/two_factor_auth" for 10.228.66.12 at 2021-10-13 17:16:02 -0400
Processing by Profiles::TwoFactorAuthsController#create as HTML
Parameters: {"authenticity_token"=>"[FILTERED]", "pin_code"=>"626335", "current_password"=>"[FILTERED]"}
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.228.66.10 - - [13/Oct/2021:17:16:02 -0400] "GET / -" 301 162 "" "-" -
==> /var/log/gitlab/gitlab-rails/production.log <==
Redirected to https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth
Filter chain halted as :validate_current_password rendered or redirected
Completed 302 Found in 99ms (ActiveRecord: 6.6ms | Elasticsearch: 0.0ms | Allocations: 9246)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"POST","path":"/-/profile/two_factor_auth","format":"html","controller":"Profiles::TwoFactorAuthsController","action":"create","status":302,"location":"https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth","time":"2021-10-13T21:16:02.187Z","params":[{"key":"authenticity_token","value":"[FILTERED]"},{"key":"pin_code","value":"626335"},{"key":"current_password","value":"[FILTERED]"}],"remote_ip":"10.228.66.12","user_id":2,"username":"jgilfoil","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","correlation_id":"01FHXR66MQ3MEP9KT1BZ05DYCF","meta.user":"jgilfoil","meta.caller_id":"Profiles::TwoFactorAuthsController#create","meta.remote_ip":"10.228.66.12","meta.feature_category":"authentication_and_authorization","meta.client_id":"user/2","redis_calls":2,"redis_duration_s":0.001021,"redis_read_bytes":453,"redis_write_bytes":1436,"redis_cache_calls":1,"redis_cache_duration_s":0.000202,"redis_cache_read_bytes":203,"redis_cache_write_bytes":43,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000819,"redis_shared_state_read_bytes":250,"redis_shared_state_write_bytes":1393,"db_count":14,"db_write_count":3,"db_cached_count":2,"cpu_s":0.099469,"mem_objects":11692,"mem_bytes":1701296,"mem_mallocs":3062,"mem_total_bytes":2168976,"pid":108812,"queue_duration_s":0.009393,"db_duration_s":0.00661,"view_duration_s":0.0,"duration_s":0.09947}
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR66MQ3MEP9KT1BZ05DYCF","duration_ms":117,"host":"gitlabqa.it.ucf.edu","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/-/","status":302,"system":"http","time":"2021-10-13T17:16:02-04:00","ttfb_ms":117,"uri":"/-/profile/two_factor_auth","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","written_bytes":119}
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.228.66.12 - - [13/Oct/2021:17:16:02 -0400] "POST /-/profile/two_factor_auth HTTP/2.0" 302 119 "https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0" -
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/-/profile/two_factor_auth" for 10.228.66.12 at 2021-10-13 17:16:02 -0400
Processing by Profiles::TwoFactorAuthsController#show as HTML
Rendered layout layouts/profile.html.haml (Duration: 39.4ms | Allocations: 22938)
Completed 200 OK in 234ms (Views: 38.6ms | ActiveRecord: 8.4ms | Elasticsearch: 0.0ms | Allocations: 123179)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/profile/two_factor_auth","format":"html","controller":"Profiles::TwoFactorAuthsController","action":"show","status":200,"time":"2021-10-13T21:16:02.443Z","params":[],"remote_ip":"10.228.66.12","user_id":2,"username":"jgilfoil","ua":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","correlation_id":"01FHXR66RSYT8SQRSM2VVHWRT1","meta.user":"jgilfoil","meta.caller_id":"Profiles::TwoFactorAuthsController#show","meta.remote_ip":"10.228.66.12","meta.feature_category":"authentication_and_authorization","meta.client_id":"user/2","redis_calls":12,"redis_duration_s":0.003263,"redis_read_bytes":1564,"redis_write_bytes":1947,"redis_cache_calls":11,"redis_cache_duration_s":0.002463,"redis_cache_read_bytes":1208,"redis_cache_write_bytes":550,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":356,"redis_shared_state_write_bytes":1397,"db_count":21,"db_write_count":3,"db_cached_count":6,"cpu_s":0.230863,"mem_objects":125289,"mem_bytes":5931432,"mem_mallocs":12203,"mem_total_bytes":10942992,"pid":108814,"queue_duration_s":0.006009,"db_duration_s":0.00838,"view_duration_s":0.03856,"duration_s":0.23439}
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR66RSYT8SQRSM2VVHWRT1","duration_ms":245,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/-/","status":200,"system":"http","time":"2021-10-13T17:16:02-04:00","ttfb_ms":244,"uri":"/-/profile/two_factor_auth","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0","written_bytes":219013}
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.228.66.12 - - [13/Oct/2021:17:16:02 -0400] "GET /-/profile/two_factor_auth HTTP/2.0" 200 30560 "https://gitlabqa.it.ucf.edu/-/profile/two_factor_auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0" 7.17
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/" for 10.228.66.10 at 2021-10-13 17:16:03 -0400
Processing by RootController#index as HTML
Redirected to https://gitlabqa.it.ucf.edu/users/sign_in
Filter chain halted as :redirect_unlogged_user rendered or redirected
Completed 302 Found in 7ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 3149)
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"location":"https://gitlabqa.it.ucf.edu/users/sign_in","time":"2021-10-13T21:16:03.169Z","params":[],"remote_ip":"10.228.66.10","user_id":null,"username":null,"ua":null,"correlation_id":"01FHXR67PHFZSHHS39YG3CKKM1","meta.caller_id":"RootController#index","meta.remote_ip":"10.228.66.10","meta.feature_category":"projects","meta.client_id":"ip/10.228.66.10","db_count":0,"db_write_count":0,"db_cached_count":0,"cpu_s":0.01447,"mem_objects":8360,"mem_bytes":1120488,"mem_mallocs":1619,"mem_total_bytes":1454888,"pid":108812,"queue_duration_s":0.003968,"db_duration_s":0.0,"view_duration_s":0.0,"duration_s":0.00719}
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"text/html; charset=utf-8","correlation_id":"01FHXR67PHFZSHHS39YG3CKKM1","duration_ms":16,"host":"gitlabqa.it.ucf.edu","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":302,"system":"http","time":"2021-10-13T17:16:03-04:00","ttfb_ms":16,"uri":"/","user_agent":"","written_bytes":107}
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.228.66.10 - - [13/Oct/2021:17:16:03 -0400] "GET / -" 302 107 "" "-" -
==> /var/log/gitlab/gitaly/gitaly_ruby_json.log <==
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:16:04Z","grpc.time_ms":0.153,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108522,"correlation_id":"45f1f144c89aa34235e5201cbecb76e7","time":"2021-10-13T21:16:04.066Z"}
{"type":"gitaly-ruby","grpc.start_time":"2021-10-13T21:16:04Z","grpc.time_ms":0.167,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":108524,"correlation_id":"5f9ddfab1a994d47f8b3867976a6e793","time":"2021-10-13T21:16:04.070Z"}
Output of checks
Expand for output related to GitLab status
[root@xxxx /]# gitlab-ctl status
run: alertmanager: (pid 108441) 2141s; run: log: (pid 96706) 4851s
run: gitaly: (pid 108533) 2135s; run: log: (pid 95981) 5028s
run: gitlab-exporter: (pid 108451) 2140s; run: log: (pid 96636) 4865s
run: gitlab-workhorse: (pid 108431) 2141s; run: log: (pid 96553) 4878s
run: grafana: (pid 108454) 2140s; run: log: (pid 97436) 4723s
run: logrotate: (pid 108461) 2139s; run: log: (pid 95931) 5032s
run: nginx: (pid 108468) 2139s; run: log: (pid 96566) 4877s
run: node-exporter: (pid 108474) 2138s; run: log: (pid 96605) 4871s
run: postgres-exporter: (pid 108485) 2138s; run: log: (pid 96745) 4840s
run: postgresql: (pid 96079) 5012s; run: log: (pid 96078) 5012s
run: prometheus: (pid 108492) 2138s; run: log: (pid 96674) 4857s
run: puma: (pid 108501) 2137s; run: log: (pid 96493) 4890s
run: redis: (pid 95962) 5030s; run: log: (pid 95961) 5030s
run: redis-exporter: (pid 108511) 2137s; run: log: (pid 96647) 4863s
run: sidekiq: (pid 108517) 2136s; run: log: (pid 96513) 4884s
Results of GitLab environment info
Expand for output related to GitLab environment info
[root@xxxxx /]# gitlab-rake gitlab:env:info System information System: RedHatEnterpriseServer 7.9 Current User: git Using RVM: no Ruby Version: 2.7.4p191 Gem Version: 3.1.4 Bundler Version:2.1.4 Rake Version: 13.0.6 Redis Version: 6.0.14 Git Version: 2.33.0. Sidekiq Version:5.2.9 Go Version: unknown GitLab information Version: 14.3.1 Revision: c362490dfe2 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 12.7 URL: https://gitlabqa.it.ucf.edu HTTP Clone URL: https://gitlabqa.it.ucf.edu/some-group/some-project.git SSH Clone URL: git@gitlabqa.it.ucf.edu:some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 13.21.0 Repository storage paths: - default: /data/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
[root@xxxxx /]# gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.21.0 ? ... OK (13.21.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet) Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes Redis version >= 5.0.0? ... yes Ruby version >= 2.7.2 ? ... yes (2.7.4) Git version >= 2.31.0 ? ... yes (2.33.0) Git user has default SSH configuration? ... yes Active users: ... 3 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
I can't point to a line of code, but I was able to narrow this bug down to update 14.3.1. I tested this on 14.3.2 and 14.3.0. The current password
box did not appear in version 14.3.0 and it does appear in versions 14.3.1 and 14.3.2. It seems like it would be tied to this change from the release notes