Skip to content

terraform state init fails

Summary

After configuring terraform to use gitlab as http backend, running terraform init can not copy the local terraform state to the newly created backend.

Steps to reproduce

  1. Create API token for user
  2. configure terraform http backend:
terraform {
  backend "http" {
    address        = "https://[hostname]/api/v4/projects/[id]/terraform/state/[statename]"
    lock_address   = "https://[hostname]/api/v4/projects/[id]/terraform/state/[statename]/lock"
    unlock_address = "https://[hostname]/api/v4/projects/[id]/terraform/state/[statename]/lock"
    lock_method    = "POST"
    unlock_method  = "DELETE"
    username       = "user"
    password       = "tokentoken"
  }
}
  1. run terraform init 
Initializing modules...

Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "http" backend. No existing state was found in the newly
  configured "http" backend. Do you want to copy this state to the new "http"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes


Error: Error copying state from the previous "local" backend to the newly configured
"http" backend:
    Failed to upload state: POST https://[hostname]/api/v4/projects/[id]/terraform/state/[statename] giving up after 3 attempts

The state in the previous backend remains intact and unmodified. Please resolve
the error above and try again.

What is the current bug behavior?

Terraform can not create a new state in the gitlab http backend.

What is the expected correct behavior?

Terraform can create a new state in the gitlab http backend.

Relevant logs and/or screenshots

/var/log/gitlab/gitlab-rails/api_json.log

{"time":"2020-04-22T16:56:36.147Z","severity":"INFO","duration_s":0.08,"db_duration_s":0.01,"view_duration_s":0.07,"status":500,"method":"POST","path":"/api/v4/projects/[id]/terraform/state/[statename]","params":[{"key":"version","value":null},{"key":"terraform_version","value":"0.12.24"},{"key":"serial","value":null},{"key":"lineage","value":"a8ab9bef-dd14-435a-2dc6-e733f97e4e40"},{"key":"outputs","value":{}},{"key":"truncated","value":"..."}],"host":"[hostname]","remote_ip":"[someips]","ua":"Go-http-client/1.1","route":"/api/:version/projects/:id/terraform/state/:name","user_id":3,"username":"[username]","exception.class":"NoMethodError","exception.message":"undefined method `string' for #<Unicorn::TeeInput:0x00007fd573ec6a58>","exception.backtrace":["lib/api/terraform/state.rb:49:in `block (3 levels) in <class:State>'"],"queue_duration_s":0.05,"redis_calls":1,"redis_duration_s":0.0,"correlation_id":"Vg7r6rPz0F8"}

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:         Debian 9.12
Current User:   git
Using RVM:      no
Ruby Version:   2.6.5p114
Gem Version:    2.7.10
Bundler Version:1.17.3
Rake Version:   12.3.3
Redis Version:  5.0.7
Git Version:    2.26.2
Sidekiq Version:5.2.7
Go Version:     unknown

GitLab information
Version:        12.10.0
Revision:       869bc26464b
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     11.7
URL:            https://[hostname]
HTTP Clone URL: https://[hostname]/some-group/some-project.git
SSH Clone URL:  git@[hostname]:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version:        12.2.0
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 12.2.0 ? ... OK (12.2.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
6/1 ... yes
6/2 ... yes
18/3 ... yes
6/4 ... yes
16/6 ... yes
3/7 ... yes
3/9 ... yes
7/10 ... yes
3/12 ... yes
6/18 ... yes
6/21 ... yes
3/22 ... yes
3/23 ... yes
7/24 ... yes
6/27 ... yes
6/28 ... yes
3/29 ... yes
6/30 ... yes
3/31 ... yes
6/32 ... yes
6/33 ... yes
18/35 ... yes
18/38 ... yes
6/39 ... yes
18/40 ... yes
18/41 ... yes
18/42 ... yes
18/43 ... yes
18/45 ... yes
6/47 ... yes
18/49 ... yes
18/52 ... yes
18/58 ... yes
18/59 ... yes
18/60 ... yes
18/61 ... yes
18/62 ... yes
18/63 ... yes
18/65 ... yes
16/66 ... yes
6/67 ... yes
3/68 ... yes
6/71 ... yes
6/72 ... yes
16/73 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.5)
Git version >= 2.22.0 ? ... yes (2.26.2)
Git user has default SSH configuration? ... yes
Active users: ... 10
Is authorized keys file accessible? ... yes


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

https://gitlab.com/gitlab-org/gitlab/-/blob/5b5e19c6a4b5fb0a27c3ecc745f3835cdf252a8b/lib/api/terraform/state.rb#L49 should probably be changed to:

           data = request.body.read