Return metav1.Status on Kubernetes proxy errors
Closes gitlab-org/gitlab#338305 (closed).
Current behavior:
➜ ~ k get pods
error: You must be logged in to the server (the server has asked for the client to provide credentials)
➜ ~ k -v 9 get pods
I0524 20:40:24.836334 29828 loader.go:372] Config loaded from file: ~/.kube/config
I0524 20:40:24.836798 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.837225 29828 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:8154 succeed
I0524 20:40:24.863423 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 26 milliseconds
I0524 20:40:24.863439 29828 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 25 ms ServerProcessing 0 ms Duration 26 ms
I0524 20:40:24.863445 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.863450 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.863454 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.863459 29828 round_trippers.go:580] X-Request-Id: 01G3TTEM8ZZWQ8EKTWM2CSHPJ3
I0524 20:40:24.863463 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.863467 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.863472 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.883270 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.902200 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.902220 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.902301 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.903091 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.903103 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.903109 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.903115 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.903120 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMA60E4ZXVA2C0QNYE1B
I0524 20:40:24.903125 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.903130 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.903135 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.903140 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.922101 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.941260 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.941274 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.941287 29828 shortcut.go:89] Error loading discovery information: the server has asked for the client to provide credentials
I0524 20:40:24.941366 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.941989 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.941999 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.942008 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.942014 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMBDSC2H86E15NJKMD6C
I0524 20:40:24.942019 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:24.942025 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.942031 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.942036 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.942041 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.960181 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:24.980410 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:24.980436 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:24.980550 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:24.981403 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:40:24.981420 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:40:24.981426 29828 round_trippers.go:577] Response Headers:
I0524 20:40:24.981432 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:24.981438 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:24.981445 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:24.981450 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:24.981456 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMCNTD9H79EFH22RXY63
I0524 20:40:24.981461 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:24 GMT
I0524 20:40:25.001150 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:25.021818 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:25.021857 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:25.022029 29828 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:40:25.023248 29828 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 1 milliseconds
I0524 20:40:25.023262 29828 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 1 ms
I0524 20:40:25.023270 29828 round_trippers.go:577] Response Headers:
I0524 20:40:25.023279 29828 round_trippers.go:580] Date: Tue, 24 May 2022 10:40:25 GMT
I0524 20:40:25.023286 29828 round_trippers.go:580] Content-Length: 36
I0524 20:40:25.023292 29828 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0524 20:40:25.023299 29828 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:40:25.023305 29828 round_trippers.go:580] X-Content-Type-Options: nosniff
I0524 20:40:25.023311 29828 round_trippers.go:580] X-Request-Id: 01G3TTEMDYZBPGX37MD6YNW28Z
I0524 20:40:25.043278 29828 request.go:1073] Response Body: Authorization header: invalid value
I0524 20:40:25.063848 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value
'
I0524 20:40:25.063913 29828 cached_discovery.go:119] skipped caching discovery info due to the server has asked for the client to provide credentials
I0524 20:40:25.064161 29828 helpers.go:222] server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server has asked for the client to provide credentials",
"reason": "Unauthorized",
"details": {
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "Authorization header: invalid value"
}
]
},
"code": 401
}]
error: You must be logged in to the server (the server has asked for the client to provide credentials)
As you can see, kubectl
is trying and failing to parse kas' response - I0524 20:40:24.902200 29828 request.go:1264] body was not decodable (unable to check for Status): Object 'Kind' is missing in 'Authorization header: invalid value'
. User also gets a generic error: You must be logged in to the server (the server has asked for the client to provide credentials)
rather than what kas actually sends in the response body Authorization header: invalid value
.
New behavior:
➜ ~ k get pods
error: You must be logged in to the server (GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMA86JFW3SBS31WAVDHG0)
➜ ~ k -v 9 get pods
I0524 20:43:43.985391 29895 loader.go:372] Config loaded from file: ~/.kube/config
I0524 20:43:43.986389 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" -H "Accept: application/json, */*" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:43.987981 29895 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:8154 succeed
I0524 20:43:44.017253 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 30 milliseconds
I0524 20:43:44.017264 29895 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 28 ms ServerProcessing 0 ms Duration 30 ms
I0524 20:43:44.017271 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.017275 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.017279 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.017282 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ
I0524 20:43:44.017285 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.017288 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.038388 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ","reason":"Unauthorized","code":401}
I0524 20:43:44.056407 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPRGC4Z7GJ33W6ZB5JXZ
I0524 20:43:44.056498 29895 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.059073 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 2 milliseconds
I0524 20:43:44.059084 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 2 ms
I0524 20:43:44.059088 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.059092 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.059097 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.059100 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.059104 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.059109 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.082077 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7","reason":"Unauthorized","code":401}
I0524 20:43:44.101247 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.101500 29895 shortcut.go:89] Error loading discovery information: GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPSRDKN4DKB28MVTMCV7
I0524 20:43:44.101556 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" -H "Accept: application/json, */*" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.102167 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:43:44.102176 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:43:44.102180 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.102185 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.102189 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.102194 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.102198 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPV5C52VB87VPS081815
I0524 20:43:44.102202 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.104077 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPV5C52VB87VPS081815","reason":"Unauthorized","code":401}
I0524 20:43:44.123329 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPV5C52VB87VPS081815
I0524 20:43:44.123443 29895 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.124380 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 0 milliseconds
I0524 20:43:44.124394 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 0 ms
I0524 20:43:44.124399 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.124405 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.124409 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.124414 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.124419 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPVV7627JZMX0D9VQT3N
I0524 20:43:44.124424 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.143208 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPVV7627JZMX0D9VQT3N","reason":"Unauthorized","code":401}
I0524 20:43:44.162461 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPVV7627JZMX0D9VQT3N
I0524 20:43:44.162572 29895 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.0 (darwin/arm64) kubernetes/4ce5a89" -H "Authorization: Bearer <masked>" 'https://127.0.0.1:8154/api?timeout=32s'
I0524 20:43:44.163716 29895 round_trippers.go:553] GET https://127.0.0.1:8154/api?timeout=32s 401 Unauthorized in 1 milliseconds
I0524 20:43:44.163726 29895 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 0 ms Duration 1 ms
I0524 20:43:44.163731 29895 round_trippers.go:577] Response Headers:
I0524 20:43:44.163737 29895 round_trippers.go:580] Content-Length: 231
I0524 20:43:44.163741 29895 round_trippers.go:580] Content-Type: application/json
I0524 20:43:44.163746 29895 round_trippers.go:580] Server: gitlab-kas/v0.0.0/00000000
I0524 20:43:44.163750 29895 round_trippers.go:580] X-Request-Id: 01G3TTMPX3MB77B6FY24SF9V6E
I0524 20:43:44.163755 29895 round_trippers.go:580] Date: Tue, 24 May 2022 10:43:44 GMT
I0524 20:43:44.183170 29895 request.go:1073] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E","reason":"Unauthorized","code":401}
I0524 20:43:44.203349 29895 cached_discovery.go:119] skipped caching discovery info due to GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E
I0524 20:43:44.203556 29895 helpers.go:222] server response object: [{
"metadata": {},
"status": "Failure",
"message": "GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E",
"reason": "Unauthorized",
"code": 401
}]
error: You must be logged in to the server (GitLab Agent Server: Unauthorized: Authorization header: invalid value. Correlation ID: 01G3TTMPX3MB77B6FY24SF9V6E)
Improvements:
-
GitLab Agent Server
in the message allows to distinguish kas-generated errors vs upstream-generated errors. - Actual error message is displayed to the user.
- No parsing errors.
- Correlation ID is displayed.