Skip to content

Force ASCII-8BIT encodings in CI job trace

Stan Hu requested to merge sh-force-encoding-job-logs into master

This commit adds a feature flag, ci_job_trace_force_encode, that will ensure that CI job traces that contain UTF-8 can be appended with ASCII-8BIT data.

Previously we saw encoding errors when persisted job logs that contained UTF-8 characters were appended to ASCII-8BIT data from the HTTP request. This would cause a 500 error, but the runner would retry indefinitely with identical PATCH requests. This commit should make the append work and allow the job to finish.

This is the sequence of events that causes the append to be called:

  1. Runner sends a PUT /api/v4/jobs/:id to indicate the job is canceled or finished.

  2. UpdateBuildStateService#accept_build_state! persists all live job logs to object storage.

  3. UpdateBuildStateService#accept_build_state! returns a 202 to the runner.

  4. The runner will continue to send PATCH requests with job logs until all logs have been sent and received.

  5. If the last PATCH request arrives after the job log has been persisted, we retrieve the data from object storage to append the remaining lines.

Relates to #333424 (closed)

Edited by Stan Hu

Merge request reports