Skip to content

Rails: curl_retry_netfail, allow retry from network failures

Jason Plum requested to merge rails-nodejs-fetch into master

What does this MR do?

Rails: curl_retry_netfail, allow retry from network failures

Add curl_retry_netfail script to shared/build-scripts used by rails build containers.

This script will retry on error codes 18 and 92, which is what we are currently seeing.

Related to #733

Example of this working

https://gitlab.com/gitlab-org/build/CNG/-/jobs/5363767823#L1141 NodeJS had a release, which "broke" the caching layer, and this eventually retried, and succeeded.

#41 [builder  8/20] RUN NODE_TARBALL="node-v18.17.1-linux-x64.tar.gz" ;     /build-scripts/curl_retry_netfail "https://nodejs.org/download/release/v18.17.1/${NODE_TARBALL}" ${NODE_TARBALL}     && tar --strip-components 1 -xzC /usr/local/ -f ${NODE_TARBALL}
#41 0.317 /build-scripts/curl_retry_netfail: fetching 'node-v18.17.1-linux-x64.tar.gz' from 'https://nodejs.org/download/release/v18.17.1/node-v18.17.1-linux-x64.tar.gz'
#41 0.318 /build-scripts/curl_retry_netfail: attempt # 1
#41 870.1 curl: (18) transfer closed with 24854902 bytes remaining to read
#41 870.1 /build-scripts/curl_retry_netfail: attempt # 1: 18
#41 870.1 /build-scripts/curl_retry_netfail: attempt # 2
#41 870.8 /build-scripts/curl_retry_netfail: attempt # 2: 0
#41 DONE 873.4s
#42 [builder  9/20] RUN mkdir /usr/local/yarn ; YARN_TARBALL="yarn-v1.22.18.tar.gz" ;     /build-scripts/curl_retry_netfail "https://yarnpkg.com/downloads/1.22.18/${YARN_TARBALL}" ${YARN_TARBALL}     && tar --strip-components 1 -xzC /usr/local/yarn -f ${YARN_TARBALL}     && ln -sf /usr/local/yarn/bin/yarn /usr/local/bin/     && ln -sf /usr/local/yarn/bin/yarnpkg /usr/local/bin/
#42 0.334 /build-scripts/curl_retry_netfail: fetching 'yarn-v1.22.18.tar.gz' from 'https://yarnpkg.com/downloads/1.22.18/yarn-v1.22.18.tar.gz'
#42 0.334 /build-scripts/curl_retry_netfail: attempt # 1
#42 0.839 /build-scripts/curl_retry_netfail: attempt # 1: 0
#42 DONE 0.9s

Related issues

Related to #733

Related to gitlab-com/gl-infra/production#17005 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Integration tests added to GitLab QA
  • The impact any change in container size has should be evaluated
  • New dependencies are managed with dependencies.io
Edited by Jason Plum

Merge request reports