Switch production drivers to standard Walk
Rationale
As we previously discovered with S3 in !58 (merged), WalkParallel can strain resources (in this case API requests) when ran in an unbounded fashion. With this, it seems more appropriate to give special attention to each of these drivers before merging them with truly parallel walks into a release.
Changes
The swift, azure, gcs, and oss drivers call their Walk
method in WalkParallel
, rather than storagedriver.WalkFallbackParallel
. Calling their own Walk
method, rather than storagedriver.WalkFallback
ensures that any gains made to their respective Walk
functions before we've verified their concurrent behavior are passed onto WalkParallel
Caveats
The filesystem and inmemory drivers still use storagedriver.WalkFallbackParallel
in their WalkParallel
. The inmemory is explicitly for testing and filesystem does not depend on networked resources (baring something like nfs) and is less likely to be used in large production environments.