Cache walker should crawl more efficiently and remove all stale artifacts

Relates to gitlab-com/gl-infra/production#1501 (comment 262058610)

The performance problems seen in the related issue might be related to the cache walker keeping large lists of files in memory. We divide the cache entries and cache state files into 256 directory "buckets" (similar to git's object directory). Go's file walker sorts the list of files before walking, so the entire file listing needs to be stat'd and kept in memory for the sort:

The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient.

Source: https://golang.org/pkg/path/filepath/#WalkFunc

A more efficient file walker that reduces the number of opened files and the amount of file info's kept in memory might alleviate this issue.