Cache file walker should delete empty directories

Spin off of #2288 (closed)

More info from original incident: gitlab-com/gl-infra/production#1501 (comment 262058610)

The cache file worker currently does not remove empty directories. These empty directories come at a cost when traversing the file system. To reduce this cost, empty directories should be pruned from the filesystem as they become empty or are detected to be empty on subsequent passes.