cache: Remove entries that no longer exist in the source
cache: Remove entries that no longer exist in the source

list directory with 25k files

before(1.43.1)
5m24s

after
3m21s

olihey/rclone,olihey/rclone,B4dM4n/rclone,ncw/rclone,ncw/rclone,B4dM4n/rclone,olihey/rclone,ncw/rclone,olihey/rclone,ncw/rclone,B4dM4n/rclone,B4dM4n/rclone