Tuning the filewalker

Revisiting this topic, I’ve noticed that you’re giving these numbers. Maybe I forgot about them before, sorry. I do not observe that the file walker takes so much time. Here it was ~12 minutes per terabyte (with cache cleared), which got down to ~8 minutes per terabyte with -I 128, and even less than that without clearing cache. So, more than an order of magnitude less than yours.

Yet Stob’s observation suggests the file walker process also hits >24h times.

I now wonder whether there are any common qualities between your and @Stob’s nodes that make the file walker so long.

It would also definitelly help if we could have more precise measurements of the time consumed by this process, as opposed to guessing it from IO usage graphs.

1 Like