I had the same problem with high RAM setups (3GB/TB), so I don’t think RAM is the problem here. I think the metadata operations are just too slow. Something that probably is true for almost every filesystem now: performance - Why the amount of files affects much deleting speed? - Stack Overflow
We could list up a lot of topics about the same subject here:
I’m hoping for the file-stat-cache (storagenode/blobstore: blobstore with caching file stat information (… · storj/storj@2fceb6c · GitHub). Although, the filesystem should be handling all meta data in a smooth way, reading and changing it cost a lot of random IO which is usually quite show. So, I’m hoping it will reduce the amount of random IO and make interviewde file operations considerably faster. Therefore I also hope it will be made compatible with the lazy filewalker.
Essentially I think we’re looking at some growing pains of STORJ. For the time being I reverted to ZFS with special VDEVS for meta information on SSD.