Disk fragmentation is inevitable... Do we need to prepare?

Instead of defragmenting the metadata, it’s much more effective to relocate it to the SSD using some sort of tiered storage or caching solution. It’s counterproductive to keep defragmenting the disk — it’s one of those cases when the cure is worse than the disease.

For that you need to have an SSD, but sure, it’s possible to do under Windows too, but not via GUI, only with PowerShell.
However, if you do not have a spare SSD and/or do not want to use a terminal to setup a tiered Storage, well then only the defragmentation, since it’s NTFS.

Right, but SSD is $10 on eBay. Why waste weeks defragmenting when throwing $10 at the problem solves the root cause?

Because you know, use what you have now… But the solution should work of course.

Right. But “use what you have” does not mean “it will work on anything you have”. It means “don’t build a server for storj”. It’s not a restriction to spend a nominal amount of money to improve one’s life.

So the real choice here is:

  • don’t run the node, don’t get payouts.
  • run the node and [periodically, forever] spend waste extra time babysitting the fragmentation (including stopping ingress — because it’s already at the IOps limit — to defragment). Not to mention wasting extra energy: Disk at full IO load consumes almost double power compared to idle or low activity
  • pay $10 to some fella on eBay, and solve the problem once and for all by addressing the root cause instead of treating symptoms: no time wasted defragmenting, no power waste, elevated IOPS ceiling, so node can continue working and serving customers with the increased load.

20TB = 140 million files. 30 days TTL = 4.6 million deletes and 4.6 million new files every day.

Forget about defrag.

Defrag just took it’s things and left the building. :cold_sweat:

Can’t remember my last defrag, must have been in the days of Windows 3.1. :thinking:

But it remembers you… :slightly_smiling_face: ahh, the good old MS DOS days…

Ok, @pangolin , @snorkel , @arrogantrabbit . But what are you would suggest to Windows users? Migrate to Linux? To the scary terminal?

They almost have no choice, unfortunately. So, please, understand and do not force them to migrate. Everyone should consider for themselves in my opinion.

I’m personally forced to use Windows, because Linux cannot work on my HW because of several hardware bugs (CPU and the motherboard), it works stable only under Windows. Even if I use Linux everywhere else… So, sometimes you have no choice actually (except buying a new HW… which is not recommended… you know…).

Use the toolbox to run multiple nodes not docker, move databases to SSD, avoid USB if possible, maximize RAM.

Never said that. In other thread I advised the same as you; use the os that you already have, don’t overcomplicate things by running virtual machines just to use linux, IF… you want to run nodes on your PC used for your daily things.
BUT if you want to expand and build new systems just for Storj, it’s better to go with Linux.
If you already have a windows machine, you work on it, play games, etc and want to make some bucks by running nodes, of course the best way is to use what you have and don’t build anything just for Storj. But some members here sounded to me pretty enthusiastic and eager to build mini Storj farms. Better to start on the right track.
Let’s summarise it like this: bigger nodes run better on Linux. And now, since starting a node on the biggest drive makes the most sense, Linux is the better choise for new nodes.