Currently it is being used 100% allmost all the time (95-100% ish - mostly 100%).
Now last night my server kinda was frosen and the whole docker instance was not working. That meant my two nodes where getting downtime… I suspect that the 12TB drive is causing a high CPU usage and thus a chance of a crash/freeze.
Is there anything i can do to make sure the 12TB (internal BTW) drive can keep up? Maybe reducing the usage or adding a cache SSD? something?
I know i can add more nodes to the server - but i allready have 3 so i dont think thats the solution.
If you have an SSD available, try moving the databases to the SSD first. Caching isn’t going to help a lot for the pieces themselves, but moving the db’s eliminates a lot of small write operations. How full is the HDD at this point? You probably don’t want to fill up an SMR HDD too much as it will just get worse and worse in performance.
Other things you can do is disable the filewalker from running at restart of the node. This would also make the next suggestion easier, which is to lower the assigned space and add small amounts at a time. If the node is full, writes will significantly drop and this gives the HDD time to write CMR cache to SMR regions. You might want to drop the assigned space to below what is already stored to let the HDD settle and wait until it drops from 100% usage to something more reasonable before raising it again.
Ultimately, there is no sure fire way to make SMR work well. It is not well suited for Storj. But please try these suggestions and keep this topic in mind if you’re considering spending money: SNO's please don't buy anything
Good news is that’s a CMR drive. Bad news is it’s only 5400 RPM. And since you mentioned the server froze, if you just restarted it recently the filewalker process is probably still running after the reboot. Some people say this can take a day or more in some cases. Plus, running multiple nodes on the same drive will slow things down as well. On top of that, early on in the month seems to be when a lot of data gets deleted and that of course will also increases disk IO.
Just copy the database files to where you want. If on linux I would use rsync -a in order to keep permissions and everything the same. I don’t think it matters with Windows.
I believe this is the correct config entry but wouldn’t hurt to double check:
storage2.database-dir: “”
Please be careful providing partial information only on this process. Some SNOs have really messed up their nodes by making mistakes during this process. It may be obvious to you how it should be done, but it isn’t to everyone. The link I posted has detailed steps to follow to prevent issues.
Does this load immediately stop after you stop the node? I’m guessing this is an HDD from an external USB drive? Those aren’t usually optimized for performance, but this is still quite strange behavior from any CMR HDD. I’m running super old ones that don’t behave like this. If the load on the HDD immediately stops (listen for activity as well as the OS will likely not show activity for internal maintenance operations) that’s a good confirmation that it’s indeed CMR. If it keeps rattling on… I’m starting to have my doubts about WD’s response on that support ticket.
Fair enough. But my feeling on the matter is if you really don’t understand what your doing it’s your own responsibility to ask for further clarification. If you don’t and you dig yourself a hole, this is what we call a learning experience. I have no problem holding someones hand through something or pointing them to someone who can, but in a world full of “NPC’s” I really prefer to give people the benefit of the doubt since most people here are still smarter than your average Joe.
Aside from all that though, it’s my understanding that the database files only have to do with tracking node history anyway, and if they’re deleted then they will be recreated by the node and all you actually loose is your dashboard history.
If i stop the node in docker. the activity goes to 0% within 1-2 seconds - and the drive goes from loud to very quiet.
When starting the node the drives goes from 0&-100% in 4 seconds ish…
