my node is running smoothly since May '21. Today I’ve experienced the first time an extreme high load of the CPU usage, resulting in a node dashboard, not accessible / showing correct data (zero values). Actually, it is up and running (“online”) again with v1.37.1 and bandwidth is used, too.
I’ve used storjWidget for a while as well, but since the outages / CPU load issues today, it shows a wrong value for today’s earnings - which is equal to the month’s earnings, which makes no sense.
Now I am afraid of a potential outage of the node and I am lost, what to check or to fix. What I can share is some screenshots from the load (heavy + normal, which is still higher than the days and months before) - and I would be very happy to get some troubleshooting support.
Hard to say what your seeing but from my experience this isnt cpu this is IO usage and normally it happens when you restart your node update your node. Basically filewalker likes to destroy drives on start up.
Espically the more data you have the worse it gets, and if your running a SMR drive forget about it your pretty much stuck with it less you move it to a CMR drive.
I happen to have a CMR disk free, but for that I need to move the entire node to another machine (M1 macOS) - including the data on the current HDD. Is that possible?
You have up to 288hours of downtime depending on the size of the node migrating shouldnt be an issue.
Also yes that is the correct way to migrate a node I tested a few nodes myself only failed once cause I forgot to run the delete tag for the last time. So dont do that.
Thank you! I’ll give it a try these days and provide the result (or ask for help).
Just a quick first question: migrating to a new machine means, identity and docker need to be installed first. So for the identity I assume, downloading the binary, copy the node identity from old to new machine, authorise and confirm the identity. Not to forget the port forwarding(s) to the new machine… For docker, simply install docker, then download the storage node docker container and RUN (not setup) the copied node from the old machine. So far so good? (I miss these details in the migration guide.)
For migrating everything has to be intact meaning all the original data that includes the identity, Which I believe its part of the command to transfer as well. You cant get a new identity it has to be the original one that you started with. So you are just copying the data from one hard drive to another you dont need to do anything else.
Well it depends do you have the OS on the drive the storagenode is running on? Im betting not cause you probably run off a SDcard. If not its litterally the same command just different drive.
The Debian OS is running on the SDcard, the storagenode on an USB3 connected HDD. My plan is to move the storagenode to a new (and larger), mounted HDD (CMR) and also from Debian to macOS.
Not needed. This binary used once - when you generated identity and authorized it.
It could be needed only if you decide to start from scratch (and lose held amount) or if you want to setup a new node.
If the last - you need to generate a new identity and sign it with a new authorization token and start with clean storage.
In case of move - you should not generate an identity (otherwise this will destroy your node) and identity binary is not needed too.
the Storj folder on the current hdd is using root, but not pi as a user. will it harm the node, when I change owner/group to pi? pi is a root user, that’s why I am not sure here - do not want to destroy the current storage.
pi@raspberrypi:/media/pi/my5tb $ cd storj/
pi@raspberrypi:/media/pi/my5tb/storj $ ls -al
total 56
drwxr-xr-x 4 pi pi 4096 May 11 21:48 .
drwxr-xr-x 3 root root 4096 May 11 21:43 ..
-rw------- 1 root root 8624 May 11 21:44 config.yaml
drwx------ 4 root root 4096 May 11 21:44 orders
-rw------- 1 root root 32768 May 11 21:48 revocations.db
drwx------ 6 root root 4096 May 11 21:48 storage
-rw------- 1 root root 1377 May 11 21:48 trust-cache.json
pi@raspberrypi:/media/pi/my5tb/storj $ cd ..
pi@raspberrypi:/media/pi/my5tb $ ls -al
total 12
drwxr-xr-x 3 root root 4096 May 11 21:43 .
drwxr-x---+ 3 root root 4096 May 7 23:17 ..
drwxr-xr-x 4 pi pi 4096 May 11 21:48 storj
pi@raspberrypi:/media/pi/my5tb $
The second rsync command of the migration howto copies only the storage folder, but not the other files and folder around it (e.g. orders, config.yaml etc.). Is that correct or should I rsync the parent folder instead?
Why would you not just keep using the rpi4 with the new drive?
doing that now. this questions remains valid:
The second rsync command of the migration howto copies only the storage folder, but not the other files and folder around it (e.g. orders, config.yaml etc.). Is that correct or should I rsync the parent folder instead?
This is depends on how you run the docker - with sudo or without.
If with sudo it doesn’t matter, if without - it makes difference: if folders and files owned by root, the pi will not have an access and node will be disqualified because it cannot provide any piece for audit.
yes, understood. but as docker seems to run with sudo, the files are stored as root and are not accessible with rsync, running without sudo. that’s why I’ve underlined that.
can you confirm that, @Alexey? from what I can see, while rsync is running on the rpi4, the node dashboard is not working (all values on zero, but still online) and I fear, that audits might fail, due to performance issues of the SMR HDD + rsync in progress in parallel. if so, it might be wise to stop the node, do the full rsync and then start the node again. what do you think? the node is filled with 583 GB of data.
The values must not be zero. Please, check are you moving data or copying?
More like age. But it’s tied together
From technical point of view - your node should have more than a month of audits to be eligible for 288 hours. If it’s younger - the online score could fall to 60% much faster.
