Node suspended after migration from Synology to Windows

[apitsos]

Hi there,

One of my nodes was on a Synology NAS (docker), where I had several crashed drives. I decided to move it to a Windows Server and this migration took about 4 days. I did it running the following command…

robocopy S:\data\storage E:\STORJ /MIR /ZB /R:3 /W:1

Afterwards I also moved the orders using the following command:

robocopy /MIR /MOVE S:\data\orders "$env:ProgramFiles\Storj\Storage Node\orders"

In general I tried to follow instructions given here: Migrating from Docker CLI to a GUI Install on Windows - Storj Docs

After the data migration, I realized that I couldn’t install the Windows GUI STORJ application and I was getting an error. Looking in the log file, I realized that the installation couldn’t be completed because the path I was providing was already existing.

In order to solve the problem I renamed the folder containing the migrated data and I completed the installation. Then I moved the data from the renamed folder to the correct one and configured a few more things in the “config.yaml” file, like ports, hostname, etc.

The problem occurred after this was that the service didn’t start. I realized that there was a privileges issue. I noticed on the “security” tab of the “blobs” subfolders that the owner was a non-existing user and the rights also were assigned to a non-existing user. So I started changing the ownership to “COMPUTER\Administrators” user group and the rights as well. This is started from yesterday afternoon and still running!

During all this process I managed to start the node and the service, because it was already many days down and I don’t want to get disqualified. Nevertheless, I see now the following:

The message…

Your node has been suspended on...

…frightens me a bit. I would like to ask how can I make sure that all the files are having now the correct rights and will be all accesible by the STORJ software and service. In order to change the ownership recursively I ran the command:

takeown /f E:\STORJ  /r /a

And this is still running from yesterday!

And for changing the rights afterwards, I do it on the security tab of each subfolder under “blobs”. And that takes also hours to be completed on each folder.

Am I doing it something wrong? Is there something you could suggest?

And the most important question, already placed above. How can I make sure that this will work properly and all files will be accessible.

And finally, what can I do in order to avoid disqualification?

Thanks a lot in advance for your time and supporting me!

With kind regards,
Angelos Pitsos

[nyancodex]

@apitsos

the “blobs” subfolders that the owner was a non-existing user and the rights also were assigned to a non-existing user. So I started changing the ownership to “COMPUTER\Administrators” user group and the rights as well. This is started from yesterday afternoon and still running!

During all this process I managed to start the node and the service, because it was already many days down and I don’t want to get disqualified. Nevertheless, I see now the following:

Audit score still good, I think the node still recoverable. But you need to post the error logs here so people can help.

Next time use Docker. That will save you a lot of headaches.

[apitsos]

Hi @nyancodex,

Thanks a lot for your reply. Here is part of the logs, with the tag “GET_AUDIT”. Hopefully this is what you are asking for:

2024-09-27T13:00:07+03:00	INFO	piecestore	download started	{"Piece ID": "EJRS653ICEKQ4WSMBGLFXRK3HJ56HIFY3W4XH32BPFCZNVQEHSVQ", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET_AUDIT", "Offset": 2277888, "Size": 256, "Remote Address": "34.102.121.106:40695"}
2024-09-27T13:00:08+03:00	INFO	piecestore	downloaded	{"Piece ID": "EJRS653ICEKQ4WSMBGLFXRK3HJ56HIFY3W4XH32BPFCZNVQEHSVQ", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET_AUDIT", "Offset": 2277888, "Size": 256, "Remote Address": "34.102.121.106:40695"}

If that’s not helpful, please clarify what should I search for in the logs.

With kind regards,
Angelos Pitsos

[snorkel]

Are you sure the nodes are stopped and removed from Syno? Just check the containers in Docker.

[apitsos]

Hi @snorkel,

The container on the Synology is stopped and removed 5 days ago.

[Julio]

@apitsos

Am I doing it something wrong? Is there something you could suggest?

None of that was necessary, the data got there while you were in your user space; therefore its all valid and accessible. Next time, just change security from the top drive letter, and give rights only through the top simple security box. To be sure, just add ‘everyone’ as user, & give full control. It will not need to propagate through the entire disk for days and days, all sub dirs will inherit that security setting instantly (unless there are specific settings recursive to that.) On a server version of windows, you’d want to especially give proper rights, before even transferring anything due to increased security defaults…re: no write/modify to root dir allowed unless you change privliages, etc.
Next time, set security on the drive letter first, add user ‘everyone’ to be safe, and ‘full control’ to all other listed accounts (administrators, authenticated users, system, etc), BEFORE transferring data. No need to take ownership on everything or advanced settings use, every file will inherit these permissions.

‘robocopy /MIR /MOVE’ – Which were you intending? You just needed the /MIR to be safe, move would leave you without a temp backup option until you were successful getting node to run.
/R:3 /W:1 – 3 retrys over 1 second intervals isn’t network friendly. maybe increase that to 30 or 60 seconds next time, so it actually has a chance if there’s service disruption.
If you wanted it to got faster, you might have considered the /MT: [# of threads option], depending on the quality of disk you have and whether you don’t mind a little fragmentation. That along with utilizing the /LOG: option would at lease quadruple your transfer speed.

Last pass use /MIR /MT:32 /LOG:C:\LOGOFMOVE.TXT /R:2 /W:60, if you have 32 threads available, or maybe 16 if you don’t have so many threads, or if system transferring from is lame, or destination disk isn’t all that. Default is 8, windows to windows.

You have arrived… I would not sweat it… your suspension should clear up, back to 100% in a day or two. There were no audit errors, so far - so prolly ohtay!

GL
3 1/2 cents
an extra 1 and 1/2 cent, cuz I’ve never seen anyone mention multi-threading robocopy on these forums.
P.S. If you get too greedy, you will fragment everything to cheet!
Check your drive’s ‘Active time %’/ ‘response time’ / iops, and ensure you are not totally saturating it’s capabilities, if you do: your throughput will drop like on a bell curve, and you may run out of memory having too many file handles open. ie: tweak # threads to your system (source & destination) & hdd’s capability and ops.

extra 1/2 bonus: Layering the use of /MIN: /MAX: size parameters on a storj dataset could be your friends in the quest for optimization. If do so intelligently, you could achieve 20-150 megs/s transfer speed with the right settings/equipment. Finally, check the last lines of the log file when done for comparative stats.

4 cents

[Julio]

People spending days to transfer storj data has been a constant source of amusement for me, since I arrived here. But I’m not a cruel person so those are a couple hints to point people in the right experimental direction.

Julio

[Alexey]

Hello @apitsos,
Welcome back!

You need to change the owner to the SYSTEM, not to your user, because the storagenode service is running under SYSTEM user by default.
You need also to grant at least RW permissions to it.

[apitsos]

Hi @Julio, hi @Alexey!

Thanks a lot both of you for your answers. @Julio, I appreciate much all the info you have provided and I understood them all. Next time I will try out that way of migration. I agree almost with everything, except the part of providing full access to “Everyone”. I am a guy who loves and respects security and that I wouldn’t do it. Actually when people tells me, oh just give rights to the user “Everyone”, I consider that they are not serious with security in Windows. Nevertheless, all the rest part is considerable and I will make some tests prior the next time I will need it, i order to test it out. Once again, thank s a lot.

Dear @Alexey you as always to the point! In one sentence you write it all. I just checked the ownership of the automatic created folder from the installation of the STORJ Windows app (which was kept temporarily and renamed) and noticed exactly what you say. Now I am changing the ownership recursively. And that takes time, since there are thousands of folders and millions of files underneath.

Nevertheless, the situation has improved a lot. And as @Julio said, a day or two later the Suspension negative/red percentage goes away. Here it is how it looks now:

Wishing you all a great weekend! Once again, thanks a lot for the your support and the tips. They are always much appreciated!

With kind regards,
Angelos Pitsos

[Julio]

Excellent, Glad to help. Of course you’re right - security should be a concern, would not take long to cipher 4% of your data. Worse yet, imagine recovering 40 million files from a deletion… eek!

Take care.