Hello,
I have launched a graceful exit 10 days ago. And for one satellite (europe-north-1.tardigrade.io:7777) it is stuck at 10% for few days, the others are complete now. It is only 1.12 GB from 2 TB. Why is it so long ?
kind regards.
Hello @pileus-lines,
Welcome to the forum!
It is a test satellite, and perhaps it has different settings. It also possible that SNOs blocking traffic from the customers of this satellite (because it pays less).
You may try to increase parameters of GE:
Hi,
Got same problem here. Other satellites completed fast. But this EU1 just keep stalling.
I’ve been in 58% of 1Tb like a week now without progress.
Now it only keeps saying this:
2023-06-10T12:52:17.337+0300 ERROR gracefulexit:chore worker failed {“Process”: “storagenode”, “error”: “graceful exit processing interrupted (node should reconnect and continue)”, “errorVerbose”: “graceful exit processing interrupted (node should reconnect and continue)\n\tstorj.io/storj/storagenode/gracefulexit.(*Worker).Run:90\n\tstorj.io/storj/s
toragenode/gracefulexit.(*Chore).AddMissing.func1:82\n\tstorj.io/common/sync2.(*Limiter).Go.func1:49”}
I’ve shut down and started process again without any success.
And few hours after writing that last post it suddenly started to transfer fast. Did they do something to the satellite?
Soon exit is complete.
It was fun experience, but energy prices here are way too high to keep anything like this anymore 
Bye
Hello.
In my case, graceful exit may be lasting hundreds of days, as shown in the following spreadsheet.
The second day, i had my fiber optic disconnected in the street wiring cabinet, and has been reconnected three days later by my ISP.
Is this the reason of the slow speed ?
I cannot wait as much to free some space in my server.
Do you suggest to change settings ?
Thank you.
The current situation. Online score due to three days offline.
GE will pick up speed faster and faster. Just keep the machine online. On one of my nodes, after 24h, I had like 24% completion, after 48h, like 60%, and in less than 3 days was done. I did only 2 sats, though.
Thank you for your hopeful reply, but my today’s update don’t show any acceleration, with 500 days to exit US1.Storj. The curve remains hopelessly and absolutely flat.
I’m afraid that if nothing changes by the end of the month, this will turn into a very ungracious forced exit.
Best regards.
May be you can fill support tikker, also do you see any errors in log about exit?
put to config ?
graceful-exit.num-concurrent-transfers: 5
after addin need node restart.
number also can be bigger, depend on your network speed and pc cpu
this will make more work in paralel.
Thank you Vadim.
I use Storagenode in Docker in a Synology Nas.
I can see in the Docker Terminal panel, many scrolling messages: “Pieces transferred to newstorage node” and some rare “failed to put piece”, some “ecclient upload failed”, some “delete piece sent to trash”, and some errors like those shown by Nazutin, but scrolling is continuous and i can’t stop it, but only see them by “print screen”.
What is the right way to display, freeze and keep logs… with SSH/Putty commands ?
Can i use your command directly in SSH ? or in config.yaml parameters ?
“graceful-exit.num-concurrent-transfers: 5”
Regards.
@Alexey can you help with docker commands?
I can tell almost all about windows node, but not docker.
You can put it in config.yaml, and restart the node. Check first if the parameter is already there, uncomment it (delete the "# "and “space” from the beginig of the line), and change the value. If is not, add the Vadim’s line at the end of the file. Upload it with override to synology and restart the node.
I also run Synologys; my oldest node (ian. 2021) has on the sat that I GE from 1.5TB, and now is at 34% after a week. The younger one, with only 1TB on the specified sat, has already succesfuly finished after a week. For 3.5 TB maybe will take a month, can’t tell, but is definitively accelarating the transfers at the end. A few days back was only at 35%. I also had those errors in logs. No biggy. You can export the log lines to a file, or copy paste the lines, directly from DSM interface. Can’t check right now.
1 Like
The alternative of adding to the config.yaml you may also provide it as an argument --graceful-exit.num-concurrent-transfers=5 after the image name in your docker run command.
To apply the changes you need to stop and remove the container, then run it back using your full docker run command with all parameters, include the added one.
If you want to keep the NAS after this, you could upgrade your RAM to max. It helps the node performe better and maybe accelerates the GE. It also helps the NAS perform better in general.
You can check my Synology memory upgrade guide on this forum.
How can I check what parameters that already active by default?
Because in config file the graceful exit parameters have this symbol # before them which means they are not active?
Yes if it has a # infort that means its commented or in this case not active
Aha so in that case what will be the default settings if I start graceful exit?
Hello everyone. Thanks for the advices.
I did the modification in config.yaml. I will give you the result tomorrow.
I can only see that egress has very irregular speed and vary between 1 and 3 MB/s, with some spikes at 5MB/s.
For the memory, i have 4Go, only 22% of the total is used and about 450MB by storagenode only, without memory limit.
My last node, with 1.5TB on a sat, just finished GE on that satellite; yesterday I saw 34% completion, after 8 days, and now is finished. So be patience.
The linux systems use all the available memory; that what you see are programs; the rest, even if appears free, is not; is used for cache and buffers. So no matter how much you put in, it will be used for something. I have 18GB RAM for one node/one drive. The cache is like 60% of it.
In my config, that line is also commented, and has the value 5. From what I saw in other parameters, that would be the defaut value, but I don’t know what the devs had in mind. Maybe is put there like a sugestion, not just to show the default.
You may check default values with the setup --help command, i.e.
docker exec -it storagenode ./storagenode setup --help | grep graceful
or PowerShell in case of Windows GUI:
&"C:\Program Files\Storj\Storage Node\storagenode.exe" setup --help | sls graceful
1 Like