Endless graceful exit

Node Operators troubleshooting
21

Hello Alexey, i tested your command, this is the result (shortened):
(# docker exec -it storagenode ./storagenode setup --help | grep graceful)
–graceful-exit.chore-interval duration (default 1m0s)
–graceful-exit.min-bytes-per-second memory.Size (default 5.00 KB)
–graceful-exit.min-download-timeout duration (default 2m0s)
–graceful-exit.num-concurrent-transfers int (default 5)
–graceful-exit.num-workers int (default 4)
As i didn’t really change the concurrent transfer value, already set to 5, the egress speed and the GE expected duration didn’t change.
What other values do you suggest to speed up the process ?

(The current situation)

Storj GE 4
Storj GE 41408×535 129 KB

Storj GE 5
Egress is very slow compared to my bandwith, and pre-GE period.
Storj GE 6

22

Parallel transfers, but if your router is not capable to handle hundreds open connections, your GE may fail and node will be disqualified.
You may also increase minimum transfer speed, but it will increase chances to be disqualified if no nodes will meet this requirement.
So be careful with changing parameters, you would need to monitor a number of failed transfers in your logs. Each piece is uploading with 5 retries, if it’s not transferred, the transfer considered as failed. With 10% failures the node will be disqualified.
So, you need to monitor your logs for “no route to host” or great number of “i/o timeout”, in this case you would need to reduce changed parameters.

1 Like
23

I have been observing that EU satellites went much faster in my case, US1 is taking like forever.

EU1 and US1 had about the same amount of storage used, but with both starting at the same time in a node with 540 GB occupied:

  • EU1 finished in 4.5 days
  • US1 still has 95 GB pending after 7 days.

Why US1 is much slower? Does it have smaller pieces (so more pieces to transmit) or something similar?

1 Like
24

Hi ! Some news. I tested different values for concurrent-transfers.
-15 is too much with many errors
-12 also but less errors.
-10 is ok with the same error rate than the 5 value, with still very irregular upload.

  • I tested it during 30h with no significant results and egress acceleration.
    Based on Egress curve (see hereunder), I think GE egress speed depends entirely on the servers, and GE SNO may have the lowest priority, to enhance the “normal” SNO performances.
    It is legitimate but will lead for me to a one year graceful exit for EU1 and US1 servers.
    I cannot wait so much, to get only 6$ at the end.
    I give up and wish good luck to the guys who stay, and thank you all for your help.
    Jacques.
    Storj GE 7
    Storj GE 7870×385 29.3 KB
1 Like
25

I have forgotten a last question:
Due to a fiber optics ISP cabling problems, i have been offline during thee days, just after beginning GE.
I received during those days, many emails: “Your Node has Gone Offline”,
followed three days later by: “Your Node is Back Online”.
How to prevent those sending, when i will disconnect my server form Storj network ?
Thank you.

26

Please look at the bottom of one of those emails, there should be an unsubscribe link.

2 Likes
27

:+1:. . . . . . . . . :wink: