Graceful Exit Guide

Product Discussions STORJLINGS
1

Warning: Read this carefully before you start
Please read the following information carefully and ask any questions your might have before executing graceful exit. Once graceful exit is started there is no way to stop or cancel it. If graceful exit is failing even at 99% you will not get any hold back amount. Think about the consequences twice before you call it.
Requirements:

  • Storage node has joined the network more than 15 month ago (requirement temporary reduced to 6 month).
  • Storage node is healthy and hasn’t lost data. Disqualification during graceful exit is possible.
  • Storage node has no huge downtime. If the storage node is offline for a few days while running graceful exit it will fail.
  • Storage node has no bandwidth caps. Graceful exit doesn’t care about any bandwidth caps. It will go on with the transfer even if the bandwidth cap is exceeded. You are not allowed to take the storage node offline or blocking the traffic. Disqualification would be the result for that. You are allowed to call graceful exit in the middle of the month to split the traffic over 2 months. You are allowed to call graceful exit per satellite to split the traffic even further.
  • Graceful exit traffic is unpaid. If you finish graceful exit successfully you will get your hold back amount.

Start Graceful Exit
Are you sure you want to start graceful exit? Did you read the information above? Do you understand that graceful exit can’t be canceled?
Here is how you call it:

root@kali:~# storagenode exit-satellite
Please be aware that by starting a graceful exit from a satellite, you will no longer be allowed to participate in repairs or uploads from that satellite. This action can not be undone. Are you sure you want to continue? y/n

: y
Domain Name                       Node ID                                              Space Used
satellite.stefan-benten.de:7777   118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW   1.8 TB
asia-east-1.tardigrade.io:7777    121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6  37.9 GB
us-central-1.tardigrade.io:7777   12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S  38.7 GB
europe-west-1.tardigrade.io:7777  12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs  0.8 TB
Please enter a space delimited list of satellite domain names you would like to gracefully exit. Press enter to continue:

satellite.stefan-benten.de:7777 asia-east-1.tardigrade.io:7777 us-central-1.tardigrade.io:7777 europe-west-1.tardigrade.io:7777

You can exit the satellites one by one or all at the same time. The satellite will need a few hours to create the list of pieces that needs to be transfered. Don’t expect high traffic immediately.

During Graceful Exit
You can watch the status of graceful exit like this: (example output from a test satellite)

root@kali:~# storagenode exit-status 

Domain Name      Node ID                                              Percent Complete  Successful  Completion Receipt
127.0.0.1:10000  12fbck97kqEGbWPu673CpeyrXavtqgVriyv9pCfL3mpw3yz2zN9  0.00%             N           N/A

If you get the output No graceful exit in progress. that means graceful exit didn’t start because of the node age requirement. In the storage node logs you will find additional information node is not yet eligible for graceful exit: will be eligible after 2020-04-02 01:18:23.910919 +0000 UTC.
In case of a crash, power failure or other outage please get your storage node back online. Graceful exit will continue.

Concurrency
By default the storage node will transfer 1 piece at a time. You can increase that via config. graceful-exit.num-concurrent-transfers: 5

Finish Graceful Exit
At the end you will get this output: (example output from a test satellite)

root@kali:~# storagenode exit-status

Domain Name      Node ID                                              Percent Complete  Successful  Completion Receipt
127.0.0.1:10000  12fbck97kqEGbWPu673CpeyrXavtqgVriyv9pCfL3mpw3yz2zN9  100.00%           Y           0a473045022100da86329cfb4f5bb16f0702c1d073c3a8b54787311b54855bcf01a8e245250040022003ef911b3b2b2bea86ba34cd4927223f2718cd35c3b7de7cc030cd3a8ce4959a1220db55bd9fa76e8938be5a7a25c970d48bde19936e269dcf69a3ab9fa41b5486001a207508f9a6138cdc4089ea075f1553736d472cb1d3afa4397496a8eb948d121200220c08abe5dcf0051086e6fefe01

In some situations the storage node doesn’t need to transfer 100% of the data and might finish graceful exit with a lower percentage. As long as graceful exit was successful you will get your hold back amount with the next regular payout. “Completion Receipt” contains a signature from the satellite and is your ticket to get the payback. Please keep your storagenodeID, each satelliteID and each completion receipt. With these information your can open a support ticket if needed.

23 Likes
2

Highly useful - thanks for the clear explanation

3

On my end i feel like a exit-verify could be useful to detect if it’s worth exiting or continuing to run a node that has lost data, but it otherwise in good shape. (ex: accidental temporary storage lost in docker)

This would be in hopes that the pieces lost are deleted before the audits cause the DQ, but while exiting now, may cause a DQ.

1 Like
4

if one of the satellites audits are at 99.9 % does that constitute a not healthy node and graceful exit will fail.

5

You can simply keep your storage node running. If it gets DQed than you know that graceful exit wouldn’t have saved you. If you don’t get DQed you know that you don’t need graceful exit.

Graceful exit will not prevent your node from getting DQed.

1 Like
6

Just ask yourself what you are going to do if graceful exit has a 50% change to DQ you. Would you leave the network anyway or would you stay? If you are going to leave the network one way or the other you can at least try to collect the hold back amount. If you would stay than that is what you should do instead of thinking about graceful exit.

7

Just curious what is the command for storj GUI graceful exit?

8

The question then, is the time it takes to get DQ’d by staying on the network worth the risk by not just going ahead and exiting and starting over to avoid lost escrow holdings.

Ex. If it takes 2 months to detect a DQ, Those two months may be better spent on a new node to save your escrow. As a generic rule if you’re a long time member, it would be worth sticking around due to the less withholding in escrow, but if you have a fresh node and have any doubts at all, it might just be worth exiting and starting over to reduce revenue losses over time.

EDIT: I just realized that’s not possible due to the 6 month / 12 month limit on allowing graceful exit.

Was it considered to allow graceful exit at these points, and if at risk for DQ (say you can only repair 80% of the files) Share 80% of the revenue in escrow.

2 Likes
9

Not understand why need to make GE so hard? Timing is understandable. But requaerments of 100% could be made flexible. For if you have 10TB on disk, Made Exit, and uploaded back 9 TB.
You will have tiket that 9 TB is ok and 1 TB is lost. 1TBx10$(repear upload) then it normal mathematics. Escrow - 10$ = Return to SNO. It is understadable than Sattelite have to do also some work and use trafic, Just add it also to the buill Escrow - 10$ - (Setellite file reconstruction cost) = Return to SNO. Same thing DQ it could start just GE, and all can be saved need to save, not just throw away if some data is lost. It is more cost and time effective then repair all node data from scratch.

4 Likes
10

It was considered but that would allow bad storage nodes to avoid the DQ penalty. If you are getting DQed even at 99% you will get 0% from the hold back amount. We want that penalty. We want it to be a lesson.

3 Likes
11

4 posts were split to a new topic: High requirements for storagenode

15

Try this: (path to config and identity will be different)
"C:\Program Files\Storj\Storage Node\storagenode.exe" exit-status --identity-dir "C:\Users\Skunk\AppData\Roaming\Storj\Identity\storagenode" --config-dir "C:\Program Files\Storj\Storage Node" --log.output stderr

16

Do you need someone to test for windows GUI?

17

Not in production. Setup a testnode and wait 6 month is not a good way to test it. Lets wait for storage nodes that want to exit the network and offer them a graceful exit.

18

Id be willing to do it next month as My windows node was only for testing to help people who had issues with there docker node, so I just converted it to The gui

1 Like
19

I also have two nodes running as my 1st one didn’t respond properly due to a mem/swap issue and I got DQed at two satellites, the other two were still running.
I’ve setup the 2nd node and my plan was once graceful exit is available I’ll do that as I don’t want to run two… The new one is vetted except the stefan-benten (118UWp) has only 66/66 so far.
Couple of questions:

  • The two satellites were my 1st node got disqualified, there’s nothing to be gracefully exited right? :slight_smile:
  • How can I find out again how long my 1st node is active? Need to make sure it’s 6 months +?
  • If I gracefully exit the 1st node, is there any ‘penalty’ I get payout wise as the 1st one still holds 2.2 TB where as the new one which is just some weeks old holds only 120.6 GB
  • Also from the 2.2 TB I still have one the 1st node, is there still data on there from the two DQed satellites that never got deleted?

For now I’m just waiting to be completely vetted on the last satellite before giving it a try.

20
  1. You are right.
  2. You can call graceful exit and if you are not 6 month old you will get a message telling you which date it would be.
  3. What do you mean with penalty?
  4. Possible. Graceful exit should just skip the satellites that have DQed you and move only the data that is still valid.
1 Like
22

I am using linux server, what commands I need to use for performing Graceful exit.
Actually I dont have that much linux knowledge, I use this command for checking my dashboard, I request can you guys mention the exact commands I need to follow for Graceful Exit.
sudo docker exec -it storagenode /app/dashboard.sh

24

image Actually its not working

25

Sorry I havent actually ran it yet @littleskunk would know the actual command for it best wait for him.