Graceful exit overshoot issue

Node Operators troubleshooting
1

Hi!

I’m attempting a graceful exit because i got disqualified on one sat and want to use a better setup.

Do you think this is recoverable at all? Or should I just call it quits? I’ve tried to restart the node a couple of times.

This was my starting point:
Domain Name Node ID Space Used satellite.stefan-benten. de:7777 118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW 19.2 GB asia-east-1.tardigrade. io:7777 121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 0 B us-central-1.tardigrade. io:7777 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S 23.4 GB europe-west-1.tardigrade. io:7777 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs 262.6 GB

A few days in it looks like this:
Domain Name Node ID Percent Complete Successful Completion Receipt satellite.stefan-benten. de:7777 118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW 115.63% N N/A us-central-1.tardigrade. io:7777 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S 117.57% N N/A europe-west-1.tardigrade. io:7777 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs 6.51% N N/A

Some overshoot is possible I guess, but 117.57%?

There seems to be multiple issues, a few of these “out of space”:
2020-02-14T17:51:00.345Z e[31mERRORe[0m gracefulexit:chore failed to transfer piece. {"Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "protocol: out of space", "errorVerbose": "protocol: out of space\n\tstorj .io/uplink/piecestore.(*Upload).Write:160\n\tbufio.(*Writer).Flush:593\n\tbufio.(*Writer).Write:629\n\tstorj.io/uplink/piecestore.(*BufferedUpload).Write:32\n\tstorj .io/uplink/piecestore.(*LockingUpload).Write:89\n\tio.copyBuffer:404\n\tio.Copy:364\n\tstorj .io/common/sync2.Copy:22\n\tstorj .io/uplink/ecclient.(*ecClient).PutPiece:240\n\tstorj .io/storj/storagenode/gracefulexit.(*Worker).transferPiece:213\n\tstorj .io/storj/storagenode/gracefulexit.(*Worker).Run.func2:111\n\tstorj .io/common/sync2.(*Limiter).Go.func1:41"}

And also a few ERRORs:
2020-02-14T17:55:21.431Z e[31mERRORe[0m blobscache newSatPiecesTotal < 0 {"newSatPiecesTotal": -18688} 2020-02-14T17:55:21.431Z e[31mERRORe[0m blobscache newSatPiecesContentSize < 0 {"newSatPiecesContentSize": -18176}

Some advice would be appreciated.

Thanks!

2

I would definitely keep it going. This is much more likely to be a calculation issue than an actual failure in the GE process. Unfortunately I can’t help with the specific errors you’re seeing, but I think chances are still decent it will complete successfully.

3

Hi again.

Close to 200% now :confused:

Any chance that a storjling could take a peak at my node from your side?

2020-02-18T18:23:47.137Z        INFO    Configuration loaded from: /app/config/config.yaml
2020-02-18T18:23:59.804Z        INFO    Node ID: 12aBy9X83kYKgsams1BXPLmkg1uUJ2qTJ6xRCM7ib2mgghzRMFJ

Domain Name                       Node ID                                              Percent Complete  Successful  Completion Receipt
satellite.stefan-benten.de:7777   118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW   191.82%           N           N/A  
us-central-1.tardigrade.io:7777   12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S  158.43%           N           N/A  
europe-west-1.tardigrade.io:7777  12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs  27.58%            N           N/A
4

@littleskunk Your insight please. Thank you.

5

Hey, it took a couple of weeks, but it worked:

Domain Name                       Node ID                                              Percent Complete  Successful  Completion Receipt
satellite.stefan-benten.de:7777   118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW   100.00%           Y           <redacted>
us-central-1.tardigrade.io:7777   12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S  100.00%           Y           <redacted>
europe-west-1.tardigrade.io:7777  12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs  100.00%           Y           <redacted>
1 Like
6

Glad to hear it, thanks for reporting back!

7

So I got my payment this month, thank you very much. Does anyone know how to differentiate escrow pay outs vs. regular ones?

8

Not exactly, but you can use the earnings calculator to calculate the part you should have received normally.