Download succeeds for a file with upload canceled?

Node Operators troubleshooting
1

Satellite tried to upload, but cancelled it. OK, happens.

But then, it tried to download, and it succeeded. How is it possible? Or am I misreading?

2020-05-09T01:02:49.059Z        INFO    piecestore      upload started  {"Piece ID": "YKJFH5TJD6PE2NZYMLLQ62NPE6CJRM6QUQBNVKV54ESJ75ZRF6UQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Available Space": 306603519816}
2020-05-09T01:02:50.929Z        INFO    piecestore      upload canceled {"Piece ID": "YKJFH5TJD6PE2NZYMLLQ62NPE6CJRM6QUQBNVKV54ESJ75ZRF6UQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT"}
2020-05-09T01:03:01.882Z        INFO    piecestore      download started        {"Piece ID": "YKJFH5TJD6PE2NZYMLLQ62NPE6CJRM6QUQBNVKV54ESJ75ZRF6UQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2020-05-09T01:03:04.877Z        INFO    piecestore      downloaded      {"Piece ID": "YKJFH5TJD6PE2NZYMLLQ62NPE6CJRM6QUQBNVKV54ESJ75ZRF6UQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
1 Like
2

2 reasons why you see upload canceled

  1. Your node lost the race so the piece was not uploaded to your node.

  2. Your node didn’t get upload successful acknowledgement in timely fashion.

The 2nd reason seems to be applicable here.

3

But the upload was actually successful? In that case the log message should probably be adjusted. There are many of us who have real-time log analysis to tell us about upload/download success rates on our nodes and an incorrect log line skews the statistic incorrectly towards failed uploads.

5

The log message is technically correct. The upload was cancelled (likely during longtail cancellation). However, if the upload finishes before that cancelled message is processed, the node will still report that success to the satellite and the piece is still there. I guess long term this will save some repair by having higher than required redundancy to begin with. You’re still getting paid for this storage as well as downloads that will follow, so it’s good for everyone.

6

Then it seems like success should be reported in the logs when it is reported to the satellite.

1 Like