7MB per download, 0.8MB per upload on average. Why?

Node Operators troubleshooting
1

After about two days after the host restart I have the following numbers:

$ curl localhost:7777/mon/funcs
...
[...] storj.io/storj/storagenode/piecestore.(*Endpoint).doDownload
  current: 6, highwater: 12, success: 24185, errors: 16896, panics: 0

24185+16896=41081 download attempts.

$ ifconfig docker0
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    RX packets 118214883  bytes 290404238731 (270.4 GiB)
    TX packets 71212167  bytes 32009474797 (29.8 GiB)

290404238731 bytes / 41081 downloads = 7069064 bytes = 7MB per download attempt on average.

doUpload had 37891 successes and 560 errors, total 38451 attempts.
On average 32009474797 bytes / 38451 uploads = 832474 bytes/upload = 0.8MB/upload (something like 8.5x less bytes per operation than in downloads).

I understand that some portion of this traffic is not related to downloads but 7MB still seems unexpectedly larger number than 2.3MB per piece as stored on my local disk and than 0.8MB per upload.

Can anybody shed some light on this discrepancy?
Thanks!

2

Every file which get uploaded is splitted into 64MB segments, every segment into 29 pieces. That are defaults. Depending on the uploaded filesize a node will get pieces between 5KB and 2,2MB.

That is the default behavior. In december were a lot data uploaded with a different metric, which right now get downloaded!

This is the reason why your downloads are larger than your uploads. If you did this calculation in december it would look different.

1 Like
3

Thanks for the explanation!
I indeed can find files in blobs/ directory with sizes >7MB in this case.

4

These blobs are the data which get transfered either for up or download. There some edge cases like audits or partial downloads but typically the full pieces should be transferred as log your node is fast enought and didn’t get canceled while uploading.

5

Out of curiosity I built a distribution of file sizes in my blobs/ directory:

  1k: 190668
  2k: 101621
  4k:  88516
  8k:  66608
 16k:  61292
 32k:  60998
 64k:  62332
128k:  77795
256k: 123921
512k:  53416
  1M: 219292
  2M: 1293863
  4M:   3335
  8M:  16809
 16M:    811

Which suggests that pieces can be as small as 1-2 kb and max size for my node is so far 20995328 bytes.

2 Likes