Successrate.sh comparison thread

Yes, AFAIK Storj has to store each file 2.8 times to ensure it’s available, so for each request from a user, there would be 2.8 nodes that get a request for it. I assume that Storj sends requests to multiple nodes to ensure customers get a fast response instead of sending a request to one node - waiting - and then sending it to another node if the first doesn’t deliver. So based on that a success rate of 35% would mean an even distribution of request!?

If load and latency would influence how requests are distributed and so on, Storj users that have their HDD connected via USB or use slower hardware like a Raspberry Pi would have “bad cards” or not?
Also, nodes with, for example, 1GBit up- and download would have an advantage over users with slower connections?

Yeah, seeing also that almost everything just works perfectly since I changed my slow ADSL connection to a bigger cable connection:

Hardware : Synology DS1019+ (INTEL Celeron J3455, 1.5GHz, 8GB RAM) with 20.9 TB in total SHR Raid and 1TB SSD Cache (2x WDS100T3X0C-00SJG0 NVMe)
Bandwidth : Home Cable Connection with 300mbit/s down and 40mbit/s up
Location : Amsterdam
Node Version : v1.14.7
Uptime : 297h41m50s (but didnt delete logs last time)
max-concurrent-requests : DEFAULT
successrate.sh :

========== AUDIT ============== 
Critically failed:     0 
Critical Fail Rate:    0.000%
Recoverable failed:    0 
Recoverable Fail Rate: 0.000%
Successful:            25380 
Success Rate:          100.000%
========== DOWNLOAD =========== 
Failed:                50 
Fail Rate:             0.005%
Canceled:              75 
Cancel Rate:           0.008%
Successful:            948576 
Success Rate:          99.987%
========== UPLOAD ============= 
Rejected:              0 
Acceptance Rate:       100.000%
---------- accepted ----------- 
Failed:                36 
Fail Rate:             0.004%
Canceled:              1646 
Cancel Rate:           0.167%
Successful:            982531 
Success Rate:          99.829%
========== REPAIR DOWNLOAD ==== 
Failed:                0 
Fail Rate:             0.000%
Canceled:              0 
Cancel Rate:           0.000%
Successful:            104206 
Success Rate:          100.000%
========== REPAIR UPLOAD ====== 
Failed:                0 
Fail Rate:             0.000%
Canceled:              2 
Cancel Rate:           0.003%
Successful:            75277 
Success Rate:          99.997%
========== DELETE ============= 
Failed:                0 
Fail Rate:             0.000%
Successful:            163670 
Success Rate:          100.000%
1 Like

I am getting 0 for everything when I run ./successrate.sh storagenode3 the node name is definitely correct and I have tried with sudo. Any ideas?

not sure you can run it just on node name… but you can use a log file instead, if you are exporting your logs.

./successrate.sh /temp/logdir/logfile.log
or such

if you aren’t exporting your logs you can use something like this for docker.
i suggest this command line because you can basically adapt it to do whatever

docker logs --since 24hr --until 2020-11-13 storagenode >& /temp/logdir/logfile.log

else this will export whatever docker has saved, tho keep in mind this can take a bit, shouldn’t be anything major, but a few minutes atleast.

docker logs storagenode >& /tmp/logs/storagenode.log

you can also remove the -until parameter and date, if you just want the last 24, 48, 1 hour whatever… pretty self evident how it works… and there are ofc more options in the docker documentation.

That has worked perfectly. Many thanks!

1 Like

I usually update my node manually after checking the github release of a new version.
The updater usually never updates the node correctly, I don’t know why. This is a windows box.

I run the successrate script on the old storagenode.log, here are the results

03-23-2021 - 04-11-2021
========== AUDIT =============
Successful: 3192
Recoverable failed: 0
Unrecoverable failed: 0
Success Min: 100%
Success Max: 100%
========== DOWNLOAD ==========
Successful: 23551
Failed: 35
Success Rate: 99.8516068854405
========== UPLOAD ============
Successful: 638431
Rejected: 0
Failed: 532
Acceptance Rate: 100
Success Rate: 99.9167400929318
========== REPAIR DOWNLOAD ===
Successful: 5629
Failed: 0
Success Rate: 100
========== REPAIR UPLOAD =====
Successful: 5629
Failed: 10
Success Rate: 99.9928462589511

1 Like

looks good, a few failed in upload… but nothing i would worry even the slightest about…
really can’t expect much better numbers than that.

Small node (1.6TB) installed on a Windows 10 computer. Home installation.

========== AUDIT =============
Critically failed:      0
Critical Fail Rate:     0.00%
Recoverable failed:     0
Recoverable Fail Rate:  0.00%
Successful:             34180
Success Rate:           100.00%
========== DOWNLOAD ==========
Failed:                 46
Fail Rate:              0.07%
Canceled:               2506
Cancel Rate:            3.77%
Successful:             63874
Success Rate:           96.16%
========== UPLOAD ============
Rejected:               0
Acceptance Rate:        100.00%
---------- accepted ----------
Failed:                 270
Fail Rate:              1.48%
Canceled:               36
Cancel Rate:            0.20%
Successful:             17886
Success Rate:           98.32%
========== REPAIR DOWNLOAD ===
Failed:                 0
Fail Rate:              0.00%
Canceled:               0
Cancel Rate:            0.00%
Successful:             12411
Success Rate:           100.00%
========== REPAIR UPLOAD =====
Failed:                 0
Fail Rate:              0.00%
Canceled:               0
Cancel Rate:            0.00%
Successful:             909
Success Rate:           100.00%

Greetings from much months later. I just ran successrate.sh for the first time (and this is after there is no more deluge of test data, so this is under “normal” light load)

TL;DR: my slower nodes have a somewhat lower success rate for DOWNLOADS (egress) than the faster nodes, but not broken.

All my nodes use the same storage backend, which are independent ZFS disks with a L2ARC SSD cache, so the disks are pretty fast.

The fast node is running on a I7-12700K CPU with a good amount of ram. The slow node is only a single core, only 1GB of RAM, and the drives are mounted as NFS shares which adds a bunch of latency.

Here is fast node info:

========== AUDIT ==============

Critically failed:     0
Critical Fail Rate:    0.000%
Recoverable failed:    0
Recoverable Fail Rate: 0.000%
Successful:            109858
Success Rate:          100.000%
========== DOWNLOAD ===========
Failed:                24250
Fail Rate:             0.597%
Canceled:              23775
Cancel Rate:           0.586%
Successful:            4012315
Success Rate:          98.817%
========== UPLOAD =============
Rejected:              0
Acceptance Rate:       100.000%
---------- accepted -----------
Failed:                16392
Fail Rate:             0.545%
Canceled:              7570
Cancel Rate:           0.252%
Successful:            2985751
Success Rate:          99.204%
========== REPAIR DOWNLOAD ====
Failed:                1
Fail Rate:             0.001%
Canceled:              1
Cancel Rate:           0.001%
Successful:            140788
Success Rate:          99.999%
========== REPAIR UPLOAD ======

Failed:                377
Fail Rate:             1.922%
Canceled:              56
Cancel Rate:           0.286%
Successful:            19179
Success Rate:          97.792%
========== DELETE =============
Failed:                0
Fail Rate:             0.000%
Successful:            6001687
Success Rate:          100.000%

and here is data from my slowest node:

========== AUDIT ==============
Critically failed:     0
Critical Fail Rate:    0.000%
Recoverable failed:    0
Recoverable Fail Rate: 0.000%
Successful:            15036
Success Rate:          100.000%
========== DOWNLOAD ===========
Failed:                59120
Fail Rate:             2.016%
Canceled:              769109
Cancel Rate:           26.231%
Successful:            2103772
Success Rate:          71.752%
========== UPLOAD =============
Rejected:              55721
Acceptance Rate:       98.603%
---------- accepted -----------
Failed:                41207
Fail Rate:             1.048%
Canceled:              10940
Cancel Rate:           0.278%
Successful:            3880463
Success Rate:          98.674%
========== REPAIR DOWNLOAD ====
Failed:                1
Fail Rate:             0.010%
Canceled:              0
Cancel Rate:           0.000%
Successful:            9614
Success Rate:          99.990%
========== REPAIR UPLOAD ======
Failed:                2451
Fail Rate:             2.685%
Canceled:              436
Cancel Rate:           0.478%
Successful:            88391
Success Rate:          96.837%
========== DELETE =============
Failed:                0
Fail Rate:             0.000%
Successful:            0
Success Rate:          0.000%