Little audited and usually failed

I started operating storage nodes last week.
Just one week has passed, but the number of audits is just 30. (Sum of all satellites)
I have read that 100 audits from each satellite are required until the node is accepted.
I knew it was about a month, but it would take a year.

For reference, I will attach logs from the last 24 hours

$ ./successrate.sh storagenode
========== AUDIT =============
Successful: 3
Recoverable failed: 0
Unrecoverable failed: 0
Success Rate Min: 100.000%
Success Rate Max: 100.000%
========== DOWNLOAD ==========
Successful: 77
Failed: 38
Success Rate: 66.957%
========== UPLOAD ============
Successful: 1002
Rejected: 0
Failed: 25
Acceptance Rate: 100.000%
Success Rate: 97.566%
========== REPAIR DOWNLOAD ===
Successful: 0
Failed: 0
Success Rate: 0.000%
========== REPAIR UPLOAD =====
Successful: 13
Failed: 0
Success Rate: 100.000%

If I have any problem. Please tell me.

You dont have any problems audits take time as they are random it can take up to a month or more.

Okay.
I understand that the audit can take more than a month.
However, the total traffic was about 10GB in one week.
This feels very small, but is it normal for a new node?
(If you look at a lot of threads, it feels like more traffic is flowing.)

Yes but right now theres alot less data incoming because the testing is focused on downloading from the nodes. If your talking about egress yes its very busy im running about 200Gigs perday there around. But as for ingress is very low right now.

From the data shown at the beginning, the download success rate is about 66%. Is that OK?

It seems like its ok Just means someone its getting it before you, So you get context canceled.
It depends on alot of things location hardware internet etc.

I have a pretty new node I can test and show you what its getting as of right now.

Yes.
I understand it.
My node is in Japan, surrounded by the sea.
In addition, there are not many Storj users.
I think this is the biggest factor in truncating my node.

I use the fastest network possible, but there are limitations.
(Speed ​​is fast, but RTT is slow, because it is surrounded by the sea)
Thanks to you, I was reassured that my node was not in an abnormal state.

So heres mine

========== AUDIT =============
Successful:           0
Recoverable failed:   0
Unrecoverable failed: 0
Success Rate Min:     0.000%
Success Rate Max:     0.000%
========== DOWNLOAD ==========
Successful:           3
Failed:               0
Success Rate:         100.000%
========== UPLOAD ============
Successful:           3
Rejected:             0
Failed:               1
Acceptance Rate:      100.000%
Success Rate:         75.000%
========== REPAIR DOWNLOAD ===
Successful:           0
Failed:               0
Success Rate:         0.000%
========== REPAIR UPLOAD =====
Successful:           2
Failed:               1
Success Rate:         66.667%

The node is about 4 days old now

Edit: The data is useless sorry the node updated.

Yes
I updated the node an hour ago.

========== AUDIT =============
Successful: 0
Recoverable failed: 0
Unrecoverable failed: 0
Success Rate Min: 0.000%
Success Rate Max: 0.000%
========== DOWNLOAD ==========
Successful: 7
Failed: 3
Success Rate: 70.000%
========== UPLOAD ============
Successful: 1
Rejected: 0
Failed: 2
Acceptance Rate: 100.000%
Success Rate: 33.333%
========== REPAIR DOWNLOAD ===
Successful: 0
Failed: 0
Success Rate: 0.000%
========== REPAIR UPLOAD =====
Successful: 0
Failed: 1
Success Rate: 0.000%

All requests have failed, but may need to be monitored a bit.

A few hours have passed since then, but Upload failed continues to increase and worsen.

========== AUDIT =============
Successful: 1
Recoverable failed: 0
Unrecoverable failed: 0
Success Rate Min: 100.000%
Success Rate Max: 100.000%
========== DOWNLOAD ==========
Successful: 11
Failed: 10
Success Rate: 52.381%
========== UPLOAD ============
Successful: 2
Rejected: 0
Failed: 19
Acceptance Rate: 100.000%
Success Rate: 9.524%
========== REPAIR DOWNLOAD ===
Successful: 0
Failed: 0
Success Rate: 0.000%
========== REPAIR UPLOAD =====
Successful: 1
Failed: 1
Success Rate: 50.000%

Interestingly, I found a strange log.

2020-01-30T07:21:38.647Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {“count”: 4}
2020-01-30T07:21:38.647Z INFO orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S sending {“count”: 2}
2020-01-30T07:21:38.647Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs sending {“count”: 1}
2020-01-30T07:21:39.215Z INFO orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S finished
2020-01-30T07:21:39.514Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs finished
2020-01-30T07:21:39.536Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished

Usually there are four satellites.

2020-01-30T07:49:49.597Z INFO orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S sending {“count”: 2}
2020-01-30T07:49:49.597Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {“count”: 2}
2020-01-30T07:49:49.597Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs sending {“count”: 2}
2020-01-30T07:49:49.598Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 sending {“count”: 1}
2020-01-30T07:49:49.719Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 finished
2020-01-30T07:49:50.016Z INFO orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S finished
2020-01-30T07:49:50.319Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs finished
2020-01-30T07:49:50.416Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished

You can compare to my node

========== AUDIT =============
Successful:           6
Recoverable failed:   0
Unrecoverable failed: 0
Success Rate Min:     100.000%
Success Rate Max:     100.000%
========== DOWNLOAD ==========
Successful:           88
Failed:               0
Success Rate:         100.000%
========== UPLOAD ============
Successful:           40
Rejected:             0
Failed:               65
Acceptance Rate:      100.000%
Success Rate:         38.095%
========== REPAIR DOWNLOAD ===
Successful:           0
Failed:               0
Success Rate:         0.000%
========== REPAIR UPLOAD =====
Successful:           3
Failed:               9
Success Rate:         25.000%

Just a question what kinda internet connection, hardware your node is running on?

They are running on servers with 16 core CPUs.
Memory is 128GB.
Both the download and upload use an optical line with a speed of over 600Mbps.

========== AUDIT =============
Successful: 2
Recoverable failed: 0
Unrecoverable failed: 0
Success Rate Min: 100.000%
Success Rate Max: 100.000%
========== DOWNLOAD ==========
Successful: 11
Failed: 6
Success Rate: 64.706%
========== UPLOAD ============
Successful: 10
Rejected: 0
Failed: 32
Acceptance Rate: 100.000%
Success Rate: 23.810%
========== REPAIR DOWNLOAD ===
Successful: 0
Failed: 0
Success Rate: 0.000%
========== REPAIR UPLOAD =====
Successful: 3
Failed: 6
Success Rate: 33.333%

Just keep an eye on your logs and make sure not everything is failing as long as data is finishing it should be ok.

@networkdd Since the successrate.sh script uses the logs to grab audit data, you might want to use the following command to grab the audit results from the api (copy/paste into command line).

for sat in `wget -qO - localhost:14002/api/dashboard | jq .data.satellites[].id -r`; do wget -qO - localhost:14002/api/satellite/$sat | jq .data.id,.data.audit; done

These values persist across updates and give you an idea of how the node is doing even if the logs get cleared.

I have already verified AUDIT using the API.
However, these values ​​are unreliable because of the following issues:

https://forum.storj.io/t/cannot-update-audit-count-in-api/4228

(AUDIT total 32 at now)

1 Like

A post was merged into an existing topic: Error piercestore protocol: rpc error: code = canceled desc = context canceled