Lot of canceled uploads in logs

Hey!
I have recently seen the strange behavior of my node. A very large part of the upload is rejected. I put a screen from successrate.sh. It’s something wrong? Is the node too weak? Is the disk too slow?

Your upload rate is indeed very small… The download is fine though which is interesting.

Can you tell us more about your setup? What CPU/Hardware, HDD, how is the HDD connected, internet connection, …

Since you mentioned in the other topic that you’re currently using an SMR drive, I fear that this is a bottleneck on your node. But to be sure lets go over some other stuff.

How is the HDD connected?
What kind of internet speeds do you have?
Have a look at the resource use on your system. Is there anything that stands out? Specifically CPU Wait IO?

My node:
Raspberry Pi 3B
internet: 20/20 Mbps (stable)
1TB WD Blue 2.5" (SMR) connected to raspi USB via an adapter with good speed.

image

image

Internet is sufficient but will definitely not catch all the ingress because it was very often in the range of 15-30Mbps. So I can imagine that a lot of canceled uploads might also be due to your internet, especially if you download other things. That’s why the downloads of the node might still be ok because you typically don’t use the network upload much.

My first instinct would be to blame the USB2.0 connection but your iowait seems really low… Even I have higher iowait. And if the SMR drive is to blame, there should be higher iowait too.

Maybe running netdata might reveal more information. And checking the logs of the storagenode.

What’s the age of the node, and is it the only one you have running behind your public IP? Also, what’s the uptime that resulted in those success rate numbers?

I as these two questions as your successrate output looks very similar to my second node that was started up ~20 days ago. Still being vetted on all satellites except Salklake, and has had very little traffic coming to it, with ~98% download success and ~4-5% upload success…

I have noticed a lot of upload cancellations in the logs, ingress usage seems to be running very high compared to egress over the last 3 days and the upload success rate taken from the script is about 20%, would this indicate an issue with the node? Node running on nuc pc win10 ssd running under normal utilization. Node was set up Sep 2019.

Any insight would be much appreciated :slight_smile:


Looks like the canceled uploads are taking up space and it has been marked as a bug on github. Maybe canceled doesn’t mean canceled after all and just a bug in logging?

2 Likes