# 100% Disk workload without any traffic

**URL:** <https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952>\
**Category:** troubleshooting\
**Created:** [February 29, 2020, 9:11pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952 "2020-02-29T21:11:05Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [February 29, 2020, 9:11pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/1 "2020-02-29T21:11:06Z")

</div>

My Node is causing 100% workload on my disk without having any traffic, not a single piece is getting up- or downloaded since 2 days (February 28th-29th). And yes, the node seems to be online (according to CLI Dashboard and [uptimemonitor.com](http://uptimemonitor.com))  
Acording to the system monitor the workload is mainly caused by reading processes.

I also noticed that there were lot of “upload failed”-errors (12004 in 2 days (February 26th-27th) or a successrate of 35%) and suspect that it has to do with the high workload.

And how could it be differently, after hours of thinking “maybe it will end soon”, and finally writing this post, all of a sudden the workload stops. Now there’s no workload (nor traffic)

Anybody has an idea whats going on with my node?

---

<div class="post-metadata">

**Author:** ![Vadim](https://storj.bcdn.literatehosting.com/letter_avatar/vadim/32/5_5575768a8748004e209b776fc1b2916d.png) [@Vadim](https://forum.storj.io/u/Vadim)\
**Post date:** [February 29, 2020, 9:21pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/2 "2020-02-29T21:21:40Z")

</div>

i think it is not only on your node, if you see space used, i think you will find out that you are geting more and more space free. Today i see than my node got 50GB free more free space.  
So there is just lot of delete operations.

---

<div class="post-metadata">

**Author:** ![naxbc](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/naxbc/32/1125_2.png) [@naxbc](https://forum.storj.io/u/naxbc)\
**Post date:** [February 29, 2020, 9:48pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/3 "2020-02-29T21:48:01Z")

</div>

Probably it’s just the “house cleaning” in preparation for Production 🤗  
It should be about to “hit the fan” 🕶

---

<div class="post-metadata">

**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\
**Post date:** [February 29, 2020, 10:16pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/4 "2020-02-29T22:16:40Z")

</div>

It may also be the “move” operation to trash, because the node for some reason copies the file and the deletes the original instead of just moving it.

---

<div class="post-metadata">

**Author:** ![Vadim](https://storj.bcdn.literatehosting.com/letter_avatar/vadim/32/5_5575768a8748004e209b776fc1b2916d.png) [@Vadim](https://forum.storj.io/u/Vadim)\
**Post date:** [March 1, 2020, 7:42am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/5 "2020-03-01T07:42:04Z")

</div>

this is ununderstandeble overhed. seen thes place in code?

---

<div class="post-metadata">

**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\
**Post date:** [March 1, 2020, 8:42am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/6 "2020-03-01T08:42:26Z")

</div>

I did not see the part in code, but saw from the behavior - IO at 100% (write) and the trash directory filling up at about the same rate as IO write load with the total used space staying about the same.

---

<div class="post-metadata">

**Author:** ![Vadim](https://storj.bcdn.literatehosting.com/letter_avatar/vadim/32/5_5575768a8748004e209b776fc1b2916d.png) [@Vadim](https://forum.storj.io/u/Vadim)\
**Post date:** [March 1, 2020, 8:45am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/7 "2020-03-01T08:45:51Z")

</div>

then it is point to check. because it is realu undesireed behavior

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 4, 2020, 7:55am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/9 "2020-03-04T07:55:33Z")

</div>

There is no freed space, still 4TB used as before (according to CLI Dashboard).  
And so far I have not a singe byte of traffic this month.

@Pentium100 As I mentioned, the workload came mainly from reading processes, there was almost no writing process.  
And I also couldn’t understand why they would move the whole content of a disk into another directory. IMHO this would be a really unfortunate design decision for the following reasons:

- knocking out nodes by overwhelming workload decreases the entire network perfomance.
- forces hardware to brake earlier by double the read/write cycles.

Why would they do this if they also could simply mark the pieces as deleted (in the database)?

---

<div class="post-metadata">

**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\
**Post date:** [March 4, 2020, 8:44am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/10 "2020-03-04T08:44:42Z")

</div>

> [@jorma](#):
>
> And I also couldn’t understand why they would move the whole content of a disk into another directory.

That would not create a lot of IO, if it was done using the equivalent of “mv old new” instead of “cp old new && rm old”. Again, I have not looked at the code, I think this works like that from the way I saw the node behave.

> [@jorma](#):
>
> As I mentioned, the workload came mainly from reading processes, there was almost no writing process.

I missed that, sorry. The only time my node gets a lot of read IO is when it is restarted (after the whole VM is restarted so the cache is empty) as it is reading the database.

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 4, 2020, 10:45am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/11 "2020-03-04T10:45:20Z")

</div>

Guess the restart is causing this behaviour. Thanks for your answer.

In a desperate move I deleted my node and set it up again. This helped before, but not now - now the node is offline. And I can’t see why. Port forwarding is working on another port, but the SN-Port apears closed from outside.  
Well, as I read in another post it can take a while until it gets back online. I take a look again after my afternoon study session… (in 6-10 hours)

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 4, 2020, 10:50am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/12 "2020-03-04T10:50:39Z")

</div>

uh, btw, I put a invalid link in the initial post. It should be [uptimerobot.com](http://uptimerobot.com)  
…sorry for that…

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [March 4, 2020, 10:03pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/13 "2020-03-04T22:03:22Z")

</div>

> [@jorma](#):
>
> In a desperate move I deleted my node and set it up again

How you did that?  
Did you completely removed the data and identity or only docker container or the Windows GUI?

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 5, 2020, 8:49am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/14 "2020-03-05T08:49:23Z")

</div>

> [@Alexey](#):
>
> How you did that?

I removed the docker container and set it up again, but this time I added -p 14002:14002 in order to access the gui dashboard. And on the gui dashboard I could see that my node got disqualified on all satelites. That was a hard insight after running it for 9 Months successfully.

My conclusion was that I can’t do anything once it’s disqualified and that I have to take the loss of the withholdings. So I set up a new Node with a Identity and I’m deleting the data now. I however made a copy of the log file, but I actually have no time to really investigate in what happend.  
I guess it went partially offline about a week ago when I had to change subnets in my network to make my mining farm run again. But I had the feeling that I did update port forwarding correctly. I also restarted router and Node (synology ds218+) and [uptimerobot.com](http://uptimerobot.com) did not report the node to be down (better said the port to be unresponsive)…

Not sure how this happend, but if I remember right I had a similar situation where the port stayed open and reachable from the internet but the node was (at least partially) unresponsive…

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 5, 2020, 9:01am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/15 "2020-03-05T09:01:07Z")

</div>

But it might have been a bad decision to set up a new identity before deleting the old data. Deleting the data again causes 100% disk workload and as a side effect pretty low upload success rate. ![storj successRate2](https://lc-storj.ams3.digitaloceanspaces.com/original/2X/d/da2c603d3434817b01e15ae7eef6b400f6db2245.png)

---

<div class="post-metadata">

**Author:** ![fasttrackhost](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/fasttrackhost/32/1130_2.png) [@fasttrackhost](https://forum.storj.io/u/fasttrackhost)\
**Post date:** [March 5, 2020, 10:46am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/16 "2020-03-05T10:46:30Z")

</div>

Please tell me how you get that info in the image, whats the command for it? Thank you

---

<div class="post-metadata">

**Author:** ![fasttrackhost](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/fasttrackhost/32/1130_2.png) [@fasttrackhost](https://forum.storj.io/u/fasttrackhost)\
**Post date:** [March 5, 2020, 10:50am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/17 "2020-03-05T10:50:08Z")

</div>

> [@jorma](#):
>
> Acording to the system monitor the workload is mainly caused by reading processes.

This could also be your Antivirus software, and if you are on Windows once a month it does a complete system scan for malware, etc.

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 5, 2020, 10:56am UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/18 "2020-03-05T10:56:08Z")

</div>

> [@fasttrackhost](#):
>
> This could also be your Antivirus software

there’s no antivirus on my synology nas. And the workload also stoped imediatly when I stoped the container.

> [@fasttrackhost](#):
>
> how you get that info in the image

it´s called successrate .sh … you have to download it. Take a look here. But not sure if there is something like it for windows…  
[https://forum.storj.io/t/can-t-find-the-successrate-sh-file/4889](https://forum.storj.io/t/can-t-find-the-successrate-sh-file/4889)

---

<div class="post-metadata">

**Author:** ![BrightSilence](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/brightsilence/32/6048_2.png) [@BrightSilence](https://forum.storj.io/u/BrightSilence)\
**Post date:** [March 5, 2020, 8:24pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/19 "2020-03-05T20:24:02Z")

</div>

> [@jorma](#):
>
> But not sure if there is something like it for windows

There is:

> **[GitHub - AlexeyALeonov/success\_rate: Success rate for storagenode (Storj V3)](https://github.com/AlexeyALeonov/success_rate)**
>
> Success rate for storagenode (Storj V3). Contribute to AlexeyALeonov/success\_rate development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [March 5, 2020, 8:39pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/20 "2020-03-05T20:39:22Z")

</div>

> [@jorma](#):
>
> I guess it went partially offline about a week ago when I had to change subnets in my network to make my mining farm run again.

No, the only possible way right now is failing too many audits, i.e. lost or inaccessible data.  
How is your `docker run` looks like? Have you replaced the `-v` options to the `--mount`?

---

<div class="post-metadata">

**Author:** ![jorma](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jorma/32/68_2.png) [@jorma](https://forum.storj.io/u/jorma)\
**Post date:** [March 5, 2020, 9:57pm UTC](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952/21 "2020-03-05T21:57:04Z")

</div>

> [@Alexey](#):
>
> Have you replaced the `-v` options to the `--mount` ?

yes, must have been somewhen last year. It’s with the --mount option in my setup script

> [@Alexey](#):
>
> the only possible way right now is failing too many audits

I am pretty sure there were no failed audits. Although I did not make a screenshot of successrate.sh, I did check it. And there was no sign of an error, despite of the really low upload successrate (as I mentioned in the initial post).  
I also checkt the last day with traffic in the copy of the log file and it seems to me as if there was not one failed audit - but plenty of successful audits (1 entry “download startet” followed by 1 entry “downloaded” (category “GET\_AUDIT”))  
But maybe there were failed audits but for some reason they did not get saved in the logs.

Node ID was: 1MgCzp5ByecEfxUVm3rXpVX5aGXkyoSjBnS7kiBfeduuuLzink

[Next page](https://forum.storj.io/t/100-disk-workload-without-any-traffic/4952.md?page=2)
