# Huge amount of trash data? **URL:** \ **Category:** troubleshooting\ **Tags:** trash\ **Created:** [May 12, 2024, 9:28am UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168 "2024-05-12T09:28:13Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\ **Post date:** [May 17, 2024, 2:35pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/22 "2024-05-17T14:35:10Z")
> [@Roxor](#): > > The test-data info is more recent… so I _think_ we should expect overall use to go up. That assumption is correct. As described in the newer of the 2 announcements we are preparing the network for onboarding a new usecase. Unfortunatly cleaning up data that was uploaded without a TTL takes 2-3 weeks because of the way garbage collection works. So first your usage will go down a lot by the current and the following garbage collection run. I am sorry there is no way to make this a smooth transition. We are freeing up the space for a reason. So in the following weeks/months that space might or might not get quickly replaced with customer data. But lets be honest here. Currently all you have from our side is a promise and no way to verify it. You see the decrease right away and all thats left is hope that our plan works out at the end. ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [May 17, 2024, 2:44pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/23 "2024-05-17T14:44:18Z")
> [@Roxor](#): > > but [performance-related test data will be unpaid](https://forum.storj.io/t/updates-on-test-data/26034/149)? these two different data as far as I know. ---
**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\ **Post date:** [May 17, 2024, 2:45pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/24 "2024-05-17T14:45:58Z")
> [@JoshieGarza](#): > > I have many many nodes looking like this overnight. 50% test data? Possible but it sounds a bit extreme. Is that trash mostly on the SLC satellite? There was a lot of old test data we deleted from SLC. I was told it finished yesterday evening/today morning. That would be a very quick bloom filter creation but it would explain your trash situation. I have to double check. ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 17, 2024, 2:53pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/25 "2024-05-17T14:53:43Z")
> [@littleskunk](#): > > But lets be honest here. Currently all you have from our side is a promise and no way to verify it. I am 100% perfectly OK with that. I don’t need any promises, or predictions, or roadmaps. As long as I receive some coins near the start of every month… Storj can do whatever they want! 😜 Nodes pretty much run themselves anyways: I only need to take a look if I receive a “Your Node has Gone Offline” email… which is rare. ---
**Author:** ![JoshieGarza](https://storj.bcdn.literatehosting.com/letter_avatar/joshiegarza/32/5_5575768a8748004e209b776fc1b2916d.png) [@JoshieGarza](https://forum.storj.io/u/JoshieGarza)\ **Post date:** [May 17, 2024, 3:35pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/26 "2024-05-17T15:35:35Z")
avg disk space used this month for saltlake ![imagen](https://storj-s3.bcdn.literatehosting.net/original/3X/f/c/fc82ed54aff934362a884787662b6da9626941d2.png) another bigger nodes also have 3-4TB of data deleted last night but it means a lower % of total data. In small nodes the wheel chart is more surprising ---
**Author:** ![JoshieGarza](https://storj.bcdn.literatehosting.com/letter_avatar/joshiegarza/32/5_5575768a8748004e209b776fc1b2916d.png) [@JoshieGarza](https://forum.storj.io/u/JoshieGarza)\ **Post date:** [May 18, 2024, 12:14pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/27 "2024-05-18T12:14:54Z")
Things are getting interesting 😃 At least this month we will see how the payout looks like without dummy data artificially filling the storage nodes. ![imagen](https://storj-s3.bcdn.literatehosting.net/original/3X/4/9/493ee0c7ca7a13679a321de6c32c84b982895c09.png) ---
**Author:** ![ACarneiro](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/acarneiro/32/1129_2.png) [@ACarneiro](https://forum.storj.io/u/ACarneiro)\ **Post date:** [May 18, 2024, 12:18pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/28 "2024-05-18T12:18:04Z")
We know exactly what the payouts will look like… 💸😄 ---
**Author:** ![JoshieGarza](https://storj.bcdn.literatehosting.com/letter_avatar/joshiegarza/32/5_5575768a8748004e209b776fc1b2916d.png) [@JoshieGarza](https://forum.storj.io/u/JoshieGarza)\ **Post date:** [May 18, 2024, 12:33pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/29 "2024-05-18T12:33:24Z")
Sadly yes, we know 😃 The number of suspended nodes has increased in the last week… not sure if it has something to do with the new storagenode release. ![imagen](https://storj-s3.bcdn.literatehosting.net/original/3X/a/c/ac423a8d29996caf57dab9370b79fdd668d6fce4.png) ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [May 18, 2024, 12:39pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/30 "2024-05-18T12:39:15Z")
I guess - nothing. The usual thing - some “potatoes” didn’t make it. ---
**Author:** ![zip](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/zip/32/7813_2.png) [@zip](https://forum.storj.io/u/zip)\ **Post date:** [May 18, 2024, 12:56pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/31 "2024-05-18T12:56:15Z")
Spun up at August 2022 ![image](https://storj-s3.bcdn.literatehosting.net/original/3X/3/0/3010e5a7c68e3525040bb084084215af346ae0c6.png) ![image](https://storj-s3.bcdn.literatehosting.net/original/3X/a/a/aac922a9cf9bf76f261b1754f63ee5c0a421f42f.png) ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 18, 2024, 1:02pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/32 "2024-05-18T13:02:31Z")
> [@zip](#): > > Spun up at August 2022 Dude! You’ve got more trash than most SNOs have data! 😜 ---
**Author:** ![ACarneiro](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/acarneiro/32/1129_2.png) [@ACarneiro](https://forum.storj.io/u/ACarneiro)\ **Post date:** [May 18, 2024, 1:41pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/33 "2024-05-18T13:41:53Z")
Good Lord, that is brutal! ---
**Author:** ![JoshieGarza](https://storj.bcdn.literatehosting.com/letter_avatar/joshiegarza/32/5_5575768a8748004e209b776fc1b2916d.png) [@JoshieGarza](https://forum.storj.io/u/JoshieGarza)\ **Post date:** [May 18, 2024, 2:00pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/34 "2024-05-18T14:00:18Z")
A 20% increase in suspended nodes is normal? a lot of potatones decided to explode at same time. I am turning down watchtower in 50% of my fleet to avoid a storagenode upgrade. We will see. New storagenode release comes with the `fsync` option disabled by default? ---
**Author:** ![pasatmalo](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pasatmalo/32/1127_2.png) [@pasatmalo](https://forum.storj.io/u/pasatmalo)\ **Post date:** [May 18, 2024, 2:20pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/35 "2024-05-18T14:20:15Z")
Clearly a lot of synthetic, test & free tier data is being purged, which sadly will have a big effect on old nodes the most, but this was something that had to be done eventually. All we have to hope for is that new data will soon follow. If you check out the town hall of march, they show that there was about 12PB of actual consumer data, therefore a lot of the data currently in the network will eventually be purged. Given the recent focus on performance, the trend of customer data increase that was shared in the town hall, and the recent testing taking place, I do think more data will come. How much and when? I got no clule. > [@JoshieGarza](#): > > A 20% increase in suspended nodes is normal? a lot of potatones decided to explode at same time. > I am turning down watchtower in 50% of my fleet to avoid a storagenode upgrade. We will see. Never hurts to be careful about updates, but I dont believe that this recent increase of suspended nodes is related to the update. Usually a node takes about 2 weeks to be suspended when shutdown (Im recalling from memory, might be off, but takes a while). The update started rolling out 5 days ago. The sudden rise of suspended nodes was 4 days ago. Therefore, it looks to me like the suspended nodes was not a cause of the 1.104.5 update, but rather something else. > [@JoshieGarza](#): > > New storagenode release comes with the `fsync` option disabled by default? Sync writes (fsync) should be disabled by default in the new update. Can be enabled manually if you are worried about it or have an unstable system. ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [May 18, 2024, 3:13pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/36 "2024-05-18T15:13:35Z")
> [@JoshieGarza](#): > > I am turning down watchtower in 50% of my fleet to avoid a storagenode upgrade. it updates only the base image, which has a Downloader only. All docker nodes have storagenode and storagenode-updater. The last one updates nodes to the last version accordingly their NodeID. There is no way to disable it. In short - disabling the watchtower will not stop the rollout. ---
**Author:** ![Th3Van](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/th3van/32/66_2.png) [@Th3Van](https://forum.storj.io/u/Th3Van)\ **Post date:** [May 18, 2024, 3:54pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/37 "2024-05-18T15:54:28Z")
Looks like changing the retain log level from `debug` to `enable` did the trick for my nodes. ```auto Date Available hdd space Used hdd space Free hdd space --------------------------------------------------------------------------------------------- 2024-05-14 - 1.986.332.595.376.128 1.508.894.220.869.632 (75.9638 %) 477.438.374.506.496 2024-05-18 - 1.986.332.595.376.128 1.283.717.187.338.240 (64.6275 %) 702.615.408.037.888 ``` Sum of 117 BF files, that are current being processed by the nodes : 783004 KB Th3Van.dk ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 18, 2024, 5:38pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/38 "2024-05-18T17:38:13Z")
> [@Th3Van](#): > > Looks like changing the retain log level from `debug` to `enable` did the trick for my nodes. Is that saying you’re down about 225TB in the last 4 days?!? Holy crap! 😵 ---
**Author:** ![Solu](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/solu/32/62_2.png) [@Solu](https://forum.storj.io/u/Solu)\ **Post date:** [May 18, 2024, 6:09pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/39 "2024-05-18T18:09:10Z")
i know it hurts 🥲 ![Bildschirmfoto 2024-05-18 um 20.07.03](https://storj-s3.bcdn.literatehosting.net/original/3X/a/2/a20a4f5a0d88ebcfce1978c6ed10e7d0e3779975.png) ---
**Author:** ![digitalfrank](https://storj.bcdn.literatehosting.com/letter_avatar/digitalfrank/32/5_5575768a8748004e209b776fc1b2916d.png) [@digitalfrank](https://forum.storj.io/u/digitalfrank)\ **Post date:** [May 18, 2024, 7:03pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/40 "2024-05-18T19:03:50Z")
![Cattura](https://storj-s3.bcdn.literatehosting.net/original/3X/2/6/269308fe58614f40601de1bfee9ecc0e9c875d5c.jpeg) At the first day of this mounth the total disk used was 95TB. 🤒 ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 18, 2024, 7:10pm UTC](https://forum.storj.io/t/huge-amount-of-trash-data/26168/41 "2024-05-18T19:10:56Z")
> [@digitalfrank](#): > > At the first day of this mounth the total disk used was 95TB If nodes auto-upgrade every couple weeks… did your 95TB node even have time to complete a used-space-filewalker run? You must have hundreds of millions of files! [Previous page](https://forum.storj.io/t/huge-amount-of-trash-data/26168.md?page=1) [Next page](https://forum.storj.io/t/huge-amount-of-trash-data/26168.md?page=3)