# What happened with trash?

**URL:** <https://forum.storj.io/t/what-happened-with-trash/28425>\
**Category:** Node Operators\
**Tags:** trash\
**Created:** [November 2, 2024, 5:14pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425 "2024-11-02T17:14:22Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![DisaSoft](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/disasoft/32/1127_2.png) [@DisaSoft](https://forum.storj.io/u/DisaSoft)\
**Post date:** [November 2, 2024, 5:14pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/1 "2024-11-02T17:14:22Z")

</div>

It looks like almost all trash suddenly was recovered on all of my nodes regardless version.  
Used space in dashboard increased, trash decreased to symbolic values and this is not dashboard bug. Trash folder is really empty.  
Is anyone else seeing the same node behavior?

---

<div class="post-metadata">

**Author:** ![DusterCluster](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/dustercluster/32/1122_2.png) [@DusterCluster](https://forum.storj.io/u/DusterCluster)\
**Post date:** [November 2, 2024, 5:16pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/2 "2024-11-02T17:16:48Z")

</div>

Same thing for me today, I was surprised too.

---

<div class="post-metadata">

**Author:** ![Vadim](https://storj.bcdn.literatehosting.com/letter_avatar/vadim/32/6_449c533f586939b6e5f246632a46421f.png) [@Vadim](https://forum.storj.io/u/Vadim)\
**Post date:** [November 2, 2024, 5:27pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/3 "2024-11-02T17:27:03Z")

</div>

I dont know about trash, but node suddenly razed in space for at least 1-2 TB, but ingress is low.

---

<div class="post-metadata">

**Author:** ![alpharabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alpharabbit/32/1129_2.png) [@alpharabbit](https://forum.storj.io/u/alpharabbit)\
**Post date:** [November 2, 2024, 5:27pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/4 "2024-11-02T17:27:17Z")

</div>

It seems like the trash was moved to “used” again. Maybe they had to restore for some reason?

---

<div class="post-metadata">

**Author:** ![Ambifacient](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ambifacient/32/7815_2.png) [@Ambifacient](https://forum.storj.io/u/Ambifacient)\
**Post date:** [November 2, 2024, 5:44pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/5 "2024-11-02T17:44:49Z")

</div>

A rare trash restore:

```auto
2024-11-02T06:53:46-07:00	INFO	pieces:trash	restore trash started	{"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2024-11-02T06:56:32-07:00	INFO	pieces:trash	restore trash started	{"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2024-11-02T07:00:55-07:00	INFO	pieces:trash	restore trash started	{"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2024-11-02T07:13:11-07:00	INFO	pieces:trash	restore trash finished	{"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2024-11-02T07:32:34-07:00	INFO	pieces:trash	restore trash finished	{"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2024-11-02T10:13:08-07:00	INFO	pieces:trash	restore trash finished	{"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}

```

This node recovered about 400GB.

---

<div class="post-metadata">

**Author:** ![MarviBiene](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/marvibiene/32/1124_2.png) [@MarviBiene](https://forum.storj.io/u/MarviBiene)\
**Post date:** [November 2, 2024, 5:55pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/6 "2024-11-02T17:55:45Z")

</div>

Yes all my trash is gone and i now have 2 TB more space used.  
And thats why you dont touch trash yourself. This could have easily killed a node if wrong files were deleted via user themself.  
Good thing is, that storjstats dont show lost segment. Bad thing is, there has to be something wrong.

---

<div class="post-metadata">

**Author:** ![alpharabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alpharabbit/32/1129_2.png) [@alpharabbit](https://forum.storj.io/u/alpharabbit)\
**Post date:** [November 2, 2024, 6:17pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/7 "2024-11-02T18:17:10Z")

</div>

> [@MarviBiene](#):
>
> Yes all my trash is gone and i now have 2 TB more space used.

used but not paid… 😉

---

<div class="post-metadata">

**Author:** ![xgDkAbzkp9yi](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/xgdkabzkp9yi/32/7831_2.png) [@xgDkAbzkp9yi](https://forum.storj.io/u/xgDkAbzkp9yi)\
**Post date:** [November 2, 2024, 6:38pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/8 "2024-11-02T18:38:54Z")

</div>

> [@alpharabbit](#):
>
> used but not paid… 😉

If space is showing as “used”, it is being paid for at $1.5 a month, right?

> [@DisaSoft](#):
>
> It looks like almost all trash suddenly was recovered on all of my nodes regardless version.  
> Used space in dashboard increased, trash decreased to symbolic values and this is not dashboard bug. Trash folder is really empty.  
> Is anyone else seeing the same node behavior?

Same here. One node is showing 200KB, other 0 trash. Yay.

---

<div class="post-metadata">

**Author:** ![Knowledge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/knowledge/32/10681_2.png) [@Knowledge](https://forum.storj.io/u/Knowledge)\
**Post date:** [November 2, 2024, 6:48pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/9 "2024-11-02T18:48:41Z")

</div>

The engineers restored data from trash as they are conducting some research around test data that was recently deleted. It will likely move back to trash in some days as their tests complete. I don’t have a timeline for this however so enjoy the extra paid data while testing continues.

---

<div class="post-metadata">

**Author:** ![MarviBiene](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/marvibiene/32/1124_2.png) [@MarviBiene](https://forum.storj.io/u/MarviBiene)\
**Post date:** [November 2, 2024, 7:56pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/10 "2024-11-02T19:56:51Z")

</div>

So to recover trash, do you have to recover everything from all satellites? Or can you recover specific satellites or file too?

---

<div class="post-metadata">

**Author:** ![Knowledge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/knowledge/32/10681_2.png) [@Knowledge](https://forum.storj.io/u/Knowledge)\
**Post date:** [November 2, 2024, 8:56pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/11 "2024-11-02T20:56:57Z")

</div>

I believe they can do individual files or all files. From what I understand they did all in this case.

---

<div class="post-metadata">

**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\
**Post date:** [November 2, 2024, 10:55pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/12 "2024-11-02T22:55:26Z")

</div>

Restore is per satellite but not per file. It is all or nothing. Handing over a list of pieceIDs to restore doesn’t really work. At the moment we only know that something is off so we better restore everything to not risk anything over the weekend and continue on Monday with further investigations.

---

<div class="post-metadata">

**Author:** ![andrew2.hart](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/andrew2.hart/32/6415_2.png) [@andrew2.hart](https://forum.storj.io/u/andrew2.hart)\
**Post date:** [November 3, 2024, 6:51am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/13 "2024-11-03T06:51:29Z")

</div>

But why did some nodes have trash remaining?  
du -hs /node?/node?/storage/trash/  
30G /node1/node1/storage/trash/  
43M /node2/node2/storage/trash/  
30G /node3/node3/storage/trash/  
19G /node4/node4/storage/trash/  
29G /node5/node5/storage/trash/  
29G /node6/node6/storage/trash/

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [November 3, 2024, 6:57am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/14 "2024-11-03T06:57:31Z")

</div>

What folders there?

```auto
tree -L 2 /node?/node?/storage/trash/

```

---

<div class="post-metadata">

**Author:** ![andrew2.hart](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/andrew2.hart/32/6415_2.png) [@andrew2.hart](https://forum.storj.io/u/andrew2.hart)\
**Post date:** [November 3, 2024, 7:03am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/15 "2024-11-03T07:03:42Z")

</div>

> [@Alexey](#):
>
> `tree -L 2 /node?/node?/storage/trash/`

```auto
/node1/node1/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-27
│ ├── 2024-10-30
│ └── 2024-10-31
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-28
│ └── 2024-10-31
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-27
    ├── 2024-10-28
    ├── 2024-10-31
    └── 2024-11-03
/node2/node2/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-28
│ ├── 2024-10-29
│ └── 2024-10-31
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-28
│ └── 2024-11-01
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-26
    ├── 2024-10-29
    ├── 2024-10-30
    └── 2024-11-02
/node3/node3/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-27
│ ├── 2024-10-29
│ └── 2024-11-01
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-26
│ ├── 2024-10-28
│ └── 2024-10-31
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-27
    ├── 2024-10-29
    ├── 2024-10-31
    └── 2024-11-03
/node4/node4/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-27
│ ├── 2024-10-30
│ └── 2024-10-31
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-28
│ └── 2024-11-01
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-26
    ├── 2024-10-29
    ├── 2024-10-30
    └── 2024-11-03
/node5/node5/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-27
│ ├── 2024-10-29
│ └── 2024-10-31
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-28
│ └── 2024-10-31
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-27
    ├── 2024-10-28
    ├── 2024-10-30
    └── 2024-11-03
/node6/node6/storage/trash/
├── pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
├── qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
│ ├── 2024-10-28
│ ├── 2024-10-30
│ └── 2024-10-31
├── ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
│ ├── 2024-10-26
│ ├── 2024-10-28
│ └── 2024-10-31
└── v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa
    ├── 2024-10-26
    ├── 2024-10-29
    ├── 2024-10-31
    └── 2024-11-03

```

---

<div class="post-metadata">

**Author:** ![andrew2.hart](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/andrew2.hart/32/6415_2.png) [@andrew2.hart](https://forum.storj.io/u/andrew2.hart)\
**Post date:** [November 3, 2024, 7:11am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/16 "2024-11-03T07:11:43Z")

</div>

Mystery solved

```auto
du -hs /node?/node?/storage/trash/*/* | grep G
30G /node1/node1/storage/trash/v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa/2024-11-03
30G /node3/node3/storage/trash/v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa/2024-11-03
19G /node4/node4/storage/trash/v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa/2024-11-03
29G /node5/node5/storage/trash/v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa/2024-11-03
29G /node6/node6/storage/trash/v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa/2024-11-03

```

---

<div class="post-metadata">

**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/6_449c533f586939b6e5f246632a46421f.png) [@Roxor](https://forum.storj.io/u/Roxor)\
**Post date:** [November 3, 2024, 11:08am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/17 "2024-11-03T11:08:39Z")

</div>

So some data was untrashed… but will soon be retrashed?

Got it! 👍

---

<div class="post-metadata">

**Author:** ![xgDkAbzkp9yi](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/xgdkabzkp9yi/32/7831_2.png) [@xgDkAbzkp9yi](https://forum.storj.io/u/xgDkAbzkp9yi)\
**Post date:** [November 4, 2024, 3:20pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/18 "2024-11-04T15:20:33Z")

</div>

It has already been retrashed. My nodes again showing 270GB and 47GB trash, previously they were showing 0.

---

<div class="post-metadata">

**Author:** ![EasyRhino](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/easyrhino/32/1133_2.png) [@EasyRhino](https://forum.storj.io/u/EasyRhino)\
**Post date:** [November 4, 2024, 5:40pm UTC](https://forum.storj.io/t/what-happened-with-trash/28425/19 "2024-11-04T17:40:55Z")

</div>

wow catching up on this thread has been an emotional roller coaster.

---

<div class="post-metadata">

**Author:** ![andrew2.hart](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/andrew2.hart/32/6415_2.png) [@andrew2.hart](https://forum.storj.io/u/andrew2.hart)\
**Post date:** [November 5, 2024, 5:11am UTC](https://forum.storj.io/t/what-happened-with-trash/28425/20 "2024-11-05T05:11:00Z")

</div>

Yes, so if the untrash everything gets used, the very next bloom starts trashing again.  
So the “enjoy your extra data that is paid for” straight away become “enjoy your 7 days extra trash” averaged to 3.5 days of course.  
And enjoy thrashing your disk to untrash and then trash again.

It’s the trash-untrash-retrash-thrash-nocash scandal!!!

[Next page](https://forum.storj.io/t/what-happened-with-trash/28425.md?page=2)
