Trash stored more than 7 days (168h)

Node Operators
21

In total, all my nodes now have a lot of garbage. I brought only one. No matter how much I lose, the fact itself is important.

1 Like
22

Trash is kept for 7 days.

1 Like
23

Yes it is. What’s your point?

1 Like
24

You just made a post and calculation that assumes trash is cleared once a day.

1 Like
25

It is. Trash that is older than 7 days is cleared once a day.

1 Like
26

But the trash stays on the hard drive taking up space for 7 days? After 7 days, then it is cleared once a day. So maybe i’m not understanding why 84GB/8 = 10.5GB is coming from.

Shouldn’t it be (84GB * 8 days)/30 days = 22.4GB average hard drive used for that month? Maybe I’m not understanding it right.

1 Like
27

Here is the other node I have at home.

image

image
image261×688 16.5 KB

1 Like
28

I understand that trash should not be compensated since Storj themselves are not being compensated by the customer once the pieces are deleted. This is totally fine.

What is not fine, is that trash is taking up space that it is limiting other functions of the node, (Ingress). What does one do when you are out of space on your own daily use computer? We empty the trash bin. I think the node should do the same. Nodes that have enough free space can continue to keep the trash pieces for 7 days, because it is not hindering other functions of the node. I believe the Nodes that are full should have a mechanism in place to gradually start deleting the trash to free up space before having to wait the full 7 days.

2 Likes
29

Ahh I see what you mean. This topic is about it being stored for MORE than 7 days. So I was only counting the extra time. I was assuming the trash amount would stay fairly constant so each day would have new trash. That’s why I didn’t divide by 30.

1 Like
30

That is fine. Like I said, the only reason I care about trash is because it is hindering other functions of the node. Storj can keep trash as long as they like since the space is already allocated for Storj but please free up the trash when my node is full. I want ingress…

1 Like
31

I do not paid either. However, my trash is not so full
image

(Get-ChildItem Y:\storagenode2\storage\trash\ -File -Recurse | Where-Object{$_.LastWriteTime -lt (Get-Date).AddDays(-7)} | ft LastWriteTime, Name).Count
117

(Get-ChildItem Y:\storagenode2\storage\trash\ -File -Recurse | Where-Object{$_.LastWriteTime -lt (Get-Date).AddDays(-8)} | ft LastWriteTime, Name).Count
0

So nothing older than 7 days.
But I’ll ask devs, what’s date they are using.
There are:

  • CreationTime
  • LastWriteTime
  • LastAccessTime
    Also, there are an UTC versions of the same dates.

Obviously it’s not a CreationTime, because the piece can be created a lot of time ago before landing in trash. So, I selected the LastWriteTime, because it’s when the file has been written here.

Could you calculate by LastWriteTime too?

Those was a morning results. The evening ones:

(Get-ChildItem Y:\storagenode2\storage\trash\ -File -Recurse | Where-Object{$_.LastWriteTime -lt (Get-Date).AddDays(-7)} | ft LastWriteTime, Name).Count
61
1 Like
32

you sure it’s not some sort issue with your node… i got no idea how to check that tho…

but it does seem excessive… my node is 11.47tb 4 months old and today my trash is 11gb … and it seems to drop from time to time and then go back up again…

maybe try to run the commands alexey suggested… and share the results…

i for one would like to know how you can even have that amount of trash on the node… it seems like a “common” issue… and thus understanding the mechanics of why it happens is key to avoiding.

1 Like
33

Alexey’s codes is for Windows and the purpose of that code is just to check if the pieces are greater than 7 days. It does not look into the reason why the pieces are not deleted right away instead of being pushed to the garbage collection which triggers a 7 day hold of the trash.

1 Like
34

well if your node doesn’t have the spare io it might not delete stuff… it might be a pretty brilliant way for a programmer to protect your node from failing but just letting trash grow instead… thus figuring that the SNO’s will have to deal with it at some point…

verifying if your trash is older or younger than 7 days should be trivial… i take it that one can just look at the created date on the file… or last modified or whatever it’s called in linux

i have been pondering if it is lack of io that is to blame for the massive trash on some nodes.

and hey… if it was what it took to save your node from failing… :smiley:

1 Like
35

My trash is being deleted after 7 days, there is no problem there.

1 Like
36

well if it’s in the trash folder… then … i mean… xD i know a way to get rid of it…
not sure if i would be brave enough to do that… but kinda depends on how long i had tried to solve the problem…

take out the trash… you could always just change the folder name and wait a few days… see if it acts up and then delete it…

NOT THAT I WOULD RECOMMEND DOING HAKS to your node… but … welll… you know
and it’s called trash… how bad could it really be just to delete it…

1 Like
37

I would not recommend others to touch anything in the data folder. It is also not the responsibility of the SNO to do that. All i ask is that SNOs with slower IO should not be penalized for garbage collection that keeps trash for 7 days while the node is full and hinder us from other functions of the node (further ingress).

3 Likes
38

yeah it does kinda seems unfair… maybe when my node is older i will have seen bigger trash sizes… but thus far my peak … afaik is 11gb ofc only really kept an eye on it since the new update… also keep in mind… storj wouldn’t have had a great idea about how bad the problem was, before they added a trash count to the dashboard…

it might be a sign that they are considering or working on fixing it…
ofc who knows maybe trash is used for something… i really don’t know… i’m tardigrade code base illiterate

1 Like
39

Well, no, it could be the created time. Recall the longstanding issue that files are copied into trash and then the original deleted instead of being renamed (which should still be fixed as it wastes disk IOPs for no purpose). Unless the node software resets the created time, the created time will be when the file began being moved into trash.

3 Likes
40

Hi. I don’t think it has been confirmed that files being garbage collected are copied to their subfolder in trash/ and then removed from their blobs/ subdirectory. If they are, infact, renamed there may be other interesting explanations as to why high I/O has been seen by several people* during the garbage collection period, and I wouldn’t want to overlook considering those.

1 Like