ERROR pieces:trash emptying trash failed

kevink · May 15, 2020, 4:13pm

Ah right, thanks. Meaning there are no files older than 7 days, as expected.

donald.m.motsinger · May 15, 2020, 4:29pm

Garbage collection runs once a day and deletes everything older than 7 days. After that the oldest files are 6.x days old. Until the next run some files will get older than 7 days, but not older than 8 days.

Sasha · May 17, 2020, 10:49am

Confirmed files are no older than 8 days after looking at specific files in the folders.
However it seems unusual to have 270~ + GB’s in trash.

Checked this morning and it’s down to 190~GB

Sasha · June 24, 2020, 10:21am

Trash folder has now grown to 850+ GB’s

donald.m.motsinger · June 24, 2020, 10:31am

So, what is your question? Everything is answered above.

Sasha · June 24, 2020, 10:33am

See littleskunks statement… garbage collection is plan b. If node is online (which it is 100%) the deletes will happen immediately. So there is something wrong with large amounts of trash being left over like this…

thebadcat · November 17, 2020, 5:48pm

Is there an answer to the question of how to or if it is possible to clean up the trash directory? I have 35GB of trash and would like to remove it if is not needed and I am not being paid for having it.

Alexey · November 17, 2020, 9:12pm

It should be removed automatically after 7 days. So - just wait for a week.

thebadcat · November 23, 2020, 11:24pm

Thx again @Alexey, once again you were correct. I was originally skeptical because the trash went up to almost double (~60GB). However, over the past few days it’s has gone down quite rapidly and as I type right now it is at ~26GB.

bobbll · April 21, 2022, 7:51am

this is how it looks to me. Is there an error or do I have to do something.

Alexey · April 21, 2022, 8:08pm

You should not touch the trash folder. You doesn’t have files older than 7 days. After 7 days files got removed automatically.

Regarding ERROR collector unable to delete piece

Error getting current used space - related either to slow disk or corrupted databases. To check databases you can use this guide: https://support.storj.io/hc/en-us/articles/360029309111-How-to-fix-a-database-disk-image-is-malformed-
Error from the current thread has this solution:

bobbll · April 22, 2022, 8:12pm

ich habe die datenbank laut guide überprüft und es ist alles ok jetzt bin ich bei 84 fehler.

.The node running runs ok

Alexey · April 23, 2022, 6:04am

I can see only 19 errors on your screenshot, where 10 - is connectivity problem (perhaps your external IP got changed and your DDNS hostname was not updated in time), all others perhaps “file not found”.

bobbll · April 23, 2022, 7:18am

i have checkt my i-net and the ddns-host it is all ok . this is the screenshot aktually

bobbll · April 23, 2022, 9:50am

2022-04-22T15:51:06.161Z ERROR piecestore download failed {“Piece ID”: “XY2XYFK7XL4JCOWIVFYFN4YRWTAUKDB63S2U66EORXZQNFBUGOKQ”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “GET”, “error”: “context deadline exceeded”, “errorVerbose”: “context deadline exceeded\n\tstorj.io/common/rpc/rpcstatus.Wrap:73\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func6:668\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:686\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:228\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:122\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:66\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:112\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52”}

2022-04-22T21:22:15.743Z ERROR piecestore upload failed {“Piece ID”: “GMNWG5YWYQU26BK4GDGWWLAW66HJPLFZ3G3ZZC6W5HPVLYOCE6LQ”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”, “error”: “unexpected EOF”, “errorVerbose”: “unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:347\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:122\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:66\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:112\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52”, “Size”: 65536}

2022-04-22T21:22:21.298Z ERROR piecestore download failed {“Piece ID”: “DKIJ6Q5VLLO3BSTI5AS7EZHZSM3XMCFZOAVTV2IOHBBNX64LM5BQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “write tcp 172.17.0.7:28967->135.148.54.71:44284: use of closed network connection”, “errorVerbose”: “write tcp 172.17.0.7:28967->135.148.54.71:44284: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:322\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:400\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:620\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22”}

2022-04-22T21:44:06.105Z ERROR piecestore download failed {“Piece ID”: “DKIJ6Q5VLLO3BSTI5AS7EZHZSM3XMCFZOAVTV2IOHBBNX64LM5BQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “write tcp 172.17.0.7:28967->135.148.54.52:34562: use of closed network connection”, “errorVerbose”: “write tcp 172.17.0.7:28967->135.148.54.52:34562: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:322\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:400\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:620\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22”}

2022-04-22T22:32:58.487Z ERROR piecestore download failed {“Piece ID”: “DX3IHFFLLD2HZZ4TBDAMRGSJ3AW7DQCYKGROE7L3JTKYXNE4O62A”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “write tcp 172.17.0.7:28967->135.148.54.52:51736: use of closed network connection”, “errorVerbose”: “write tcp 172.17.0.7:28967->135.148.54.52:51736: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:322\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:400\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:620\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22”}

2022-04-22T22:33:31.939Z ERROR piecestore download failed {“Piece ID”: “DX3IHFFLLD2HZZ4TBDAMRGSJ3AW7DQCYKGROE7L3JTKYXNE4O62A”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “write tcp 172.17.0.7:28967->135.148.54.71:39988: use of closed network connection”, “errorVerbose”: “write tcp 172.17.0.7:28967->135.148.54.71:39988: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:322\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:400\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:620\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22”}

2022-04-23T05:29:36.349Z ERROR piecestore download failed {“Piece ID”: “AO4MSRBRMMI3QC5CFMIFMCVNUWGQ23G2SIY7NFVXEDEHCGXLFSLA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “write tcp 172.17.0.7:28967->143.198.44.54:37738: use of closed network connection”, “errorVerbose”: “write tcp 172.17.0.7:28967->143.198.44.54:37738: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:322\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:400\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:620\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22”}

thats the Errors i find

Alexey · April 23, 2022, 2:03pm

These are “context canceled” errors - your node was too slow and the customer canceled upload (“PUT”) or download (“GET”).
You actually need to check only for errors from the topic. However, they should disappear after a while, when all such missing pieces will be deleted by storagenode from the trash.
You do not need to fix anything unless these errors will appear longer than a week.