ERROR collector unable to delete piece ( 37 pieces )

I get the following error for 37 pieces, what should I do?

2022-06-27T12:19:46.340Z ERROR collector unable to delete piece {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Piece ID”: “KDRT4Y43NLVQM3L4OST5MUMK6ASHHR5TJTUGVBXAREO3IFHM6TUA”, “error”: “pieces error: filestore error: file does not exist”, “errorVerbose”: “pieces error: filestore error: file does not exist\n\tstorj.io/storj/storage/filestore.(*blobStore).Stat:103\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).pieceSizes:239\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).Delete:220\n\tstorj.io/storj/storagenode/pieces.(*Store).Delete:299\n\tstorj.io/storj/storagenode/collector.(*Service).Collect:97\n\tstorj.io/storj/storagenode/collector.(*Service).Run.func1:57\n\tstorj.io/common/sync2.(*Cycle).Run:152\n\tstorj.io/storj/storagenode/collector.(*Service).Run:53\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:40\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}

This is the only error kind I get now.

My previous issue was 100% solved and thanks to the fan I lowered the temperature by 11°C (from 47 to 36)

in my case it’s all from us1.storj.io:7777

Yes, but you can run the same checks.

1 Like

Hypotetically, is there any way to repair these missing pieces?

No, as an SNO you cannot repair pieces. This would require the satellite to trust single nodes, which is not the case.
But there are chances, that there is a bug regarding that error, so your node might be fine.

I am having the exact same issue @mer44 . Let me know if you find a solution. I will check out the link from jammerdan in the meantime.

Basically, what happen is that it is trying to delete a file that no longer exist on your system.
Easiest way to fix it is to create an empty file in that location.
On your example, the relevant part is the Satellite ID and the Piece ID.
I use this link to convert the Satellite ID to Blolbs folder Satellite info (Address, ID, Blobs folder, Hex)

12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S = ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
KDRT4Y43NLVQM3L4OST5MUMK6ASHHR5TJTUGVBXAREO3IFHM6TUA = kd/rt4y43nlvqm3l4ost5mumk6ashhr5tjtugvbxareo3ifhm6tua.sj1

So you need to create a empty file in your blobs folder ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/kd/rt4y43nlvqm3l4ost5mumk6ashhr5tjtugvbxareo3ifhm6tua.sj1
The next time it does its delete check, it will find the empty file and delete it.
I had to do this on 3-4 of my nodes, usually for this same Satellite ID.

3 Likes

@jammerdan implayed it would not be okay.

Maybe I got you wrong.
What I was referring to and what I thought what was your question, is if you as SNO can reinstate a missing piece meaning that customers would be able to download these pieces again. This is not the case. A missing piece is lost. Repair can only be triggered by the Storj controlled repair workers.
However what @lyoth suggested is a way to get rid of the warning only, by faking the existence of the piece so it can be successfully deleted. I guess this would be totally fine.
So maybe there was a misunderstanding from my side.

3 Likes

That satelite that was giving me an error just pushed my suspended score to 12.5%, probably due to those missing pieces, does it mean that they will now get repaired? What is the procedure?

I don’t think there is a general answer to that other than if repair is required then it will be performed. But under no circumstances it will be performed on your node.

Repairing means that the repair workers reconstruct missing pieces by downloading the complete file and then upload them to other nodes, but not back to your node. So I repeat myself when I am saying that a piece that your node has lost is lost. It will not return somehow magically back to your node. There is nothing you can do to get this piece back (other than a backup that you might have). (However once the satellite or Audit worker knows the file has been lost even a backup probably would not help anymore).

1 Like

Ok, thanks for taking the time to explain it!