How can I find out why this is happening ?
This happened on two nodes, both are online.
Check your log for download failed
and GET_AUDIT
NODE FIRST ERROR
2020-05-05T23:45:41.441+0200 ERROR piecestore delete failed {“Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Piece ID”: “KXIQAGEDD5NG73UUDGGB7DY7SR4WECXRQNUJKKUZPXYWEWO5TK7Q”, “error”: “pieces error: filestore error: unable to stat “L:\\STORJ FILES\\blobs\\pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\\kx\\iqagedd5ng73uudggb7dy7sr4wecxrqnujkkuzpxywewo5tk7q.sj1”: CreateFile L:\STORJ FILES\blobs\pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\kx\iqagedd5ng73uudggb7dy7sr4wecxrqnujkkuzpxywewo5tk7q.sj1: The semaphore timeout period has expired.”, “errorVerbose”: “pieces error: filestore error: unable to stat “L:\\STORJ FILES\\blobs\\pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\\kx\\iqagedd5ng73uudggb7dy7sr4wecxrqnujkkuzpxywewo5tk7q.sj1”: CreateFile L:\STORJ FILES\blobs\pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\kx\iqagedd5ng73uudggb7dy7sr4wecxrqnujkkuzpxywewo5tk7q.sj1: The semaphore timeout period has expired.\n\tstorj.io/storj/storage/filestore.(*Dir).Stat:266\n\tstorj.io/storj/storage/filestore.(*blobStore).Stat:82\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).pieceSizes:238\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).Delete:220\n\tstorj.io/storj/storagenode/pieces.(*Store).Delete:272\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).DeletePieces:190\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func4:1012\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:66\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
NODE LAST ERROR
2020-05-06T18:17:21.425+0200 ERROR piecestore download failed {“Piece ID”: “MMC2XTBFQFR3WO6GYENM35WAZDAWKHDBIBU2IVYYY6UASAZZ2BZQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “error”: “usedserialsdb error: disk I/O error: A device which does not exist was specified.”, “errorVerbose”: “usedserialsdb error: disk I/O error: A device which does not exist was specified.\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).doDownload:523\n\tstorj.io/storj/storagenode/piecestore.(*drpcEndpoint).Download:471\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:995\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:66\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
Looks like my disk is down, on it.
How is the drive connected?
USB, I just updated all drivers incase. I can see that it is now up again, but how do I get my node qualified again? does it happen automatically ?
Have you noticed it powering off?
You need to pass a few audits to get out of suspened
Okei. Then it will be good soon.
I think it may be that the driver got outdatet, not sure the status on the disk was OK all the time. But I see in my log after I updated all was good and it got from ERROR to INFO.
One other thing is this windows node?
Yes, this is windows node.
You should check to make sure USB isnt on a power saving setting, if it is the drive may go idle and turn off.
Will do that, but I dont think its on power saving as I had very long runtime on it. I will give a update when my node is not suspended anymore
Windows updates sometimes screw up some settings for power savings. Well in general windows updates are bad news for running a node that is meant to run 24/7.
I have windows update turned off, without this my uptime on this node have been 99.9%. But yeah, should go over to host in linux at a point in the future.