Node - Unsent Orders

Node Operators troubleshooting
1

Hi there,

Was wondering if I could get some help on whether or not I have an issue on my node.
It’s still in the vetting process from what I understand it but I see unsent orders daily propagating and wonder if this has any negative impact.

Regards

2

Hi @kiwilad-nz , welcome to the forum !
I’ve forwarded your question to the team for some answers.

3

If you have had unsent orders, there is problem with your node for sure.
See https://status.storj.io/

If orders would not sent, your node will not be paid for unsent orders. So make sure that there is no errors related to orders in your logs and you do not block outgoing connections. Especially to satellites.
All orders older than 72 hours will be rejected.

If you have errors related to orders, please, post them here between two new lines with three backticks (```), like this:

```
lines from logs
```
4

Hi Alexey,

Here are the logs that fall under “error” from Docker.

I can go into the unsent folder too and often see about 5 or 6 at a time too.

2021-10-06T06:01:03.431Z	ERROR	piecestore	upload failed	{"Piece ID": "VUDZMGDBCZOH36LUCD45BHV453MA74FANU5PQFWLDR47LJTBLP5Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-06T10:17:04.143Z	ERROR	piecestore	upload failed	{"Piece ID": "BSAIJROVWJLZ7OHYJVDMWHPDALNDFQ53IKIQWQRYEP5B3FYJQXIQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-07T05:28:10.760Z	ERROR	piecestore	upload failed	{"Piece ID": "RXWXPYUUKLFLBHSEYDF3ZJTZJIZ76ZHUJX54Z7BNYI55CARUBDOA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-07T05:48:12.013Z	ERROR	piecestore	upload failed	{"Piece ID": "5J4QL5W25MEOKQ25PGFINPS3CVH6GYJBVAGLEGXVYCOS5K37XTHA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-07T05:48:12.015Z	ERROR	piecestore	upload failed	{"Piece ID": "FP5DRD4M2FRXDFSNRUNKGKSBXU4JDXVJE2UTGBY5JZNE7R2HR64Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-07T10:00:36.407Z	ERROR	piecestore	upload failed	{"Piece ID": "U6BOM7OTPK3OQPKVFLU4F4NMR2CAN5ZCWNEZ4AF55AEJIIBXS2XQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-07T17:44:40.474Z	ERROR	piecestore	upload failed	{"Piece ID": "DY5BRVRCWY53E6PQJCJJ7CWWIUU52NZ4HXR5XYOU2NDLEWO25SQA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "use of closed network connection", "errorVerbose": "use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawFlushLocked:347\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:396\n\tstorj.io/common/pb.(*drpcPiecestore_UploadStream).SendAndClose:288\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload.func6:424\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22", "Size": 8448},
2021-10-08T07:22:14.680Z	ERROR	piecestore	upload failed	{"Piece ID": "OE46AEQFSKQ4RIJZ6H6636ESXMXMMLBP3ZZPFUCOFX4HGUNQBBUQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-08T08:14:22.869Z	ERROR	piecestore	download failed	{"Piece ID": "R73ZQH2BSNDV6VYPRW2TXJYYUUOLJED74JDPHCO7GTXTPV54XF2Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "write tcp 192.168.1.3:28967->45.15.192.34:46414: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->45.15.192.34:46414: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-08T08:59:13.059Z	ERROR	piecestore	download failed	{"Piece ID": "RVWIJBYXKU6KJ2VRG4RUKFTRHE2EVIAOVJU6JVV33KQMXO5LZZKQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "error": "write tcp 192.168.1.3:28967->144.76.136.153:56770: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->144.76.136.153:56770: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-08T11:52:35.905Z	ERROR	piecestore	download failed	{"Piece ID": "JKZY7TR6TBYQ6Z3XQZ5VRIEW5JZFX7IPAHP2GERSCNNOBDUIQ2TQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "error": "write tcp 192.168.1.3:28967->144.76.136.153:60460: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->144.76.136.153:60460: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-08T13:23:15.060Z	ERROR	piecestore	download failed	{"Piece ID": "GI7M3CERBRKR64Z72O52E3R2WBVXTFGN42NJXC4UE7WJE7Z2DIIQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "write tcp 192.168.1.3:28967->145.40.69.109:54232: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->145.40.69.109:54232: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-08T20:05:18.428Z	ERROR	piecestore	download failed	{"Piece ID": "UNWSR2VDYS6MCSMNPHEBKFTH2GKKXM2COZIA2BGDW6DMP4FR2GMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "error": "write tcp 192.168.1.3:28967->144.76.136.153:41738: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->144.76.136.153:41738: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-09T04:29:42.940Z	ERROR	piecestore	upload failed	{"Piece ID": "VKWJ5D63TF2UPPRB5NAHAHX4KVSMOLFLRLLM2V2MT7QJEIVZDBIA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "context deadline exceeded", "errorVerbose": "context deadline exceeded\n\tstorj.io/common/rpc/rpcstatus.Wrap:73\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:344\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-09T06:28:08.328Z	ERROR	piecestore	download failed	{"Piece ID": "GJLCXYK52DSI7BU7WEGCWWFRWMUVX6SCMSWIVG7KTBAT5JOIFL5Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "write tcp 192.168.1.3:28967->51.81.66.114:41056: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->51.81.66.114:41056: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-09T07:04:12.795Z	ERROR	piecestore	upload failed	{"Piece ID": "HCSRY4RDLN763EV2TN2RWIMZWSP6SFP5TJ2BFIGUL2BSS3J6HNBQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "context deadline exceeded", "errorVerbose": "context deadline exceeded\n\tstorj.io/common/rpc/rpcstatus.Wrap:73\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:344\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-09T07:06:16.475Z	ERROR	piecestore	upload failed	{"Piece ID": "FCKEW2IOSBAU5G6WQ4HRJKI7LU3DMSCOOIGHIF53RIH3SXFAVLNA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "context deadline exceeded", "errorVerbose": "context deadline exceeded\n\tstorj.io/common/rpc/rpcstatus.Wrap:73\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:344\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-09T08:40:24.114Z	ERROR	piecestore	download failed	{"Piece ID": "N6LUV7VPHW7IJO2T6RTQ7VYNJDBQ52LC3LVC4XR6UEUH62UI4A3Q", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET", "error": "write quic 192.168.1.3:28967->165.227.200.143:33575: connection reset by peer", "errorVerbose": "write quic 192.168.1.3:28967->165.227.200.143:33575: connection reset by peer\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-09T11:44:12.031Z	ERROR	piecestore	download failed	{"Piece ID": "PD234V4353NSDJTMZSY52ZLRCDL2J5CPWRJKTBAJI5MPML6BKLXA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET", "error": "write quic 192.168.1.3:28967->165.227.200.143:55646: connection reset by peer", "errorVerbose": "write quic 192.168.1.3:28967->165.227.200.143:55646: connection reset by peer\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-09T18:46:25.843Z	ERROR	piecestore	upload failed	{"Piece ID": "BA57SUFDBD55UUIDWNS7YRH6WUICMFVLSC3DNP7INDUX4Y6VA3NQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 65536},
2021-10-09T19:20:25.689Z	ERROR	piecestore	upload failed	{"Piece ID": "YJ4CK4H247FITQD367S4DFCGSK7Q2GTQ2F7ACQWFU2AJHG6RAOZA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "PUT", "error": "context deadline exceeded", "errorVerbose": "context deadline exceeded\n\tstorj.io/common/rpc/rpcstatus.Wrap:73\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:344\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-09T20:10:40.635Z	ERROR	piecestore	upload failed	{"Piece ID": "G4DHDG7STYR5G67H4D75PAWBMRK4ZSWQCU5PWVJWQ4SNTCUYRSZA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-09T21:32:24.706Z	ERROR	piecestore	download failed	{"Piece ID": "QC6YDOTR4Q25KPPNCPWUHSPCJD5G4LJW4TUGUZ5C2IQJOJ7SA63Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "write tcp 192.168.1.3:28967->136.144.50.95:57350: use of closed network connection", "errorVerbose": "write tcp 192.168.1.3:28967->136.144.50.95:57350: use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:317\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:392\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:317\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download.func5.1:608\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22"},
2021-10-10T10:05:07.401Z	ERROR	piecestore	upload failed	{"Piece ID": "2ASLTW5VHKP5GBXV4ZILI5LMBVIDHC43C2VL44G6EP5CER55GBMQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-10T10:51:22.785Z	ERROR	piecestore	upload failed	{"Piece ID": "RAOUWYRGT35VKPOWYZ2QYY7NUIROEQUNLXFVFAIWW2Z5L5LPUBAQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "unexpected EOF", "errorVerbose": "unexpected EOF\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:342\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:220\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:104\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:97\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 0},
2021-10-10T14:04:25.783Z	ERROR	piecestore	upload failed	{"Piece ID": "SAULFUBXFSSC42E7HK2EKDZAREZWHSOKGTXYLRV3CNDZITRWSHWA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "use of closed network connection", "errorVerbose": "use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawFlushLocked:347\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:396\n\tstorj.io/common/pb.(*drpcPiecestore_UploadStream).SendAndClose:288\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload.func6:424\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22", "Size": 33024},
2021-10-10T23:00:45.613Z	ERROR	piecestore	upload failed	{"Piece ID": "PA6UCTCNOSQ6MJNQV6SFQV5TYJ3UBAM6XK2SHS3K4S2HGGWWAZ3A", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "use of closed network connection", "errorVerbose": "use of closed network connection\n\tstorj.io/drpc/drpcstream.(*Stream).rawFlushLocked:347\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:396\n\tstorj.io/common/pb.(*drpcPiecestore_UploadStream).SendAndClose:288\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload.func6:424\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22", "Size": 5376}

Here’s the pending orders:

root@NAS:/STORJ/storagenode/orders/unsent# ls -l
total 454
-rw-r--r-- 1 root root   8344 Oct 11 14:59 unsent-orders-121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6-1633914000000000000.v1
-rw-r--r-- 1 root root   5885 Oct 11 15:44 unsent-orders-121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6-1633917600000000000.v1
-rw-r--r-- 1 root root 140128 Oct 11 14:59 unsent-orders-12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S-1633914000000000000.v1
-rw-r--r-- 1 root root  67426 Oct 11 15:44 unsent-orders-12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S-1633917600000000000.v1
-rw-r--r-- 1 root root  29539 Oct 11 15:02 unsent-orders-12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs-1633914000000000000.v1
-rw-r--r-- 1 root root  17773 Oct 11 15:40 unsent-orders-12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs-1633917600000000000.v1
-rw-r--r-- 1 root root    480 Oct 11 14:58 unsent-orders-12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB-1633914000000000000.v1
-rw-r--r-- 1 root root    956 Oct 11 15:24 unsent-orders-12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB-1633917600000000000.v1
-rw-r--r-- 1 root root   2888 Oct 11 14:59 unsent-orders-1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE-1633914000000000000.v1
-rw-r--r-- 1 root root   1921 Oct 11 15:37 unsent-orders-1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE-1633917600000000000.v1
5

The pending orders looks fine. The most recent orders are submitted on a schedule back to the satellite.

My folder looks like this:

image
image706×437 35.8 KB

If you have order files older than a few hours (6+) then there may be a problem which needs investigating.

Regarding the log errors please can you find all entries for one of the pieces, e.g. “Piece ID”: “HCSRY4RDLN763EV2TN2RWIMZWSP6SFP5TJ2BFIGUL2BSS3J6HNBQ”. It said context deadline exceeded - this normally relates to a slow node or slow storage drive.

6

Hi kiwilad-nz. Just to clarify - in the very beginning node worked as expected?
Is it a chance that you accidentally change the connection? (different Wi-Fi, for example with no open port)

7

From what I can tell everything is still working as expected.

@kiwilad-nz Just a couple of notes:

  • It is perfectly normal to have up to 3 files with unsent orders from each satellite. Currently up to 18 total.
  • The errors you posted are seen by everyone every now and then and they seem to be infrequent for you as well. That’s nothing to worry about either.
    Ps: If you filter the log output for errors, it’s a good idea to mention that. If others weren’t paying attention to the timestamps they may think you only have errors in the logs.

Given these two notes, is there still something that makes you think your node might have problems? If so, could you elaborate a bit more?

2 Likes
8

Hi all,

Thanks for the replies, there is no reason to think there is an issue, I just presumably thought that there’ shouldn’t be any unsent orders at all as I read another thread on here on how to remove them and from the DB (but then they reappeared).

https://forum.storj.io/t/a-lot-of-unsent-orders/9923/18

My Storj node uploads/downloads fine (just really slowly in the vetting phase which I can understand why).

Unsure this “It is perfectly normal to have up to 3 files with unsent orders from each satellite. Currently up to 18 total.” is any documentation I have read but would save some people the worry if it were.

My Node hasn’t indicated any issues really otherwise.

Cheers

3 Likes