"error": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled"

Hi @Alexey and @Vadim I just stopped the node and ran a security update but I am getting this error in the logs

2021-02-02T01:35:43.111302796Z 2021-02-02T01:35:43.110Z	INFO	piecestore	downloaded	{"Piece ID": "PM6K4EFI3D77TMLK2GLQ6SK5CD7AY5N7RDC6F3VCCDORSX6HBPBA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-02-02T01:35:47.302810220Z 2021-02-02T01:35:47.302Z	INFO	piecestore	download started	{"Piece ID": "K3RZ6RRFJ7Y466BQCWXTQFXWU3266U6UFWJL4UTZGCGDWNMJZZ4Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2021-02-02T01:35:47.382155934Z 2021-02-02T01:35:47.380Z	ERROR	piecestore	download failed	{"Piece ID": "K3RZ6RRFJ7Y466BQCWXTQFXWU3266U6UFWJL4UTZGCGDWNMJZZ4Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "error": "untrusted: unable to get signee: trust: rpc: dial tcp 35.205.38.124:7777: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: dial tcp 35.205.38.124:7777: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:462\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}
2021-02-02T01:37:21.679372434Z 2021-02-02T01:37:21.675Z	INFO	piecestore	download started	{"Piece ID": "DHAVOQFLPBSNE5HDNBTQ2Q3RM75DAUVUCKULWNDOIYCGHJ2MXTXQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "GET"}
2021-02-02T01:37:21.785013735Z 2021-02-02T01:37:21.783Z	ERROR	piecestore	download failed	{"Piece ID": "DHAVOQFLPBSNE5HDNBTQ2Q3RM75DAUVUCKULWNDOIYCGHJ2MXTXQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "GET", "error": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:462\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}
2021-02-02T01:38:47.595379675Z 2021-02-02T01:38:47.594Z	INFO	piecestore	download started	{"Piece ID": "VWVKXTORXLT3WY3ZEXG7Z46VIPQTIGEPH5ZJPPSXI3MXVSKVGAMA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-02-02T01:38:50.613557998Z 2021-02-02T01:38:50.612Z	INFO	piecestore	downloaded	{"Piece ID": "VWVKXTORXLT3WY3ZEXG7Z46VIPQTIGEPH5ZJPPSXI3MXVSKVGAMA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-02-02T01:38:52.125399286Z 2021-02-02T01:38:52.124Z	INFO	piecestore	download started	{"Piece ID": "ZQHDEZFEV66OJAHZ5SCVQIZQDWQ6N7TEKV67Q427ABNY5LKR5N3Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2021-02-02T01:38:52.190809410Z 2021-02-02T01:38:52.189Z	ERROR	piecestore	download failed	{"Piece ID": "ZQHDEZFEV66OJAHZ5SCVQIZQDWQ6N7TEKV67Q427ABNY5LKR5N3Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "error": "untrusted: unable to get signee: trust: rpc: dial tcp 35.205.38.124:7777: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: dial tcp 35.205.38.124:7777: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:462\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}
2021-02-02T01:42:17.175220049Z 2021-02-02T01:42:17.174Z	INFO	piecestore	download started	{"Piece ID": "2RMOK5VY6PUVCTJH26FYUVZKVEM3UJ6U4XZUGCAUHCQUO63XL7MA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-02-02T01:42:21.089538859Z 2021-02-02T01:42:21.088Z	INFO	piecestore	downloaded	{"Piece ID": "2RMOK5VY6PUVCTJH26FYUVZKVEM3UJ6U4XZUGCAUHCQUO63XL7MA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-02-02T01:43:17.491428288Z 2021-02-02T01:43:17.489Z	INFO	piecestore	download started	{"Piece ID": "JGEFDWPEUU2E2EYVA3YWOAEZJUBQXCFNTYO36C7UC3BMWEZVB4OA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2021-02-02T01:43:17.494821836Z 2021-02-02T01:43:17.493Z	ERROR	piecestore	download failed	{"Piece ID": "JGEFDWPEUU2E2EYVA3YWOAEZJUBQXCFNTYO36C7UC3BMWEZVB4OA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:462\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}
2021-02-02T01:43:36.038551007Z 2021-02-02T01:43:36.038Z	INFO	piecestore	download started	{"Piece ID": "XM4FPAJ4FPI25MVQWWLKGX3VPMZ7M562XLSV5U3SBVYJRMOA6QRA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2021-02-02T01:43:36.194178438Z 2021-02-02T01:43:36.192Z	ERROR	piecestore	download failed	{"Piece ID": "XM4FPAJ4FPI25MVQWWLKGX3VPMZ7M562XLSV5U3SBVYJRMOA6QRA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "error": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: dial tcp: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:462\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}

Then I don’t have any data ingress today

Do you know if there is any problem with my node or if I am missing something?

Thanks!

It’s a usual long tail cancellation (your node is too slow to win the race with other nodes).
It’s impossible to be closest to every customer in the world, so your node cannot win all the races.

1 Like

Thank you @Alexey. I was curious but that makes a lot sense.