Здравствуйте. После обновления Storja с 1.85.1 на 1.86.1 появилась такая проблема - на 2х сателлитах, saltlake и us1 стал падать онлайн, хотя UptimeRobot показывает почти идеальный онлайн (на других сателлитах 99,98, но теперь тоже начал падать), порты так же открыты, никаких работ не проводил, оборудование и софт не менял, ничего не трогал. В чем может быть причина? И как это пофиксить?
Hi. After last Storj updates from 1.85.1 to 1.86.1 I found issue - 2 satellites, saltlake and us1 online start falling, but my UptimeRobot shows near perfect online (other 4 satellites start falling too now), my ports still open, nothing was changed in my hardware or software, and I did nothing on this machine. What happend and how can I fix this?
часть ошибок из логов, прошу помощи в разъяснении:
part of my logs:
2023-09-17T10:18:43+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 4, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T13:27:39+05:00|ERROR|piecestore|download failed|{Piece ID: SCTW6VYBWGCEN5MCPLUEOXTTVHIQMIMTS5LKDEH4SB4L43T4WPFA, Satellite ID: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, Action: GET, Offset: 0, Size: 311296, Remote Address: 51.77.227.245:39216, error: write tcp 192.168.1.3:28967->51.77.227.245:39216: wsasend: An existing connection was forcibly closed by the remote host., errorVerbose: write tcp 192.168.1.3:28967->51.77.227.245:39216: wsasend: An existing connection was forcibly closed by the remote host.\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:367\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:458\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:349\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).sendData.func1:830\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22}|
|2023-09-17T14:13:50+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 1, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T14:14:00+05:00|ERROR|orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE|failed to settle orders for satellite|{satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, error: order: failed to start settlement: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: failed to start settlement: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:254\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func1:205\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T14:15:40+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T14:16:02+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 2, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T14:16:25+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 3, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T14:16:50+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 4, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:13:50+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 1, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:16:02+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:16:03+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 2, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:16:27+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 3, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:18:34+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 2, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:18:51+05:00|ERROR|nodestats:cache|Get stats query failed|{error: nodestats: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: nodestats: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:20:47+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 3, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:23:22+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 4, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:24:12+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 5, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:26:41+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 6, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:27:55+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 7, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:31:11+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 8, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:33:20+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 9, error: ping satellite: check-in ratelimit: node rate limited by id, errorVerbose: ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T15:38:18+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 10, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T15:49:22+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 11, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T16:07:08+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 12, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T16:13:50+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 1, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T16:14:12+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 2, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T16:14:35+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 3, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T16:15:39+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T16:16:01+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, attempts: 2, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T16:43:47+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T17:15:40+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T17:16:02+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 2, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T17:16:25+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 3, error: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: ping satellite: rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190}|
|2023-09-17T17:18:41+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo, attempts: 4, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T18:14:18+05:00|ERROR|orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S|failed to settle orders for satellite|{satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, error: order: failed to start settlement: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: failed to start settlement: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:254\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func1:205\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T18:15:39+05:00|ERROR|contact:service|ping satellite failed |{Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, attempts: 1, error: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out, errorVerbose: ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 95.105.113.83:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75}|
|2023-09-17T19:28:39+05:00|ERROR|piecestore|download failed|{Piece ID: P5KZ5WUJXZLPL4ZSKNP4AR5V2NN4HUTHUC2QK55QIRD6CUCQ4Q6A, Satellite ID: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, Action: GET, Offset: 2029056, Size: 114688, Remote Address: 38.88.241.43:33300, error: write tcp 192.168.1.3:28967->38.88.241.43:33300: wsasend: An existing connection was forcibly closed by the remote host., errorVerbose: write tcp 192.168.1.3:28967->38.88.241.43:33300: wsasend: An existing connection was forcibly closed by the remote host.\n\tstorj.io/drpc/drpcstream.(*Stream).rawFlushLocked:401\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:462\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:349\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).sendData.func1:830\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22}|
|2023-09-17T19:30:08+05:00|ERROR|piecestore|download failed|{Piece ID: SCTW6VYBWGCEN5MCPLUEOXTTVHIQMIMTS5LKDEH4SB4L43T4WPFA, Satellite ID: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, Action: GET, Offset: 0, Size: 311296, Remote Address: 51.77.227.245:34598, error: write tcp 192.168.1.3:28967->51.77.227.245:34598: wsasend: An existing connection was forcibly closed by the remote host., errorVerbose: write tcp 192.168.1.3:28967->51.77.227.245:34598: wsasend: An existing connection was forcibly closed by the remote host.\n\tstorj.io/drpc/drpcstream.(*Stream).rawWriteLocked:367\n\tstorj.io/drpc/drpcstream.(*Stream).MsgSend:458\n\tstorj.io/common/pb.(*drpcPiecestore_DownloadStream).Send:349\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).sendData.func1:830\n\tstorj.io/common/rpc/rpctimeout.Run.func1:22}|
Здравствуйте @hemlock,
Добро пожаловать на форум!
Что-то блокирует подключения с вашего узла и к вашему узлу. Причём не всегда, а как будто используется throttling или какой-то фильтр у вас или у ISP. Проверьте, не включена ли какая-либо защита от DDoS или подобное на вашем роутере?
С помощью этих скриптов вы можете посмотреть, когда ваш узел был недоступен:
Ещё может быть, что ваш роутер сбрасывает подключения. Могу порекомендовать попробовать его перезагрузить - уменьшится ли количество ошибок подключения?
Здравствуйте, вы не одиноки. У меня такая же проблема появилась после обновления с 1.85.1 на 1.86.1. Настройки в роутер не вносил и ddos защиты не стоит на нем, спрашивал у провайдера, мне ответили что ничего не блокируют, порт открыт. Причем у меня проблема на тех же сателлитах. Странная ситуация.
@Alexey А не подскажите, можно как-то откатиться на старую версию Storj node ради экспериментов ?
Здравствуйте @arenonode,
Добро пожаловать на форум!
Я не думаю, что это связано с новой версией, так что откат ничего не даст. Но вы можете попробовать откатиться, но только Windows/Linux GUI, для docker это будет сделать непросто.
- остановить сервисы storagenode и storagenode-updater,
- переименовать
storagenode.exe, - переименовать предыдущую версию в
storagenode.exe, - запустить сервис storagenode
Однако можно попробовать иначе:
- измените параметр
server.address: :28967
на использование адреса вашего ПК:
server.address: 192.168.1.20:28967
- перезапустите сервис storagenode
А можете скинуть инструкцию по откату с docker на linux.
Я как понимаю путь примерно такой:
- Остановка storagenode и watchtower.
- Удаление образа последней версии storagenode.
- Скачивания с docker hub нужной версии.
- Запуск ноды
Но вот такой откат не скажется на ноде как-то?
Для docker не сработает - образ больше не содержит storagenode. Там только downloader и supervisor. При запуске он скачает storagenode-updater и запустит его, а storagenode-updater скачает последнюю версию storagenode и запустит её. Поэтому для docker возможности отката нет вообще, либо очень сложно.
Инструкции по миграции с docker на Linux binary нету, но вы можете установить Linux binary: [Tech Preview] Linux Installer Part 2 и указать правильное расположение identity и storage.
Зато можно так же попробовать закрепить за одним интерфейсом, а не всеми, указав локальный IP в портах:
-p 192.168.1.20:28967:28967/tcp \
-p 192.168.1.20:28967:28967/udp \
Здравствуйте, @hemlock и @Alexey .
@hemlock не подскажешь удалось ли решить проблему?
У меня пока туго все. Провайдер и настройки сетевого оборудования отпадают, так как проводил эксперимент на разных провайдерах и сетевом оборудовании.Заметил, что если делать telnet до спутника мин 5 все ок, а потом мин 2 нет связи. Да еще пробовал пинговать спутник, 17% потерянных пакетов. Есть идея, что какой-то узел сети вне зоны ответственности моего провайдера производит блокировку и при смене маршрута до спутника попадаю на этот узел.
@Alexey а на скольки % online происходит дисквалификация нод с спутников? У меня сейчас на проблемных узлах 89% online, хочу примерно прикинуть сколько у меня есть времени для решения проблем, не хотелось бы терять ноду после 9 месяцев работы(.
Что-то фильтрует трафик и периодически его блокирует. Очень похоже на “умную” систему блокировки “подозрительной активности”.
Кстати, попробуйте проверить сеть не с этого ПК, может дело в драйверах сетевой карты.
Если есть возможность - попробуйте подключить интернет напрямую к этому ПК, минуя роутер. Не забудьте включить брандмауэр на ПК! И добавить разрешающие правила для TCP и UDP портов узла.
Дисквалификация происходит при 0% online score или больше 30 дней offline.
Проверял с 3 разных ПК, с 3 разных провайдеров и 1 ПК был подключен напрямую. Роутеры были разных фирм. Попробую сопоставить маршруты с 3 разных ПК. Просто должна быть совпадающая точка сети, где находиться “умная” система блокировки “подозрительной активности”.
Можете использовать команду mtr:
mtr us1.storj.io
mtr ap1.storj.io
mtr eu1.storj.io
Но если это не ваш провайдер, то можно попросить вашего провайдера скорректировать маршрут.
Альтернативой будет использование VPN с функцией переадресации портов, такие как portmap.io, ngrok, PIA, AirVPN, PureVPN, и т.д., либо настроить свой через арендованный сервер.