Hello,
I am experiencing an issue with my nodes, specifically with sending orders. After every satellite maintenance window, the updated satellite would stop working properly on my end. Today, the last remaining satellite (us1) dropped out.
Node is up and running and traffic is passing through, but orders are failing to send. Because of this, it looks like I stop receive payment for the work.
This is not a case of corrupted orders or database errors. I test it, when I enabled Cloudflare WARP, all orders were successfully sent. However, doing this naturally caused the node to lose its regular incoming traffic.
What are my options to fix this issue?
Summary
|2026-07-08T18:23:46+05:00|ERROR|orders|rpc client when sending new orders settlements|{satellite_id: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, error: order: sending settlement agreements returned an error: EOF, errorVerbose: order: sending settlement agreements returned an error: EOF\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:274\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93, request: {limit:{serial_number:AAAAAADKJ57YJDT3VSQFYDGJBQ,satellite_id:12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S,uplink_public_key:PFBOTQAFWC4L4KQVNVIDXIFRI4XA2QMQFJMUIDU2ZBJOL763T5UQ,storage_node_id:12iAtWvEZu1hMxRNcZaRYyqa4DrPTkQM34QtNzdYTKThbHNcM8A,piece_id:K5Y6KRLVH6KD45TMNC7XUVWHOVQ3UXAPDLNM6I4WP627WETPXDLQ,limit:2319360,action:4,piece_expiration:0001-01-01T00:00:00Z,order_expiration:2026-07-09T11:01:24.57256155Z,order_creation:2026-07-08T11:01:24.57256155Z,encrypted_metadata_key_id:+Qu3n5/wGic=,encrypted_metadata:tLbMYy0VEajNBN4Mb2RZlaSF4mjHxlXcPEvKK6RA4Dpjug==,satellite_signature:MEUCIF8yPnYF8JldYexzhaKb0W8x1E9awi6WJgSE6BETvx3kAiEA+cI01gxNF+vH6a6j65fVU6Up4T739t+secZNNcyilVs=},order:{serial_number:AAAAAADKJ57YJDT3VSQFYDGJBQ,amount:2319360,uplink_signature:2RqGFo/FwDiFrf3Bv8ME7He3egNjjunyCJ28AEWmJnDixdatzdsLOQ4GOQxlegHyIHftn+f9lqwtSXu8nFopAw==}}}|
|2026-07-08T18:23:46+05:00|INFO|orders|finished|{satellite_id: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, count: 6540}|
|2026-07-08T18:23:46+05:00|ERROR|orders|failed to settle orders for satellite|{satellite_id: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, satellite_id: 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S, error: order: sending settlement agreements returned an error: EOF, errorVerbose: order: sending settlement agreements returned an error: EOF\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:274\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
|2026-07-08T18:24:05+05:00|ERROR|orders|rpc client when sending new orders settlements|{satellite_id: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, error: order: sending settlement agreements returned an error: write tcp 192.168.88.244:65095->79.127.163.226:7777: wsasend: An existing connection was forcibly closed by the remote host., errorVerbose: order: sending settlement agreements returned an error: write tcp 192.168.88.244:65095->79.127.163.226:7777: wsasend: An existing connection was forcibly closed by the remote host.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:274\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93, request: {limit:{serial_number:AAAAAADKJ2PV2JFECHGUULJN3Y,satellite_id:12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs,uplink_public_key:5QCK6SVUX3WCW3XGSPK2BZUFZKSIZOCH4JORJP42EER7Q75L2AAA,storage_node_id:12iAtWvEZu1hMxRNcZaRYyqa4DrPTkQM34QtNzdYTKThbHNcM8A,piece_id:KM3Y5BG6L26UAPKZMRFFGCIPXNO4Q2XP6V53ZTRYWMNAL6KNEMKQ,limit:2319360,action:1,piece_expiration:0001-01-01T00:00:00Z,order_expiration:2026-07-08T19:05:01.185554289Z,order_creation:2026-07-07T19:05:01.185554289Z,encrypted_metadata_key_id:bND91WidtMw=,encrypted_metadata:wFBZ5o9HuZKQ2zPJp6x8Q9PryeDWBnBnJ9o1EaHx6o9n91vrqEUr7PyBDJQdri3JgKadVPEK,satellite_signature:MEQCIHCCk+VFJYCRBB5Ed9u2OgM8qf/fANgu6yrl/9eEWdFWAiALMnxUohs0zyo58bnUqNQfr9MBEvKTr2/0KFiEVThucg==},order:{serial_number:AAAAAADKJ2PV2JFECHGUULJN3Y,amount:262144,uplink_signature:sYGa1WTdDE9f90ZbTtRo3Ox49rnwXgSeG7g72ZJAYSmEQXuEuMfghveSs2M6AKDOvFQ3ii6d/HjklDOWSnczDQ==}}}|
|2026-07-08T18:24:05+05:00|INFO|orders|finished|{satellite_id: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, count: 1784}|
|2026-07-08T18:24:05+05:00|ERROR|orders|failed to settle orders for satellite|{satellite_id: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, satellite_id: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, error: order: sending settlement agreements returned an error: write tcp 192.168.88.244:65095->79.127.163.226:7777: wsasend: An existing connection was forcibly closed by the remote host., errorVerbose: order: sending settlement agreements returned an error: write tcp 192.168.88.244:65095->79.127.163.226:7777: wsasend: An existing connection was forcibly closed by the remote host.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:274\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
|2026-07-08T18:24:20+05:00|ERROR|orders|rpc client error when closing sender |{satellite_id: 121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6, error: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65097->79.127.205.244:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65097->79.127.205.244:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:285\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
|2026-07-08T18:24:20+05:00|INFO|orders|finished|{satellite_id: 121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6, count: 51}|
|2026-07-08T18:24:20+05:00|ERROR|orders|failed to settle orders for satellite|{satellite_id: 121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6, satellite_id: 121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6, error: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65097->79.127.205.244:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65097->79.127.205.244:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:285\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
|2026-07-08T18:24:21+05:00|ERROR|orders|rpc client error when closing sender |{satellite_id: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, error: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65096->79.127.205.238:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65096->79.127.205.238:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:285\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
|2026-07-08T18:24:21+05:00|INFO|orders|finished|{satellite_id: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, count: 30}|
|2026-07-08T18:24:21+05:00|ERROR|orders|failed to settle orders for satellite|{satellite_id: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, satellite_id: 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE, error: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65096->79.127.205.238:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond., errorVerbose: order: CloseAndRecv settlement agreements returned an error: manager closed: read tcp 192.168.88.244:65096->79.127.205.238:7777: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/storj/storagenode/orders.(*Service).settleWindow:285\n\tstorj.io/storj/storagenode/orders.(*Service).SendOrders.func2:219\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93}|
