Workaround for order sender errors (execute this before they expire)

I have two that look just like cdhowie’s errors near the same time.
Since then I do have logs that say orders.118… finished

as long as I have a finished after the error am I ok or do I still need to restart?
If I do is this the correct and only command needed?
docker restart storagenode

You should do nothing. Please, keep it online

1 Like

Grep for order_expiration. You will always get an finished even if it wasn’t able to submit any orders. For 5 days you can ignore that. As soon as you are getting close to the order_expiration date it it time to contact us. Most of the time the issue will resolve itself by just waiting a few hours. You don’t have to panic on every error message.

@littleskunk
So my node was paused due to 15 failed audits (suspect in a row), the node was un-paused 2 days ago and everything was going great. The audit score has increased and no new failed audits however…

Now I see this in my logs.

2019-09-11T05:10:06.714Z        ERROR   orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       failed to settle orders {"error": "order: unable to connect to the satellite: transport error: context deadline exceeded", "errorVerbose": "order: unable to connect to the satellite: transport error: context deadline exceeded\n\tstorj.io/storj/storagenode/orders.(*Service).settle:260\n\tstorj.io/storj/storagenode/orders.(*Service).Settle:192\n\tstorj.io/storj/storagenode/orders.(*Service).sendOrders.func2:171\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2019-09-11T04:09:09.406Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       finished
2019-09-11T05:09:06.713Z        INFO    orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs      sending {"count": 1}
2019-09-11T05:09:06.713Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       sending {"count": 187}
2019-09-11T05:09:06.714Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      sending {"count": 10}
2019-09-11T05:09:07.857Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      finished
2019-09-11T05:09:08.341Z        INFO    orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs      finished
2019-09-11T05:10:06.714Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       finished
2019-09-11T05:10:06.714Z        ERROR   orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       failed to settle orders {"error": "order: unable to connect to the satellite: transport error: context deadline exceeded", "errorVerbose": "order: unable to connect to the satellite: transport error: context deadline exceeded\n\tstorj.io/storj/storagenode/orders.(*Service).settle:260\n\tstorj.io/storj/storagenode/orders.(*Service).Settle:192\n\tstorj.io/storj/storagenode/orders.(*Service).sendOrders.func2:171\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2019-09-11T06:09:06.715Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      sending {"count": 12}
2019-09-11T06:09:06.715Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       sending {"count": 321}
2019-09-11T06:09:07.934Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      finished
2019-09-11T06:09:09.907Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       finished
2019-09-11T07:09:06.712Z        INFO    orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs      sending {"count": 2}
2019-09-11T07:09:06.713Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       sending {"count": 148}
2019-09-11T07:09:06.713Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      sending {"count": 16}
2019-09-11T07:09:08.061Z        INFO    orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs      finished
2019-09-11T07:09:08.082Z        INFO    orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6      finished
2019-09-11T07:09:09.526Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       finished

Says it’s already expired, so there’s nothing I can do?

Context deadline exceeded doesn’t mean that the orders have expired. It just means the process didn’t finish fast enough. Those orders were resent the next hour.

That’s why this one has a higher number of orders than the other two for this satellite.

2019-09-11T06:09:06.715Z        INFO    orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW       sending {"count": 321}

Nothing to worry about. All is fine.

3 Likes

4 posts were split to a new topic: Node is trying to send orders that don’t belong to it