Suspension Email work wrong

Vadim · February 21, 2022, 8:18pm

I have found out that during hole this month 2 of my nodes was offline, but i got email about suspension only when i fixed problem.
Online status is about 22% only, I shold get suspended already when it was below 60%?
Why I got email only then when I fixed offline problem, it shold be when problem Started.

BrightSilence · February 21, 2022, 8:57pm

This has been reported before. I think the email only goes out if a node checks in or something, which doesn’t make sense for an uptime warning email…

naxbc · February 21, 2022, 9:34pm

Now you see what I was talking about

Alexey · February 22, 2022, 4:55am

And this is why I suggests to use [Tech Preview] Email alerts with Grafana and Prometheus and do not lean on these emails from the satellites, because when they come it’s already too late.

github.com/storj/storj

[storagenode] The email about suspension is coming only after check-in to satellite

opened 05:06AM - 21 Oct 21 UTC

AlexeyALeonov

Bug

**Description**  See * https://forum.storj.io/t/whats-the-point-of-sending-the-emails-about-node-suspension/15753 * https://forum.storj.io/t/power-outage-wiped-my-node-config-then-more-woes-came/14994/6?u=alexey The suspension email is coming when the problem either fixed or at least when the node is performed check-in on the satellites, not before. The whole point of suspension email is to notify operator ASAP. **Steps to reproduce the issue:** 1. Make node offline to have a suspension level - online score below 60% 2. There would not be any email 3. Bring your node online 4. Receive suspension emails. **Describe the results you expected:** Suspension email should come when suspension is happened, not when the node become online. **Describe the results you received:** Suspension email is coming when your bring your node online. **Your environment** - Operating system and version: any - Additional environment details (Raspberry PI, Docker, VMWare, etc.): binary, docker

Bivvo · February 22, 2022, 6:24am

Or this one for Linux / Docker users, without the need of Grafana

Roberto · February 22, 2022, 6:40am

I lost a node due to a hardware problem in September… I never got the suspension email because I never brought it back online. I believe that the suspension email only arrives when the node comes back online

Alexey · February 23, 2022, 8:37am

Email about suspension can come only if the suspension score fall below 60% or online score fall below 60% and only if the node is online.
The suspension score can fall only for unknown audit errors. Known audit errors like “file not found”, “disk i/o”, etc. are known and will affect audit score instead.
So there is no suspension before disqualification, they are accounted independently. Just when the suspension score is affected the node has higher chances to be disqualified.