Low Online Statistics despite no connection issue

[JeanS]

I am having 2* 20TB nodes.
I have stable fiber connection (1Gbps) with Cellular backup connection and I have not had any connection breaks for several months. Also server that is running nodes has been running 24/7 without issues.

Still Online statistics shows quite low procentage:
saltlake.tardigrade.io is worst with about 95% Online on both nodes.

I am located at Finland. Any idea why I am having low online statistics? And any harm on this?

[nerdatwork]

How is the online score calculated? - Storj Docs

[Roxor]

Nothing wrong with 95%: you’re fine.

[andrew2.hart]

Mine is at 95% too. There is no way that the node is offline for an hour a day

[JeanS]

I have big difference on satellites;
us1.storj.io is on first node 100% and other node 99.45% → these are close to reality

[Julio]

Maybe your DNS is acting a little flakey, or theirs.

[Alexey]

You can use these scripts to find out when your node was not available and for which satellite:

My uptime should be 100% on all satellites i have not gotten any uptime robot notifications of downtime in months

Would you mind to provide dashboard from the uptimerobot? And also, please, provide result of the command: PowerShell foreach ($item in ((Invoke-WebRequest http://localhost:14002/api/sno).Content | ConvertFrom-Json).satellites.id) { ((Invoke-WebRequest http://localhost:14002/api/sno/satellite/$item).Content | ConvertFrom-Json).auditHistory.windows | Where-Object{$_.totalCount -ne $_.onlineCount} | ForEach-Object{Write-Host $item; $_ | Format-Table -AutoSize} } bash for item in `…

[JeanS]

I had some short breaks based on Uptime Robot, but well beyond 95% uptime

I am using Proxmox / UbuntuServer for this Storj Node.
Is there similar script for Linux?

[Alexey]

Yes, use the bash version.

[JeanS]

storj@storj2:~$ for item in curl -sL http://localhost:14002/api/sno | jq '.satellites[].id' -r; do
curl -s http://localhost:14002/api/sno/satellite/$item |
jq ‘{id: .id, auditHistory: [.auditHistory.windows | select(.totalCount != .onlineCount)]}’
done
{
“id”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”,
“auditHistory”: [
{
“windowStart”: “2024-07-12T00:00:00Z”,
“totalCount”: 271,
“onlineCount”: 172
},
{
“windowStart”: “2024-07-12T12:00:00Z”,
“totalCount”: 205,
“onlineCount”: 57
},
{
“windowStart”: “2024-07-27T12:00:00Z”,
“totalCount”: 384,
“onlineCount”: 210
},
{
“windowStart”: “2024-07-28T00:00:00Z”,
“totalCount”: 389,
“onlineCount”: 233
},
{
“windowStart”: “2024-08-02T00:00:00Z”,
“totalCount”: 398,
“onlineCount”: 286
},
{
“windowStart”: “2024-08-06T12:00:00Z”,
“totalCount”: 243,
“onlineCount”: 80
}
]
}
{
“id”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”,
“auditHistory”: [
{
“windowStart”: “2024-07-12T00:00:00Z”,
“totalCount”: 308,
“onlineCount”: 279
},
{
“windowStart”: “2024-07-12T12:00:00Z”,
“totalCount”: 352,
“onlineCount”: 289
},
{
“windowStart”: “2024-07-13T12:00:00Z”,
“totalCount”: 210,
“onlineCount”: 207
},
{
“windowStart”: “2024-07-20T00:00:00Z”,
“totalCount”: 326,
“onlineCount”: 325
},
{
“windowStart”: “2024-08-05T12:00:00Z”,
“totalCount”: 518,
“onlineCount”: 463
}
]
}
{
“id”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”,
“auditHistory”: [
{
“windowStart”: “2024-07-10T12:00:00Z”,
“totalCount”: 76,
“onlineCount”: 73
},
{
“windowStart”: “2024-07-12T00:00:00Z”,
“totalCount”: 106,
“onlineCount”: 101
},
{
“windowStart”: “2024-07-12T12:00:00Z”,
“totalCount”: 84,
“onlineCount”: 72
},
{
“windowStart”: “2024-07-13T12:00:00Z”,
“totalCount”: 94,
“onlineCount”: 93
},
{
“windowStart”: “2024-07-18T12:00:00Z”,
“totalCount”: 181,
“onlineCount”: 180
},
{
“windowStart”: “2024-07-20T00:00:00Z”,
“totalCount”: 214,
“onlineCount”: 206
}
]
}
{
“id”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”,
“auditHistory”: [
{
“windowStart”: “2024-07-10T12:00:00Z”,
“totalCount”: 247,
“onlineCount”: 241
},
{
“windowStart”: “2024-07-12T00:00:00Z”,
“totalCount”: 410,
“onlineCount”: 378
},
{
“windowStart”: “2024-07-12T12:00:00Z”,
“totalCount”: 352,
“onlineCount”: 285
},
{
“windowStart”: “2024-07-13T12:00:00Z”,
“totalCount”: 425,
“onlineCount”: 424
},
{
“windowStart”: “2024-07-18T12:00:00Z”,
“totalCount”: 540,
“onlineCount”: 531
},
{
“windowStart”: “2024-07-20T00:00:00Z”,
“totalCount”: 746,
“onlineCount”: 733
},
{
“windowStart”: “2024-07-29T00:00:00Z”,
“totalCount”: 733,
“onlineCount”: 669
},
{
“windowStart”: “2024-08-01T00:00:00Z”,
“totalCount”: 610,
“onlineCount”: 531
},
{
“windowStart”: “2024-08-05T00:00:00Z”,
“totalCount”: 728,
“onlineCount”: 613
}
]
}

[Alexey]

So now you have dates, when your node was not available for these satellites. You may use this info to find out - why. You may check logs of your firewall or the router on that date.
I would like to suggest to also disable a “smart” protection on your router, like DDoS prevention and similar, to do not block requests from the satellites.

[JeanS]

I am using pfSense and it has Suricata. I disabled Suricata to check whether any impact to Online statistics. How satellites check connection?

[Alexey]

When the node will check-in (every hour by default), the satellite would try to contact the node by the provided external address. And you can see in the logs the result of this check. The satellite would check the QUIC connection with a fallback to TCP.