Online status percentage

Hello all.
Funny thing happens in one of my nodes, this case, a Synology:


And it doesn’t change for over 3 weeks!
It’s online on a 100/100 ISP connection, no errors in the log, no errors in the volume.
I’ve changed it’s fisycal location and issue remains.
Anyone had the same issue that can share?

The online score can stay steady for many weeks before recovering, depending on the type of offline event that occurred.

If the node got disconnected from Internet for some reason, during 1 whole week for instance, it would roughly drop all online scores to 75%, and it would stay at this number for 3 weeks because these score are the mean value over 4 weeks.

Then, it would gradually increase from 75% to 100% again during the 4th week.

Give it a few more days and see what happens. As long as your online scores do not keep dropping, I reckon they should recover soon.

4 Likes

Hi :slight_smile:
I did some digging and it doesn’t make sense…it doesn’t go up 73% online percentage, but still Uptime robot monitor shows:

You can check, when your node were offline:

 "id": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo",
  "auditHistory": [
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 31,
      "onlineCount": 29
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 28,
      "onlineCount": 2
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 30,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 35,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 25,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 35,
      "onlineCount": 3
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 29,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 36,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 28,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 26,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 25,
      "onlineCount": 6
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 24,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 21,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 40,
      "onlineCount": 6
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 14,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 29,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 22,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-29T00:00:00Z",
      "totalCount": 28,
      "onlineCount": 25
    },
    {
      "windowStart": "2021-11-29T12:00:00Z",
      "totalCount": 30,
      "onlineCount": 29
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 34,
      "onlineCount": 21
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 38,
      "onlineCount": 23
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 20,
      "onlineCount": 17
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 37,
      "onlineCount": 34
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 33,
      "onlineCount": 30
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 29,
      "onlineCount": 27
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 29,
      "onlineCount": 28
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 22,
      "onlineCount": 21
    }
  ]
}
{
  "id": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE",
  "auditHistory": [
    {
      "windowStart": "2021-11-16T00:00:00Z",
      "totalCount": 1104,
      "onlineCount": 1100
    },
    {
      "windowStart": "2021-11-16T12:00:00Z",
      "totalCount": 1434,
      "onlineCount": 1430
    },
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 967,
      "onlineCount": 935
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 463,
      "onlineCount": 45
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 167,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 152,
      "onlineCount": 16
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 229,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 242,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 204,
      "onlineCount": 12
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 288,
      "onlineCount": 31
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 177,
      "onlineCount": 12
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 136,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 188,
      "onlineCount": 50
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 379,
      "onlineCount": 57
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 341,
      "onlineCount": 45
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 448,
      "onlineCount": 39
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 228,
      "onlineCount": 18
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 143,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 111,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-29T00:00:00Z",
      "totalCount": 333,
      "onlineCount": 325
    },
    {
      "windowStart": "2021-11-29T12:00:00Z",
      "totalCount": 457,
      "onlineCount": 450
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 844,
      "onlineCount": 383
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 391,
      "onlineCount": 183
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 822,
      "onlineCount": 670
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 732,
      "onlineCount": 625
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 481,
      "onlineCount": 466
    },
    {
      "windowStart": "2021-12-06T12:00:00Z",
      "totalCount": 1068,
      "onlineCount": 1066
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 1086,
      "onlineCount": 1036
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 667,
      "onlineCount": 609
    },
    {
      "windowStart": "2021-12-10T12:00:00Z",
      "totalCount": 1341,
      "onlineCount": 1312
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 1351,
      "onlineCount": 1233
    }
  ]
}
{
  "id": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6",
  "auditHistory": [
    {
      "windowStart": "2021-11-16T12:00:00Z",
      "totalCount": 471,
      "onlineCount": 470
    },
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 228,
      "onlineCount": 223
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 160,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 52,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 53,
      "onlineCount": 6
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 71,
      "onlineCount": 3
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 59,
      "onlineCount": 6
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 57,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 58,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 49,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 46,
      "onlineCount": 2
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 55,
      "onlineCount": 11
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 154,
      "onlineCount": 48
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 76,
      "onlineCount": 15
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 101,
      "onlineCount": 9
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 65,
      "onlineCount": 11
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 65,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 43,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 280,
      "onlineCount": 175
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 110,
      "onlineCount": 50
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 311,
      "onlineCount": 273
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 198,
      "onlineCount": 166
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 114,
      "onlineCount": 111
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 321,
      "onlineCount": 309
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 201,
      "onlineCount": 188
    },
    {
      "windowStart": "2021-12-10T12:00:00Z",
      "totalCount": 376,
      "onlineCount": 373
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 335,
      "onlineCount": 304
    }
  ]
}
{
  "id": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S",
  "auditHistory": [
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 168,
      "onlineCount": 166
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 111,
      "onlineCount": 20
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 32,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 26,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 38,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 43,
      "onlineCount": 3
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 46,
      "onlineCount": 3
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 84,
      "onlineCount": 8
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 52,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 64,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 38,
      "onlineCount": 9
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 104,
      "onlineCount": 19
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 86,
      "onlineCount": 13
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 93,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 66,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 52,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 23,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-29T00:00:00Z",
      "totalCount": 108,
      "onlineCount": 104
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 150,
      "onlineCount": 75
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 77,
      "onlineCount": 40
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 133,
      "onlineCount": 109
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 165,
      "onlineCount": 138
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 105,
      "onlineCount": 102
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 244,
      "onlineCount": 232
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 175,
      "onlineCount": 164
    },
    {
      "windowStart": "2021-12-10T12:00:00Z",
      "totalCount": 201,
      "onlineCount": 198
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 259,
      "onlineCount": 228
    }
  ]
}
{
  "id": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs",
  "auditHistory": [
    {
      "windowStart": "2021-11-16T12:00:00Z",
      "totalCount": 438,
      "onlineCount": 437
    },
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 215,
      "onlineCount": 207
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 191,
      "onlineCount": 13
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 52,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 64,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 136,
      "onlineCount": 26
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 68,
      "onlineCount": 2
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 53,
      "onlineCount": 2
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 64,
      "onlineCount": 3
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 125,
      "onlineCount": 29
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 61,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 53,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 159,
      "onlineCount": 53
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 110,
      "onlineCount": 23
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 82,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 59,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 45,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 44,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-29T00:00:00Z",
      "totalCount": 87,
      "onlineCount": 85
    },
    {
      "windowStart": "2021-11-29T12:00:00Z",
      "totalCount": 85,
      "onlineCount": 84
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 304,
      "onlineCount": 166
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 76,
      "onlineCount": 41
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 227,
      "onlineCount": 193
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 180,
      "onlineCount": 155
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 91,
      "onlineCount": 87
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 240,
      "onlineCount": 236
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 184,
      "onlineCount": 170
    },
    {
      "windowStart": "2021-12-10T12:00:00Z",
      "totalCount": 336,
      "onlineCount": 327
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 267,
      "onlineCount": 245
    }
  ]
}
{
  "id": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB",
  "auditHistory": [
    {
      "windowStart": "2021-11-16T12:00:00Z",
      "totalCount": 587,
      "onlineCount": 586
    },
    {
      "windowStart": "2021-11-20T12:00:00Z",
      "totalCount": 693,
      "onlineCount": 660
    },
    {
      "windowStart": "2021-11-21T00:00:00Z",
      "totalCount": 271,
      "onlineCount": 35
    },
    {
      "windowStart": "2021-11-21T12:00:00Z",
      "totalCount": 56,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-22T00:00:00Z",
      "totalCount": 95,
      "onlineCount": 20
    },
    {
      "windowStart": "2021-11-22T12:00:00Z",
      "totalCount": 117,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-23T00:00:00Z",
      "totalCount": 89,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-23T12:00:00Z",
      "totalCount": 67,
      "onlineCount": 5
    },
    {
      "windowStart": "2021-11-24T00:00:00Z",
      "totalCount": 89,
      "onlineCount": 4
    },
    {
      "windowStart": "2021-11-24T12:00:00Z",
      "totalCount": 128,
      "onlineCount": 13
    },
    {
      "windowStart": "2021-11-25T00:00:00Z",
      "totalCount": 65,
      "onlineCount": 1
    },
    {
      "windowStart": "2021-11-25T12:00:00Z",
      "totalCount": 50,
      "onlineCount": 10
    },
    {
      "windowStart": "2021-11-26T00:00:00Z",
      "totalCount": 89,
      "onlineCount": 23
    },
    {
      "windowStart": "2021-11-26T12:00:00Z",
      "totalCount": 124,
      "onlineCount": 22
    },
    {
      "windowStart": "2021-11-27T00:00:00Z",
      "totalCount": 111,
      "onlineCount": 9
    },
    {
      "windowStart": "2021-11-27T12:00:00Z",
      "totalCount": 77,
      "onlineCount": 7
    },
    {
      "windowStart": "2021-11-28T00:00:00Z",
      "totalCount": 64,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-28T12:00:00Z",
      "totalCount": 33,
      "onlineCount": 0
    },
    {
      "windowStart": "2021-11-29T00:00:00Z",
      "totalCount": 91,
      "onlineCount": 87
    },
    {
      "windowStart": "2021-11-29T12:00:00Z",
      "totalCount": 170,
      "onlineCount": 169
    },
    {
      "windowStart": "2021-11-30T00:00:00Z",
      "totalCount": 196,
      "onlineCount": 97
    },
    {
      "windowStart": "2021-11-30T12:00:00Z",
      "totalCount": 221,
      "onlineCount": 105
    },
    {
      "windowStart": "2021-12-01T00:00:00Z",
      "totalCount": 277,
      "onlineCount": 220
    },
    {
      "windowStart": "2021-12-02T12:00:00Z",
      "totalCount": 376,
      "onlineCount": 333
    },
    {
      "windowStart": "2021-12-05T12:00:00Z",
      "totalCount": 232,
      "onlineCount": 217
    },
    {
      "windowStart": "2021-12-07T12:00:00Z",
      "totalCount": 389,
      "onlineCount": 380
    },
    {
      "windowStart": "2021-12-08T12:00:00Z",
      "totalCount": 387,
      "onlineCount": 352
    },
    {
      "windowStart": "2021-12-10T12:00:00Z",
      "totalCount": 476,
      "onlineCount": 464
    },
    {
      "windowStart": "2021-12-11T12:00:00Z",
      "totalCount": 461,
      "onlineCount": 421

From those AuditHistory logs your storagenode is not reliable. There is not a single 12 hour window (period) in the last 28 days when all of the audits were successful.

The worst days were the 21st and 28th November when 0 audits were successful:

“windowStart”: “2021-11-21T12:00:00Z”, “totalCount”: 167, “onlineCount”: 0

“windowStart”: “2021-11-28T12:00:00Z”, “totalCount”: 111, “onlineCount”: 0

Any actions I should take to prevent this? Node is online on a corporate internet connection, and it´s the only out of 5 with this issue.
Just came across this one:


and also enabled Data Scrubbing…let´s see if it improves.

The online score is not related to data. It just counts online audits and offline.
The quality of audits is measured in suspension score (for unknown audit errors) and audit score (for known audit errors such as file doesn’t exist/inaccessible, corrupted or cannot be read within 5 minutes after 3 attempts).

The online score is related to the quality of node’s availability for the customers (and for audits in particular). You channel is not reliable - your node periodically doesn’t answer on audit requests at all.

I’d just have to find out why…
Been monitoring it for the past few days:
Disks never go over 85%
No errors
Plugged on Ethernet CAT6 cable

Because of other network infrastructure - edge services, since you use an Enterprise network.

In the home network it’s a router. In the Enterprise networks - edge services (between internet and the local network).
If you say that only this one server is affected - perhaps it’s related to its network interfaces, OS configuration, overall load on the system, network misconfiguration. A lot of possibilities to make it wrong.

1 Like