Most of my nodes say QUIC is Misconfigured [Resolved]

Node Operators troubleshooting
, ,
1

Most of my nodes say QUIC is Misconfigured, Since a couple of minutes ago.

2

Hello @Raptor,
Welcome to the forum!

Please use this troubleshooting guide to figure out why is that:

3

The problem is not on my end, they just cant ping the us satellite
anymore.

4

And therefore in the dashboard they say QUIC is Misconfigured.

5

Hi, Alexey!

I have the same problem right now, one-by-one already 3 of my 4 nodes got QUIC -Misconfigured.
Data is still uploading and downloading, but i have these error in logs:

2024-06-11T02:58:53Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 1, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:58:55Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 2, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:58:56Z ERROR nodestats:cache Get pricing-model/join date failed {“Process”: “storagenode”, “error”: “payouts service: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)”, “errorVerbose”: “payouts service: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetAllPaystubs:114\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:87\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run:86\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:58:58Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 3, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:59:00Z INFO reputation:service node scores updated {“Process”: “storagenode”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Total Audits”: 0, “Successful Audits”: 0, “Audit Score”: 1, “Online Score”: 1, “Suspension Score”: 1, “Audit Score Delta”: 0, “Online Score Delta”: 0, “Suspension Score Delta”: 0}
2024-06-11T02:59:00Z INFO reputation:service node scores updated {“Process”: “storagenode”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Total Audits”: 0, “Successful Audits”: 0, “Audit Score”: 1, “Online Score”: 1, “Suspension Score”: 1, “Audit Score Delta”: 0, “Online Score Delta”: 0, “Suspension Score Delta”: 0}
2024-06-11T02:59:01Z INFO reputation:service node scores updated {“Process”: “storagenode”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Total Audits”: 228, “Successful Audits”: 227, “Audit Score”: 1, “Online Score”: 0.9973684210526316, “Suspension Score”: 1, “Audit Score Delta”: 0, “Online Score Delta”: 0, “Suspension Score Delta”: 0}
2024-06-11T02:59:01Z ERROR nodestats:cache Get stats query failed {“Process”: “storagenode”, “error”: “nodestats: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)”, “errorVerbose”: “nodestats: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:59:03Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 4, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:59:11Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 5, “error”: “ping satellite: check-in ratelimit: node rate limited by id”, “errorVerbose”: “ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T02:59:28Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 6, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T03:00:01Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 7, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T03:01:07Z ERROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “attempts”: 8, “error”: “ping satellite: contact: overlay: failed to get node info from DB”, “errorVerbose”: “ping satellite: contact: overlay: failed to get node info from DB\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T03:03:10Z ERROR nodestats:cache Get disk space usage query failed {“Process”: “storagenode”, “error”: “nodestats: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)”, “errorVerbose”: “nodestats: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}
2024-06-11T03:03:13Z ERROR nodestats:cache payouts err {“Process”: “storagenode”, “satellite”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”}
2024-06-11T03:03:14Z ERROR nodestats:cache Get held amount query failed {“Process”: “storagenode”, “error”: “payouts service: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)”, “errorVerbose”: “payouts service: satellitedb: ERROR: failed indeterminate commit recovery: replica unavailable: (n25,s25):5 unable to serve request to r180119:/Table/240/{6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"-9/NULL/NULL/NULL/1/1915-11-19T02:40:27.058286999Z/"\xe1\x02\x13f\x1b)=N\x9c\x93t\xd6yr;f\xd3XHN\xae\xad\x99\xb2\xf4\xc29\xe0\x00\x00\x00\x00"} [(n18,s18):1, (n25,s25):5, (n29,s29):6, next=7, gen=13854, sticky=1710993014.124108885,0]: lost quorum (down: (n18,s18):1,(n29,s29):6); closed timestamp: 1718072696.032994625,0 (2024-06-11 02:24:56); raft status: {"id":"5","term":291,"vote":"1","commit":92548870,"lead":"0","raftState":"StatePreCandidate","applied":92548870,"progress":{},"leadtransferee":"0"}: have been waiting 61.00s for slow proposal RequestLease [/Table/240/6/NULL/NULL/2020-11-22T00:06:48.481998Z/2020-11-27T17:34:56.997818Z/2020-11-20T16:48:02.266082Z/"\xfe\xadgD\x1a\n\x16J\xb8\x86\x99ܚF-\x99\x97\xe41.\x12\xa2Ej\xaf\r’@\x00\x00\x00\x00"] (SQLSTATE XXUUU)\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78”}

6

I have the exact same error on all my nodes that i restarted after they went offline in Uptime Kuma

7

Seeing ping too for 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S

8

If that’s only US1 (12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S), then it’s likely the same issue, as

You may subscribe to the status updates:

1 Like
9

This is interesting, my single node dashboards shows all nodes online, but docker ones have a QUIC misconfigured mark, the Windows service node both green.

11

All of my nodes are also showing online, but QUICK is showing misconfigured. Using docker.

12

Sorry, I meant QUIC is Misconfigured, All of them are online.

2 Likes
13

Now and Windows one become QUIC Misconfigured too.

14

My nodes shows QUIC OK now.