Node throwing `Warning: QUIC: rpc: quic: timeout`

[Ninetou]

Hi,
I have had a hardware failure of my old server but managed to restore all of storj data (I think) during which I had a network downtime of about ~15days. Now that everything is migrated and storj is running, dashboard shows that node is online, I’ve noticed that my online score keeps going down.

In logs storj seems to do audits but every once in a while it thows this QUIC: rpc: quic: timeout. Could it be related? What does it mean? I run a docker node on a 300/300 fiber+8thread cpu with low % usage so HW is modest.

2021-10-27T22:18:18.657Z	INFO	piecestore	downloaded	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:18:22.902Z	INFO	piecestore	download started	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:18:23.716Z	INFO	piecestore	download started	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:18:27.801Z	INFO	piecestore	downloaded	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:18:28.659Z	INFO	piecestore	downloaded	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:26:26.483Z	INFO	piecestore	download started	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:26:31.811Z	INFO	piecestore	downloaded	{"Piece ID": "DAWALK7MGPSYTZMB7JWXVSM66W7L53L5CLQNLWOFCSG7PO2XQUSA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:27:52.134Z	INFO	bandwidth	Performing bandwidth usage rollups
2021-10-27T22:27:57.651Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:27:57.789Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:27:58.149Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:27:58.190Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:27:58.566Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:27:59.200Z	WARN	contact:service	Your node is still considered to be online but encountered an error.	{"Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Error": "contact: failed to dial storage node (ID: 1r2QJ95d2bkvB3kErr1KnyF9Si6Uo3QCgY4svPJuBNnrCMNFga) at address my.domain.com:28967 using QUIC: rpc: quic: timeout: no recent network activity"}
2021-10-27T22:28:04.203Z	INFO	orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE	sending	{"count": 89}
2021-10-27T22:28:04.203Z	INFO	orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	sending	{"count": 3}
2021-10-27T22:28:04.203Z	INFO	orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	sending	{"count": 4}
2021-10-27T22:28:04.203Z	INFO	orders.12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo	sending	{"count": 2}
2021-10-27T22:28:04.391Z	INFO	orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	finished
2021-10-27T22:28:04.595Z	INFO	orders.12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo	finished
2021-10-27T22:28:04.694Z	INFO	orders.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	finished
2021-10-27T22:28:05.085Z	INFO	orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE	finished
2021-10-27T22:30:08.103Z	INFO	piecestore	download started	{"Piece ID": "ZN5NZYH2HKHH3KZG47IBXGXZDSPBMRM6ROX4KOH2A5PGXXMDW7UA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:30:12.958Z	INFO	piecestore	downloaded	{"Piece ID": "ZN5NZYH2HKHH3KZG47IBXGXZDSPBMRM6ROX4KOH2A5PGXXMDW7UA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET"}
2021-10-27T22:30:52.197Z	INFO	piecestore	download started	{"Piece ID": "VE2LZYARGO5I4FT47JORLMFUHZZZ6DVIDW5D77D52K462SS2HFNA", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "Action": "GET_AUDIT"}
2021-10-27T22:30:52.433Z	INFO	piecestore	downloaded	{"Piece ID": "VE2LZYARGO5I4FT47JORLMFUHZZZ6DVIDW5D77D52K462SS2HFNA", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "Action": "GET_AUDIT"}
2021-10-27T22:31:22.139Z	INFO	piecestore	download started	{"Piece ID": "KFJDFVF6CPWN2DMV4Z4QUCYR6AXXIBIIQ6RBLS2D77KVKKK4WADA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2021-10-27T22:32:01.725Z	INFO	piecestore	download started	{"Piece ID": "LTJ47E5ZYO6O7CMFLVTIJHILRPCUUN6GD4DH2HQOVPR2XNNUUAOQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}

[Alexey]

Hi!
This is mean that you did not setup UDP in your docker run command and did not forward UDP

Some Errors(Warns) at log

For docker setups you would need to update the run command as well. Check this page for the latest version. Storage Node - Node Operator Also make sure you have any firewall exceptions that may be required depending on your setup.

The online score is calculated within 30 days windows. To fully recover your node should be online in the next 30 days.
Make sure that you do not block outgoing traffic from your node (from any port to any port and any destination) and to your node (from any source and any port to the port of your node).

[Ninetou]

That did it. I’ve been watching the node over last few hours and the issue is gone. Thanks @Alexey

[ProTeXmining]

Hello all!

So, I read thru these and I dont see the error i have. Its probably nothing but, here it is…

2021-11-03T20:23:34.913-0400 WARN contact:service Your node is still considered to be online but encountered an error.

[ProTeXmining]

this is a few entries to my storj node log… back to back to back, it said warn, contact sutomer service…

2021-11-03T21:14:28.170-0400 INFO piecestore uploaded {“Piece ID”: “JSPQVNEQ2XOFEVP3ZBZNLXAJ2QCZ4JTPZOZBXTHEAOO43CWLGOEA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Size”: 181504}
2021-11-03T21:14:56.220-0400 INFO bandwidth Performing bandwidth usage rollups
2021-11-03T21:15:01.649-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:01.681-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:01.750-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:01.887-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:02.042-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:02.549-0400 WARN contact:service Your node is still considered to be online but encountered an error. {“Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Error”: “contact: failed to dial storage node (ID: 1a336mCuqx1nFmZoxehHYnS6hhBjF4KPL1Cs6SfHqUV4ZDFeBA) at address protexstorjnode.ddns.net:28967 using QUIC: rpc: quic: timeout: no recent network activity”}
2021-11-03T21:15:07.979-0400 INFO piecestore upload started {“Piece ID”: “SIQ5O72B66G3YVDCII7XCWFQVBQM53IPHX5RV7YEWLGJU5EPE25A”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 6999888207104}
2021-11-03T21:15:10.001-0400 INFO piecestore uploaded {“Piece ID”: “SIQ5O72B66G3YVDCII7XCWFQVBQM53IPHX5RV7YEWLGJU5EPE25A”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Size”: 181504}
2021-11-03T21:15:31.160-0400 INFO piecestore upload started {“Piece ID”: “5LUB4VVDYNURCOD5QTBIYCSUGZB3NZ7OYSSCR3WN2RALPVW7I6JQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 6999888025088}
2021-11-03T21:15:34.651-0400 INFO piecestore uploaded {“Piece ID”: “5LUB4VVDYNURCOD5QTBIYCSUGZB3NZ7OYSSCR3WN2RALPVW7I6JQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Size”: 181504}
2021-11-03T21:15:52.480-0400 INFO piecestore upload started {“Piece ID”: “AB5WRKGUNL64IBU542RBGUWCPNLWMNNMGAMQE4CIFEZOYHZLLMPQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 6999887843072}
2021-11-03T21:15:52.501-0400 INFO piecestore uploaded {“Piece ID”: “AB5WRKGUNL64IBU542RBGUWCPNLWMNNMGAMQE4CIFEZOYHZLLMPQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Size”: 26112}
2021-11-03T21:15:52.571-0400 INFO piecestore upload started {“Piece ID”: “FWSRJISZJFPKT7EXIPTWMZXH6ZBOJXFPECHACCAY6G3NOOFAEQDA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”, “Available Space”: 6999887816448}
2021-11-03T21:16:03.799-0400 INFO piecestore uploaded {“Piece ID”: “FWSRJISZJFPKT7EXIPTWMZXH6ZBOJXFPECHACCAY6G3NOOFAEQDA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”, “Size”: 227072}
2021-11-03T21:16:21.583-0400 INFO piecestore upload started {“Piece ID”: “LIR6NJYTS7S76JASSWLDJS752I2WUT7YAESRCYWDQVJL22LWZTJA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 6999887588864}

[Alexey]

Some Errors(Warns) at log

For docker setups you would need to update the run command as well. Check this page for the latest version. Storage Node - Node Operator Also make sure you have any firewall exceptions that may be required depending on your setup.

if you running a binary version (include Windows/Linux GUI), you need to forward the UDP port and allow it in your firewall.

[ProTeXmining]

@Alexey

UDP port and allow it in your firewall.

i am using the windows gui install.

all ports are forwarded and firewalle rule added already.

mynode is up and running , has been for the last 25 hours now, and i have 2gb alreayd but, it syas that error every once and a while

[ProTeXmining]

im sure its fine. everyuthing looks great now, that was when it first started.