You need to stop the storagenode service, check the disk for errors and fix them, after that start the service back.
Make sure that disk E: is exist and it’s a correct one.
The node is unable to finish a database migration - the disk is unresponsive.
The RAID6 is slow and you run several nodes on the same RAID, they affects each other, so it’s expected to be unresponsive in a such setup.
However, please check permissions on databases - the SYSTEM user should have read-write permissions for database files. It’s better to make a SYSTEM be an owner for databases and folders with node’s data.
You may also try to move databases to SSD, if the reason is not permissions:
2023-10-02T17:39:43+02:00 WARN trust Failed to fetch URLs from source; used cache {“source”: “https://www.storj.io/dcs-satellites”, “error”: “HTTP source: "https://www.storj.io/dcs-satellites\”: unexpected status code 403: ""“, “errorVerbose”: “HTTP source: "https://www.storj.io/dcs-satellites\”: unexpected status code 403: ""\n\tstorj.io/storj/storagenode/trust.(*HTTPSource).FetchEntries:76\n\tstorj.io/storj/storagenode/trust.(*List).fetchEntries:90\n\tstorj.io/storj/storagenode/trust.(*List).FetchURLs:49\n\tstorj.io/storj/storagenode/trust.(*Pool).fetchURLs:251\n\tstorj.io/storj/storagenode/trust.(*Pool).Refresh:188\n\tstorj.io/storj/storagenode.(*Peer).Run:904\n\tmain.cmdRun:120\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:399\n\tstorj.io/private/process.cleanup.func1:417\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:113\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:79\n\tstorj.io/private/process.ExecWithCustomConfig:74\n\tstorj.io/private/process.Exec:64\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”}
2023-10-02T17:39:43+02:00 INFO preflight:localtime start checking local system clock with trusted satellites’ system clock.
2023-10-02T17:40:04+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo”, “error”: “rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 35.229.65.196:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:04+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “error”: “rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 34.94.153.46:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:04+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “error”: “rpc: tcp connector failed: rpc: dial tcp 35.228.31.57:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 35.228.31.57:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:04+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “error”: “rpc: tcp connector failed: rpc: dial tcp 34.80.215.116:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 34.80.215.116:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:25+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “error”: “rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:25+02:00 ERROR preflight:localtime unable to get satellite system time {“Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “error”: “rpc: tcp connector failed: rpc: dial tcp 34.159.134.91:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”, “errorVerbose”: “rpc: tcp connector failed: rpc: dial tcp 34.159.134.91:7777: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190”}
2023-10-02T17:40:25+02:00 ERROR Failed preflight check. {“error”: “system clock is out of sync: system clock is out of sync with all trusted satellites”, “errorVerbose”: “system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:908\n\tmain.cmdRun:120\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:399\n\tstorj.io/private/process.cleanup.func1:417\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:113\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:79\n\tstorj.io/private/process.ExecWithCustomConfig:74\n\tstorj.io/private/process.Exec:64\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”}
2023-10-02T17:40:25+02:00 FATAL Unrecoverable error {“error”: “system clock is out of sync: system clock is out of sync with all trusted satellites”, “errorVerbose”: “system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:908\n\tmain.cmdRun:120\n\tmain.newRunCmd.func1:32\n\tstorj.io/private/process.cleanup.func1.4:399\n\tstorj.io/private/process.cleanup.func1:417\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomOptions:113\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:79\n\tstorj.io/private/process.ExecWithCustomConfig:74\n\tstorj.io/private/process.Exec:64\n\tmain.(*service).Execute.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”}
no issues with the communication from the node to internet.
Super weird, I could see that there were issues resolving DNS, or well, that error: contact service ping satellite failed. So it clearly isn’t resolving like you said. Switched from Cloudflare’s to Google’s and now it’s working. But it’s only this machine that has this issue, though I did go through a VPN tunnel. Anyway, it seems to be working fine now, such a strange issue. Thanks a lot for the help daki82, have a great evening
