System clock is out of sync with all trusted satellites

Node Operators troubleshooting
1

Hi all,

I’m running the storagenode binary inside a FreeNAS jail so that it has direct access to my storage. When attempting to run it, I get the following error:

2020-10-20T15:18:52.144-0500    FATAL   storagenode/peer.go:741 Failed preflight check. {"error": "system clock is out of sync: system clock is out of sync with all trusted satellites", "errorVerbose": "system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:740\n\tmain.cmdRun:209\n\tstorj.io/private/process.cleanup.func1.4:362\n\tstorj.io/private/process.cleanup.func1:380\n\tgithub.com/spf13/cobra.(*Command).execute:842\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:950\n\tgithub.com/spf13/cobra.(*Command).Execute:887\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.ExecCustomDebug:70\n\tmain.main:335\n\truntime.main:204"}

The FreeNAS host has its time synced via NTP and the jail gets its time from the host. I’ve confirmed that both the jail and the host have the correct time according to date. I’ve also confirmed that the jail has proper network access (ping, DNS).

Any ideas what I can do to fix this?

2

I’ve gotten this when I tried to use non-release storagenode versions. How did you download the initial docker container?

You can use the following to get the shasum of the containers you’re currently storing locally:

docker images --digests

What does the following return:

docker pull storjlabs/storagenode:latest
3

I should have specified - I’m not running the Docker version since I can’t run Docker in a FreeNAS jail. I cloned the Git repo and checked out the latest version here. Then, I did a make binaries and copied the storagenode binary to my jail.

4

sometimes it can take a few before the time sync’s but should work after a few minutes…
you can also try to select the NTP servers you are using, as latency to the server may result in a slightly inaccurate time…

to you it might not seem like an incorrect time, but this time code is used for the encryption or some such thing and thus is accurate down to a few decimal points of a second.

so check your NTP servers, and pick a set that is close, or simply wait a bit and see if it doesn’t sync up…
else maybe a reboot boot will help… but i doubt it

5

I would check that the timezone (and DST if applicable) is correctly set in your NAS, since all storagenode times reference GMT (Zulu) time, this could be an issue if set incorrectly. If you provide a bit more of the logs around the start up time, this could help. Sometimes the clock error can just be a side effect of a different problem.

1 Like
6

@handlebar When you say you cloned the latest version did you download from the releases (this one: https://github.com/storj/storj/releases/download/v1.14.7/storagenode_freebsd_amd64.zip) or do you mean you did a git clone https://github.com/storj/storj.git? If you cloned, did you run git checkout v1.14.7?

I haven’t built StorJ from source if you went the clone route, I have seen a few projects by other organizations where they required a few extra steps to configure if the source wasn’t a direct release download. I don’t know if someone else can comment on the source build procedure?

7

I cloned the project and then checked out the latest tag. I’ll try that binary - I totally forgot I could just grab it from the releases.

8

That worked! There must have been some option that didn’t get included when I compiled it or something. Thanks!

1 Like