My Storage Node is still trying to contact the 118U (Stefan B) satellite

This is filling up the logs unnecessarily.

image

image

Itā€™s shutdown. Those errors should gone, when that satellite will be removed from trusted list.
This change requires a fix in the storagenodeā€™s code.
I have no ETA when it will be fixed.
You can ignore those errors at the moment

3 Likes

I hope the satellite will be removed from the trusted list soon. As it is now, it teaches SNOs a bad habit of ignoring ERRORs.

2 Likes

This is just an opinion we have been ignoring certain errors since the beginning of being a SNO. Thereā€™s worse errors you do need to worry about though. Like Audit errors and missing file errors, A simple satellite being offline after being told its shutdown shouldnā€™t make you worry anymore.

4 Likes

i like how you thinkā€¦ still doesnā€™t keep my color log from flashing red once in a whileā€¦
but been learning to ignore that tooā€¦ lol

1 Like

Yeah. Marking log entries as ERROR is now useless, because a lot of entries are not actually errors.

There is no way to make it not an error, besides making a code exception. That sounds like a more awful thing causing issues in the future, then waiting for the next release which should contain the fix.
After that is live, we can safely untrust.

5 Likes

If you check the timestamp in the screenshot this error in the log happens 3 to 4 times a minute. That is a lot of errors in the log file.

@Alexey, hopefully this is addressed quickly.

1 Like

@Sasha do Not get me wrong, I operate nodes myself and find the error message not great. There is simply a balance between we need to fix this immediately and itā€™s ok to roll it out with the next release, that we need to keep in mind.

If I am not mistaken, the release is started today and thus by end of the week, your node might have the fix :+1:

2 Likes

i agreeā€¦ itā€™s certainly not a real issue, but it doesnā€™t look pretty and nor am i convinced it would be considered an error for the nodeā€¦

if seen from a nodes perspective it would make it question the internet connection maybeā€¦ ofc that looks fine, but still it would be mechanisms like that which eventually would be able to error correct and problem solve, for all the minor rookie issuesā€¦ like say somebody set a wrong ip in the run command, but ā€œ127.0.0.1ā€ works for the system, so it just keep using that until the configuration is actually good enough that it will work without reverting to hard defaultā€™s

not exactly problem solving, but for a rookie it might beā€¦

On the contrary, this is an issue to any serious sysadmin. Clarity of error messages is a reliability issue, because it should be as easy as possible to set up alerting with no false alerts and no missed alerts. Usually itā€™s as simple as considering all messages above certain logging level as alerts. This is unfortunately not the case with storage nodes.

2 Likes

i fully agree with thatā€¦

however i also understand that the storj team will have a list of priorities, and making sure what is essentially sorting of spelling or wording in log files, well it might not really be very high on the list of prioritiesā€¦

but itā€™s open sourceā€¦ you could fix it :smiley: and submit it

Sure. Though, Iā€™d consider this a regression. In the place I work, regressions are usually high-priority bugs.

Oh, indeed, I considered that. go is scary though ^^ I understand it enough to read the code, but the syntax feels so foreignā€¦

1 Like

Here is the PR that makes the error message go away :slight_smile:

4 Likes

Ah, this is the reason why the dashboard api stopped working with a 500 error "storageNode console web error: storage node dashboard service error: trust: satellite \"118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW\" is untrusted"

Will there be an update for the docker image of storagenodes, so that the wbe ui works again?

The above PR was just merged. All versions from v1.14.1 on will have the fix included and should work.
As one can see via https://version.storj.io, the rollout has not yet finished (cursor is not at the end) and thus docker nodes might are not updated yet.

1 Like

Docker images need to be tagged and do not use the cursor to my knowledge, thus none of the docker nodes were using a version with the fix. I see the commit got reverted since: https://github.com/storj/tardigrade.io/pull/169

docker nodes are like 1 week behind windows nodes in updates, to ensure the entire network isnā€™t shut down by mistake or update

so updating your docker images now would have put you back at an unmodified v1.13.3 image

you can keep up with the docker releases hereā€¦

The docker images get updated as the end of the rollout.
We would like to have kept the Satellite removed from the list, but the not working dashboard has higher priority than verbose error logs.

Just a reminder I am still getting this error even though 118U is not visible on the dashboard UI.

Running 1.16.1.