Changelog v0.27.1

[Pavmer]

Downloaded a new installer and launched

https://documentation.storj.io/setup/gui-windows/storage-node

The installer updated the version very quickly…

[edonaldson]

My Windows GUI install updated automatically. Very nice.

[mitchellds]

Here is a great example of why the 5 hours of downtime is simply unacceptable.

Upon release of 27.1, my storage node’s bandwidth and orders databases got trashed.

It took my system over 20 hours to rebuild just the orders database.

My storage node worked perfectly…actually, it was rocking…before 27.1, and then BAM, an emergency.

My node is now back up and running; however, I have no idea what the damage is. I’ve been a SNO since May so if I get disqualified, I lose everything in escrow (plus the downtime for the last 20 hours) through no fault of my own.

[heunland]

Just to make it clear to SNOs that may not have seen our earlier announcements regarding properly stopping a node, manual stopping the node without the -t 300 qualifier in the docker stop -t 300 storagenode command as described in our manual update instructions, or not adding this time delay to the watchtower command as described in our documentation for automatic node updating can lead to the type of database corruption described in this thread. The update to this release is not what caused the corruption of the database.

[deathlessdd]

I feel like people might of forced a stop because they didn’t see there node updating and updated it manually without using the correct commands to stop it.

[heunland]

There is no need to force your node to immediately update when we issue a new release. If all nodes updated at the same time, the customers may not be able to download any files while all the nodes are offline. To prevent this from happening, we purposefully implemented auto updates to not happen on all nodes at the same time. Your node may update right away or it may take many hours to do so. There is no urgency in getting the update done immediately. So it is best to wait for the automatic update rather than taking measures to force an update, and worse, without using the proper safe method that allows proper stop without data corruption.

I may add that whenever you make any changes to your node configuration, it is best that you go back to our documentation and check that you are using the latest recommended instructions, and not outdated commands. As this is still a beta, we may have found bugs and updated our instructions accordingly. While we announce important changes promptly, a SNO may have missed the announcement and not be aware of an important change because of this. Please also always read the entire changelog when we do any new release, as there are important details in the description that alert our users about changes that may affect them.

[deathlessdd]

Of course, Also Maybe should remove the documentation on the site showing how to update the node manually. I have a good feeling we have alot of impatient people here and when they see everyone else’s nodes have been updated, they dont want to wait or feel like there node didnt update correctly because of watchtower.

[heunland]

It is clearly shown already that manual updates are not the preferred method, however there are nodes that need to use manual updating for various reasons and we prefer they can see how to do it properly. If we do not show the instructions it is even more likely they will leave out the -t 300 from the docker stop command. Windows GUI installs already have the automatic updating function integrated so are not affected.

[deathlessdd]

Your right probably a bad Idea to remove it from the site, but I guess theres no way to pervent people from running the wrong commands to stop there nodes. I guess well have to wait for docker to no longer be used for linux to be able to achieve that right?

[mitchellds]

I now realize that the downtime is likely 100% my fault. I had no idea that there was an updated watchtower command. I had been using the same watchtower command that I put in place waaaaaay back in May 2019. I’m usually on top of all of the changes and updates but I missed this particular update.