Today I wanted to set up a new node. As usual, I made sure the signed identity files are placed in the right directory, I copied my config.yaml template and filled it with info for the new storage node, and started docker container. But…
2020-10-10T19:48:57.611Z INFO Configuration loaded {"Location": "/app/config/config.yaml"}
2020-10-10T19:48:57.657Z INFO Operator email {"Address": "…"}
2020-10-10T19:48:57.658Z INFO Operator wallet {"Address": "…"}
Error: Error starting master database on storagenode: group:
--- stat config/storage/blobs: no such file or directory
--- stat config/storage/temp: no such file or directory
--- stat config/storage/garbage: no such file or directory
--- stat config/storage/trash: no such file or directory
Only when I removed the config.yaml it actually started.
Why couldn’t I use my template for config.yaml to set up a new node?
Please, use the storagenode setup instead. Because all needed info, include databases will be created only with this command.
The storagenode run will not create them to prevent the start from scratch if there is something missed - either config.yaml or databases or folders tree (this useful when your disk accidentally unmounted for example).
If it is a docker version - then there should not be any info - the destination folder must be empty because of
It’s quite unusual to use a configuration file as a flag this way. From my sysadmin experience the usual practice is to drop configuration files for a service in the right places, only then install/run the actual service—this is for example what is done on Debian, as all services are already being started as part of apt install expecting their configuration to already be set up. This simplifies automated deployments.
But ok. If that’s the Storj way, then it would at least be nice to have it documented.
Yes, it’s a tweak after setup. You are right there is no any mention of config.yaml at the setup process.
I’ll create a ticket in the documentation project to add a sentence like “your destination folder must be empty for a new node”, since it’s not obvious as turned out.
Please make sure that you explicitly mention that not even config.yaml can be there, this will be very helpful for sysadmins used to usual conventions regarding service configuration.
The usual convention - to install the service then configure. In most cases you even will not have a folder or configs before the service got installed.
But you can configure before the installation, if service is allowing to do so.
The storagenode container do not allowing to do so.
How the destination can be empty and have a file in the same time?
But ok, I will add this that empty mean that folder must not have anything in it, neither files nor folders.
Update: as turned out, we actually can have a different folders in the storage destination, but you should have an empty storage subfolder (or do not have at all) and no files in the folder, especially config.yaml.
So, reformulated this way:
If you setup a brand new node, make sure that destination folder for storage doesn’t have a config.yaml or/and non-empty storage folder inside, otherwise the storagenode container could fail to start.
Depends on the system, probably. The Debian practice is to drop configuration files where they’re supposed to, thenapt install service. This is because (1) the postinst scripts will detect missing files and only if they’re not there, they open configuration dialogs, (2) the services usually are started by the postinst scripts. As I wrote before, this simplifies automated deployments.
In other words, the existence of a configuration file is a flag for the service being configured or not, not for the service being already deployed.
It is a good documentation practice to not only provide a raw description of a procedure, but also point out its interactions with other practices. The Storj procedure is explicitly at odds with some sysadmin practices, hence it is a good idea to explicitly state the conflict.
Could you please, show that in the Debian documentation, as a reference? For honestly I didn’t heard such requirement anywhere, unless you build the software from the source.
We do not provide the corresponding daemons currently! Everything that you use currently is a binary without a daemon.
The tooling you are complaining about not working as expected are not built/released yet!!
We are currently working on the necessary set of tooling to install a systemd service that also walks you through the setup. It will be shipped as a simple .deb to install it.
And it is not what I am complaining about. I wanted to automate creating new nodes on my own without waiting for official Debian packages and found a hurdle with the configuration file, that’s it.
Sorry, I do not see requirements to create a config file before installing the service from the package manager.
Scripts and symlinks are far away from the text config file in a YAML format.
But we do not talking about daemons!
In your excerpt the init system is already installed, when you configure what it should run, places scripts and symlinks on scripts to do so.
In our case it is a user-space application. The daemon in our case the only docker which we do not maintain.
Also, you can install the ssh server for example and then configure it. Perhaps it would work, if you make all configurations before install, but this is a rare case and this is why I did not see the requirement to do so.
You are the first one, who asked.
I do not see the use case for that except your specific situation.
However, for everyone interested what commands they could use, there is an article: https://documentation.storj.io/resources/faq/other-commands