Technically you should be able to do that, but the docker image isn’t really built for that. It’s easier to just start the node once, check that it’s successfully up and running. Stop it, make the changes in config.yaml and move the db files. Then start it again.
Edit: btw, I have not tried this myself yet, but I looked into how to do it because I’m planning on starting a new node with db’s in a different location as well, after another node is done vetting. My first 2 nodes don’t use separate locations for the db’s, because it’s really not necessary. But the third will be my attempt to use a slow USB2 drobo device as a node. That may or may not work and is a very specific case in which I want to move the db’s. I still maintain that most SNOs should not need this feature and are just introducing an unnecessary additional point of failure.
Or maybe create the config.yaml before the first node start ? I will try. Really thnak you for your help it work now for me.
I am in the same case than you my storage is not very stable, and I prefere use local disk for my databases
also, get that warning when i start the node:
2020-05-21T00:41:18.270371086Z 2020-05-21T00:41:18.270Z WARN piecestore:monitor Disk space is less than requested. Allocating space {“bytes”: 1051450920960}
The Docker environment variables override the config.yaml parameters due to how they are mapped in the container setup. You wouldn’t necessarily pass through an environment variable Storage2.Database-Dir unless the container was set up to grab that environment variable and override the config.yaml file.
Given that there hasn’t been any guidance yet regarding the use of a specific Docker environment variable for the database dir, the only way to change it currently is directly in the config.yaml file by including the Storage2.Database-Dir line item.
Did you move the existing databases to the new location? It seems like your node just built new databases instead. So either the db’s weren’t moved correctly or the setting wasn’t implemented right and it’s still using the old location, but has created new db’s there.
This is exactly the kind of thing I was warning people about. If you don’t do it right you end with multiple db versions, non of which contain all the data for your node anymore.
Copied all the DB files to this folder /STORJNODE1DBS
Added this line in config.yaml :
Storage2.Database-Dir: /STORJNODE1DBS
remade/started a container as usual.
Since, I rolled back everything above in reverse but nothing change about the available disk space. Otherwise the node is working perfectly according to the logs except the available space is broken.
The path in the config file points to a path inside the container. Instead you should map the path you want to use for db’s to a path inside the container, like you do for config and identity as well. Then point the config file to that path.
Your node has now created new databases inside the container path. If you have removed the container in the mean time, you’ve already lost the data in those databases, though I don’t think there is a simple way to merge them anyway. Your databases will be missing data and incorrect data will be displayed on the dashboard, api and earnings calculator.
The data itself will mostly be fine and accessible. Luckily the db’s don’t store any data vital to the core operation of the node. But data displayed in the dashboards could be off. Additionally used space calculation could be off. Hopefully that corrects itself. And differences are likely not that big. That said I recommend taking a slightly bigger overhead margin of maybe only assigning 80% of available space until you have a better idea about space available compared to the nodes calculations.
Since I followed your messages, I’m now a bit confused because some users talk about the Windows service node, others talk about the Docker version.
So, because I would like also move again my dbs on my SSD (I did it in the early alpha version by mapping the path in the docker-run command), on Docker Desktop for Windows, if my SSD drive and databases folder are located on S:\Storj\db\ and my S: drive is also shared in Docker Desktop settings :
I stop the node and remove it: docker stop -t 300 storagenode && docker rm storagenode
I add in config.yaml: Storage2.Database-Dir: "db"
I add in my docker run command: --mount type=bind,source=""S:\\Storj\\db\\"",destination=/app/db
I move my dbs in the created folder: S:\Storj\db\
I start again my node with the updated run command.
About the command, on Powershell, I always have to double all the commas and backslashes.
By the way, I just found the Odmin tutorial, but it adds me ever more confusion now, because he uses destination=/app/dbs with a “s” now. And in config, “dbs” in the command (why “dbs”, from app/dbs ?). So Storage2.Database-Dir: “dbs” ? Maybe I might ask to him directly but a concrete use-case for docker for windows and for linux like documentation should be more useful to avoid some confusions. (I thought it was hard-coded, fixed, sorry)
Going to be a bit blunt here but if you are just copying and pasting these commands and configurations and not understanding what is happening with them, then this change might not be the best to undertake at this time. This change adds another layer of problems and troubleshooting for SNOs and if implementing it is causing confusion then being able to support it yourself or getting assistance on the forums when something goes wrong is going to just increase your frustration.
Might be best to sit tight on making this change until something a bit more solid from Storj Labs is put out, especially considering this came with a warning of being very careful from @littleskunk, not something to take lightly.
I already answered a very similar question and pointed out the same mistake you made to someone else in this topic a few posts back, including an explanation of why it is this way. But I really have to agree with @Cmdrd here. If you don’t understand why things should be the way the tutorial explained, you shouldn’t be messing with this. If you want to find out yourself, look up what the --mount option actually does and keep in mind the node runs inside a docker container. It also sounds like your node has been running just fine for a pretty long time. So I don’t know why you’re making this move to begin with. Please look at buchette’s experience right above your post.
I already used the commands successfully to move the databases with docker run in the alpha, I’ve still minimal knowledge. I know the risk of added point of failure with the SSD, I don’t try to copy everything like this, that’s why I’m comparing the informations here.
I mean, I managed succesfully remotely linux debian systems in the past for different projects, but that’s doesn’t mean I didn’t make any mistakes the first times (like everyone) and I learned step by step to understand the system deeper, and in that case, obviously, I will not take the risk if I didn’t understood the things clearly.
It’s alright, I didn’t anything before clarification yes, because in the past, I mapped the databases directly in the docker-run (i’ve still the old command “–mount type=bind,source=”“M:\Storj_db\info.db”",destination=/app/config/storage/info.db"), so the mapping was fine and working (I was also looking the ressources manager and task manager to see the i/o working right in the db on the ssd). But, since this feature is new and the config is pointing inside the container, I was not sure about the good path for config.yaml. What I understood is it was pointed internally to /app/db in the docker container and we map from the separate SSD path. Sorry, I just try to understand how that works, that’s why I’m there and learn a bit deeper the things. No worries, I will not break my nodes for no reason by doing a stupid mistake creative new databases in the wrong location.
I mean I thought app/db was hard-coded, so is that mean if we point to db inside the container without s, so I might locate to Storage2.Database-Dir: “db” if my destination in docker-run is /app/db ?
It’s not hard coded, the mount will create that path inside the container. In the settings you can use either relative or direct paths. So db would work, but /app/db would work as well.
And the same - the dbs and /app/dbs will work as well, it just depends on your mount option.
Just do not mix them. If you specified in the mount option the /app/dbs, then the storage2.database-dir: should be either "dbs" or "/app/dbs" (it will even work without quotes unless you have spaces in the paths).
Thank you Alexey, now its clearer, and yes of course that totally makes sense for me, but I had still doubts how it was made. I knew actually it runs inside the docker container and with its own structure (which I actually understood when I linked the databases in the alpha, I never copy/paste without try to get the logic, but sometimes, I forget that the config file is actually inside too, since I’m used to use more physical places (I’m used to edit local configs files, that’s why when I see to enter database path dir, machinally, I think to the physical path, not the virtual one)! Sorry about that, and no problem. The nodes are still here and running just fine, that’s because I’m not -that- bad, but even when I play Ravel, I continue to do the same -noobish- mistakes. Still, I enjoy that.
