How to move DB’s to SSD on Docker

So I redirected all logs to my DB location on the SSDs too now, the difference is quite visible.
(and the compression at zfs recordsize=1M for logs is quite impressive with a compressratio of 5x. Good for my just 120GB big OS drive…)

Before something like this:

After moving DBs and Logs to the SSD:

And just for fun from my full node that gets no ingress :smiley:

4 Likes

By the way, what software are you using to monitor your IOPS? I’m unfamiliar with what monitoring softwares to use if you want to try and optimize your StorJ setup as best as one could. Do you have some recommendations for Windows?

I’m using netdata. not sure how easy it is to use on windows though

Yes. I had zero race lost pieces on synology nas working for a week but after plugging usb3.1 ssd into it and moving dbs to ssd I got 55 race lost errors in a night after ssd was added. Put everything back now. 1 race lost error in 8 hours.

1 Like

just be sure the usb is stable… it can vary a lot from one device to another… it might work without issue, it might need power manage disable before it will work without issue, or it might just intermittently reconnect which can cause issues.

so certainly keep an eye on it until you are confident there is no such issues, but good to hear that you got your io problems resolved.

Hi, I just moved my databse-files to a nvme ssd, cause the databases are getting larger and it takes minutes to load it from the storage drive.
In case my cache SSD with the database-files on will fail, is the node simply rebuilding the databses if not found in case there lost?

Your local stat and history will be lost, but node will survive.

1 Like

But what exactly happens when you lose a current db-fileset and then replace it with an older db-fileset?

Lets say you are making today a backup of the db-fileset, you can call it then db-fileset-2022-10-22.

When you are running the nodes till 2022-12-31 and then the SSDs gets destroyed with the db-fileset, what happens when you setup the db-fileset-2022-10-22 from 2023-01-01 on?

1 Like

You will lose the history since 2022-10-22
And your usage will be wrong until the end of the month.

@Odmin I like to know how big can the data bases can grow lets say a 25 TB storage node

@sorry2xs there are wide variances but as an example my 31 month old node storing 14.92TB, of data, has databases 746MB in size.

Note - some broken or damaged databases have grown very large.

finally tried this getting an parsing error in the config.yaml

Most likely there’s a format or typing error. The log file will specify what’s wrong with the config.yaml

1 Like

Hi everyone,

after i changed my YAML file , i for errors in the config.yaml and node wouldnt start, so i reverted back.

have any steps been missed / require updating from the above ?

Cheers

You need to use a plain text editor to edit config.yaml, however, you may also pass this parameter as an argument after the image name, i.e. --storage2.database-dir=dbs

in the

docker run

command ?

Yes

docker run -d ... \
...
--mount type=bind,source=/mnt/storj/storagenode/identity,destination=/app/identity \
--mount type=bind,source=/mnt/storj/storagenode,destination=/app/config \
--mount type=bind,source=/mnt/ssd/storagenode/dbs,destination=/app/dbs \
...
storjlabs/storagenode:latest \
--storage2.database-dir=dbs
1 Like

Hi
Have problem getting it too work .
Have add
–mount type=bind,source=“/mnt/user/storj-database/”,destination=/app/dbs
Have cpy all files only the files not folder from
My storj folder
/mnt/user/storj/storage
But can see the files still get use like
bandwidth.db-wal
What the reasons it not use from the ssd

Edit have miss th erdit config.yaml part

You need also either update your config.yaml with a parameter

Or use it as an argument

1 Like

Yes forgot that work now have just add the thing in the config

1 Like