[image] storj/storj Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure. - storj/storj v0.22.0 will contain a storage node database migration that needs several minutes to hours. To reduce the time…

Looking at the source code on github… It seems the database is transitioning from sqlite to postgresql. Is that accurate? If so, this is great news! And will there be a db backup function enabling daily db export external to the docker container?

I think that a node with a day old db would fail a lot of audits, since there probably is no way to signal the satellite “hey, I had to restore a backup that’s x hours old”.

[image] anon27637763: Looking at the source code on github… It seems the database is transitioning from sqlite to postgresql. Can you show me the code line? I am not aware of any postgres migration.

storj/storj/blob/6e6d0ad9b8924d0c63478ef2a119b43baebc051a/storage/postgreskv/schema/migrate.go#L38 if err != nil { return errs.New("error parsing schema: %+v", err) } if schema != "" { err := pgutil.CreateSchema(db, schema) if err != nil { return errs.New("error creating schema:…

External db backup as well as external log backup is very useful for troubleshooting and general record keeping. It’s also very useful for interface development using actual network acquired data.

That code line was added 11 month ago. I don’t see any affect on storage nodes. Yes we are using postgres for the satellite for a long time now.

[image] littleskunk: Yes we are using postgres for the satellite for a long time now. That’s good to know. Well… I, for one, vote for postgres migration for storagenodes. In any case, I was paging through the source code to figure out what exactly was meant by “database migration” … Can you…

[image] anon27637763: Well… I, for one, vote for postgres migration for storagenodes. You have 2 options. You can implement postgres for storagenodes and we will most likely merge it even if it will be slower compared to sqlite3 You can show us how slow sqlite3 is and the improvement you wo…

[image] littleskunk: We split the sqlite DB into multiple sqlite files. That way we can lock them individualy. That will allow us to scale better on bigger storage nodes. Doesn’t this answer the first part of response to my post about voting for postgresql? I’m not complaining about sqlite3.…

[image] anon27637763: However, there should be little doubt that postgres is a better fit for the application… especially since it’s currently being used for satellites. I have to disagree. For the storage nodes there should be no doubt that posgres is slower than sqlite3. Postgres will open …