Changelog v0.21.4

littleskunk · October 3, 2019, 2:39pm

You have 2 options.

You can implement postgres for storagenodes and we will most likely merge it even if it will be slower compared to sqlite3
You can show us how slow sqlite3 is and the improvement you would expect from postgres. I recommend starting here: Guide to debug my storage node, uplink, s3 gateway, satellite

A simple vote without any data to proof your point doesn’t work

storj/storj/blob/6e6d0ad9b8924d0c63478ef2a119b43baebc051a/storagenode/storagenodedb/database.go#L799-L835


      
          			{
          				DB:          db.deprecatedInfoDB,
          				Description: "Split into multiple sqlite databases",
          				Version:     23,
          				Action: migrate.Func(func(log *zap.Logger, _ migrate.DB, tx *sql.Tx) error {
          					// Migrate all the tables to new database files.
          					if err := db.migrateToDB(ctx, BandwidthDBName, "bandwidth_usage", "bandwidth_usage_rollups"); err != nil {
          						return ErrDatabase.Wrap(err)
          					}
          					if err := db.migrateToDB(ctx, OrdersDBName, "unsent_order", "order_archive_"); err != nil {
          						return ErrDatabase.Wrap(err)
          					}
          					if err := db.migrateToDB(ctx, PieceExpirationDBName, "piece_expirations"); err != nil {
          						return ErrDatabase.Wrap(err)
          					}
          					if err := db.migrateToDB(ctx, PieceInfoDBName, "pieceinfo_"); err != nil {
          						return ErrDatabase.Wrap(err)
          					}
          					if err := db.migrateToDB(ctx, PieceSpaceUsedDBName, "piece_space_used"); err != nil {
          						return ErrDatabase.Wrap(err)

This file has been truncated. show original

We split the sqlite DB into multiple sqlite files. That way we can lock them individualy. That will allow us to scale better on bigger storage nodes.