"satellites" table missing on the database after unexpected shutdown

FoRTu · August 3, 2020, 4:04pm

I run a node using a docker container on a GNU/Linux system and after a unexpected shutdown I get this error:

2020-08-03T15:41:28.019Z INFO Configuration loaded {“Location”: “/app/config/config.yaml”}
2020-08-03T15:41:28.037Z INFO Operator email {“Address”: “XXXX@XXXX.XXX”}
2020-08-03T15:41:28.037Z INFO Operator wallet {“Address”: “0x98--------------------------------”}
2020-08-03T15:41:30.631Z INFO Telemetry enabled
2020-08-03T15:41:30.645Z INFO db.migration.25 Remove address from satellites table
Error: Error creating tables for master database on storagenode: migrate: no such table: satellites
storj.io/storj/private/migrate.SQL.Run:252
storj.io/storj/private/migrate.(*Migration).Run.func1:171
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
main.cmdRun:180
storj.io/private/process.cleanup.func1.4:359
storj.io/private/process.cleanup.func1:377
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:320
runtime.main:203

I think that a table of the database is missing or malformed. Could be the table be created it again? How? is it possible or I have to reset all my Node from scratch?

Thanks.

BrightSilence · August 3, 2020, 4:37pm

Please stop your node and rename the satellites.db file so you have a backup.

Then from the database path run:

sqlite3 satellites.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE satellites (
                                                node_id BLOB NOT NULL,
                                                added_at TIMESTAMP NOT NULL,
                                                status INTEGER NOT NULL,
                                                PRIMARY KEY (node_id)
                                        );
CREATE TABLE "satellite_exit_progress" (
                                                        satellite_id BLOB NOT NULL,
                                                        initiated_at TIMESTAMP,
                                                        finished_at TIMESTAMP,
                                                        starting_disk_usage INTEGER NOT NULL,
                                                        bytes_deleted INTEGER NOT NULL,
                                                        completion_receipt BLOB,
                                                        FOREIGN KEY (satellite_id) REFERENCES satellites (node_id)
                                                );

.quit

Try starting your node again after this.

FoRTu · August 3, 2020, 5:12pm

It works! but now another table is missing: “piece_expirations”

2020-08-03T17:10:01.091Z INFO db.migration.25 Remove address from satellites table
2020-08-03T17:10:01.106Z INFO db.migration.26 Add Trash column to pieceExpirationDB
Error: Error creating tables for master database on storagenode: migrate: no such table: piece_expirations
storj.io/storj/private/migrate.SQL.Run:252
storj.io/storj/private/migrate.(*Migration).Run.func1:171
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
main.cmdRun:180
storj.io/private/process.cleanup.func1.4:359
storj.io/private/process.cleanup.func1:377
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:320
runtime.main:203

Do you know how to restore it?
Thanks!

BrightSilence · August 3, 2020, 7:38pm

Rename piece_expiration.db, then:

sqlite3 piece_expiration.db

CREATE TABLE piece_expirations (
                                                satellite_id       BLOB      NOT NULL,
                                                piece_id           BLOB      NOT NULL,
                                                piece_expiration   TIMESTAMP NOT NULL, -- date when it can be deleted
                                                deletion_failed_at TIMESTAMP, trash INTEGER NOT NULL DEFAULT 0,
                                                PRIMARY KEY (satellite_id, piece_id)
                                        );
CREATE INDEX idx_piece_expirations_piece_expiration ON piece_expirations(piece_expiration);
CREATE INDEX idx_piece_expirations_deletion_failed_at ON piece_expirations(deletion_failed_at);
CREATE TABLE versions (version int, commited_at text);
CREATE INDEX idx_piece_expirations_trashed
                                                ON piece_expirations(satellite_id, trash)
                                                WHERE trash = 1;

.quit

FoRTu · August 3, 2020, 8:27pm

It works again! but now I think there is a duplicate column on a .db file:

 2020-08-03T20:24:53.559Z INFO Telemetry enabled
2020-08-03T20:24:53.563Z INFO db.migration.26 Add Trash column to pieceExpirationDB
Error: Error creating tables for master database on storagenode: migrate: duplicate column name: trash
storj.io/storj/private/migrate.SQL.Run:252
storj.io/storj/private/migrate.(*Migration).Run.func1:171
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
main.cmdRun:180
storj.io/private/process.cleanup.func1.4:359
storj.io/private/process.cleanup.func1:377
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:320
runtime.main:203

Thanks.

BrightSilence · August 3, 2020, 8:59pm

You seem to be a migration step behind. Try the same procedure again but like this.

sqlite3 piece_expiration.db

CREATE TABLE piece_expirations (
                                                satellite_id       BLOB      NOT NULL,
                                                piece_id           BLOB      NOT NULL,
                                                piece_expiration   TIMESTAMP NOT NULL, -- date when it can be deleted
                                                deletion_failed_at TIMESTAMP,
                                                PRIMARY KEY (satellite_id, piece_id)
                                        );
CREATE INDEX idx_piece_expirations_piece_expiration ON piece_expirations(piece_expiration);
CREATE INDEX idx_piece_expirations_deletion_failed_at ON piece_expirations(deletion_failed_at);
CREATE TABLE versions (version int, commited_at text);

.quit

FoRTu · August 3, 2020, 9:27pm

Now another table is missing “main.order_archive_”:

2020-08-03T21:24:52.716Z INFO Telemetry enabled
2020-08-03T21:24:52.721Z INFO db.migration.26 Add Trash column to pieceExpirationDB
2020-08-03T21:24:52.734Z INFO db.migration.27 Add index archived_at to ordersDB
Error: Error creating tables for master database on storagenode: migrate: no such table: main.order_archive_
storj.io/storj/private/migrate.SQL.Run:252
storj.io/storj/private/migrate.(*Migration).Run.func1:171
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
main.cmdRun:180
storj.io/private/process.cleanup.func1.4:359
storj.io/private/process.cleanup.func1:377
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:320
runtime.main:203

Thanks again.

BrightSilence · August 3, 2020, 10:19pm

Oof, what did you do to your databases?

Rename orders.db this time.

sqlite3 orders.db

CREATE TABLE unsent_order (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB      NOT NULL, -- serialized pb.OrderLimit
                                                order_serialized       BLOB      NOT NULL, -- serialized pb.Order
                                                order_limit_expiration TIMESTAMP NOT NULL, -- when is the deadline for sending it

                                                uplink_cert_id INTEGER NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE TABLE order_archive_ (
                                                satellite_id  BLOB NOT NULL,
                                                serial_number BLOB NOT NULL,

                                                order_limit_serialized BLOB NOT NULL,
                                                order_serialized       BLOB NOT NULL,

                                                uplink_cert_id INTEGER NOT NULL,

                                                status      INTEGER   NOT NULL,
                                                archived_at TIMESTAMP NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE UNIQUE INDEX idx_orders ON unsent_order(satellite_id, serial_number);
CREATE TABLE versions (version int, commited_at text);
CREATE INDEX idx_order_archived_at ON order_archive_(archived_at);

.quit

Since this migration step attempts to add the archive_at index, you may need to omit that index if it doesn’t work like this. Just remove that last create index and try again if that’s the case.

FoRTu · August 3, 2020, 10:49pm

Sorry, I’m here again.

You are right, something very strange has happened to my databases.

I follow you last steps (the las index wasn’t needed) but now another database lost a table: “piece_space_used” this time.

Here is the log:

2020-08-03T22:38:58.937Z INFO Telemetry enabled
2020-08-03T22:38:58.941Z INFO db.migration.27 Add index archived_at to ordersDB
2020-08-03T22:38:58.955Z INFO db.migration.28 Create notifications table
2020-08-03T22:38:58.959Z INFO db.migration.29 Migrate piece_space_used to add total column
Error: Error creating tables for master database on storagenode: migrate: no such table: piece_space_used
storj.io/storj/private/migrate.SQL.Run:252
storj.io/storj/private/migrate.(*Migration).Run.func1:171
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:170
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:293
main.cmdRun:180
storj.io/private/process.cleanup.func1.4:359
storj.io/private/process.cleanup.func1:377
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:320
runtime.main:203

I hope this will be the last time .

Thank you very much for your time and explanations.

BrightSilence · August 3, 2020, 11:19pm

Here’s my worry… db migration is now failing at nearly every step. You’re at 29. There are 43 steps…

Rename piece_spaced_used.db

sqlite3 piece_spaced_used.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE piece_space_used (
                                                total INTEGER NOT NULL DEFAULT 0,
                                                content_size INTEGER NOT NULL,
                                                satellite_id BLOB
                                        );
CREATE UNIQUE INDEX idx_piece_space_used_satellite_id ON piece_space_used(satellite_id);

.quit

You may need to remove this line if it complains about duplicate columns.

                                                total INTEGER NOT NULL DEFAULT 0,

FoRTu · August 4, 2020, 5:42am

…step 34: “no such table: reputation”

2020-08-04T05:38:02.976Z INFO Telemetry enabled
2020-08-04T05:38:02.981Z INFO db.migration.29 Migrate piece_space_used to add total column
2020-08-04T05:38:02.992Z INFO db.migration.30 Initialize piece_space_used total column to content_size
2020-08-04T05:38:02.992Z INFO db.migration.31 Remove all 0 values from piece_space_used
2020-08-04T05:38:02.996Z INFO db.migration.32 Create paystubs table and payments table
2020-08-04T05:38:02.996Z INFO db.migration.33 Remove time zone from created_at in paystubs and payments
2020-08-04T05:38:03.001Z INFO db.migration.34 Add suspended field to satellites db
Error: Error creating tables for master database on storagenode: migrate: no such table: reputation
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

We are close to 43…

Thank you again.

Alexey · August 4, 2020, 5:50am

sqlite3 reputation.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE IF NOT EXISTS "reputation" (
                                                        satellite_id BLOB NOT NULL,
                                                        uptime_success_count INTEGER NOT NULL,
                                                        uptime_total_count INTEGER NOT NULL,
                                                        uptime_reputation_alpha REAL NOT NULL,
                                                        uptime_reputation_beta REAL NOT NULL,
                                                        uptime_reputation_score REAL NOT NULL,
                                                        audit_success_count INTEGER NOT NULL,
                                                        audit_total_count INTEGER NOT NULL,
                                                        audit_reputation_alpha REAL NOT NULL,
                                                        audit_reputation_beta REAL NOT NULL,
                                                        audit_reputation_score REAL NOT NULL,
                                                        audit_unknown_reputation_alpha REAL NOT NULL,
                                                        audit_unknown_reputation_beta REAL NOT NULL,
                                                        audit_unknown_reputation_score REAL NOT NULL,
                                                        disqualified TIMESTAMP,
                                                        updated_at TIMESTAMP NOT NULL,
                                                        joined_at TIMESTAMP NOT NULL, suspended TIMESTAMP,
                                                        PRIMARY KEY (satellite_id)
                                                );
.exit

FoRTu · August 4, 2020, 5:59am

Thanks you @Alexey

One more: “duplicate column name: suspended”

2020-08-04T05:57:45.195Z INFO Telemetry enabled
2020-08-04T05:57:45.200Z INFO db.migration.34 Add suspended field to satellites db
Error: Error creating tables for master database on storagenode: migrate: duplicate column name: suspended
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

Alexey · August 4, 2020, 6:10am

Please, rename the reputation.db and start the same script but without

, suspended TIMESTAMP

FoRTu · August 4, 2020, 6:21am

Perfect next step…

Step 36: “duplicate column name: joined_at”

2020-08-04T06:19:16.282Z INFO Telemetry enabled
2020-08-04T06:19:16.287Z INFO db.migration.34 Add suspended field to satellites db
2020-08-04T06:19:16.301Z INFO db.migration.35 Create pricing table
2020-08-04T06:19:16.301Z INFO db.migration.36 Add joined_at field to satellites db
Error: Error creating tables for master database on storagenode: migrate: duplicate column name: joined_at
storj.io/storj/private/migrate.SQL.Run:274
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

Thanks.

BrightSilence · August 4, 2020, 7:19am

Rename reputation.db

sqlite3 reputation.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE "reputation" (
                                                        satellite_id BLOB NOT NULL,
                                                        uptime_success_count INTEGER NOT NULL,
                                                        uptime_total_count INTEGER NOT NULL,
                                                        uptime_reputation_alpha REAL NOT NULL,
                                                        uptime_reputation_beta REAL NOT NULL,
                                                        uptime_reputation_score REAL NOT NULL,
                                                        audit_success_count INTEGER NOT NULL,
                                                        audit_total_count INTEGER NOT NULL,
                                                        audit_reputation_alpha REAL NOT NULL,
                                                        audit_reputation_beta REAL NOT NULL,
                                                        audit_reputation_score REAL NOT NULL,
                                                        audit_unknown_reputation_alpha REAL NOT NULL,
                                                        audit_unknown_reputation_beta REAL NOT NULL,
                                                        audit_unknown_reputation_score REAL NOT NULL,
                                                        disqualified TIMESTAMP,
                                                        updated_at TIMESTAMP NOT NULL,
                                                        suspended TIMESTAMP,
                                                        joined_at TIMESTAMP NOT NULL,
                                                        PRIMARY KEY (satellite_id)
                                                );

.quit

I think you may need to remove the line

                                                        joined_at TIMESTAMP NOT NULL,

from that.

Also it says it’s adding these columns to the satellites db, but this is a modification on reputation.db going by the code. So this one had me looking in the wrong place for a bit.

FoRTu · August 4, 2020, 8:12am

Step 42 (last one?):

“no such table: used_serial_”

2020-08-04T08:04:38.396Z INFO Telemetry enabled
2020-08-04T08:04:38.403Z INFO db.migration.34 Add suspended field to satellites db
2020-08-04T08:04:38.413Z INFO db.migration.36 Add joined_at field to satellites db
2020-08-04T08:04:38.414Z INFO db.migration.37 Drop payments table as unused
2020-08-04T08:04:38.418Z INFO db.migration.38 Backfill joined_at column
2020-08-04T08:04:38.419Z INFO db.migration.39 Add unknown_audit_reputation_alpha and unknown_audit_reputation_beta fields to satellites db
2020-08-04T08:04:38.420Z INFO db.migration.40 Add unknown_audit_reputation_score field to satellites db
2020-08-04T08:04:38.421Z INFO db.migration.41 Make satellite_id foreign key in satellite_exit_progress table
2020-08-04T08:04:38.455Z INFO db.migration.42 Drop used serials table
Error: Error creating tables for master database on storagenode: migrate: no such table: used_serial_
storj.io/storj/storagenode/storagenodedb.(*DB).Migration.func10:1484
storj.io/storj/private/migrate.Func.Run:285
storj.io/storj/private/migrate.(*Migration).Run.func1:179
storj.io/storj/private/dbutil/txutil.withTxOnce:75
storj.io/storj/private/dbutil/txutil.WithTx:36
storj.io/storj/private/migrate.(*Migration).Run:178
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:295
main.cmdRun:183
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

Thank you very much.

BrightSilence · August 4, 2020, 8:17am

That one is a bit more tricky as my node already doesn’t have that table anymore. Lets hope it’ll just accept a dummy table.

Rename used_serial.db

sqlite3 used_serial.db

CREATE TABLE versions (version int, commited_at text);
CREATE TABLE used_serial_ (col int);

.quit

FoRTu · August 4, 2020, 8:36am

Here comes the big one:

2020-08-04T08:23:15.115Z INFO Telemetry enabled
2020-08-04T08:23:15.124Z INFO db.migration.42 Drop used serials table
2020-08-04T08:23:15.148Z INFO db.migration.43 Add table payments
2020-08-04T08:23:15.167Z INFO db.migration Database Version {"version": 43}
Error: Error during preflight check for storagenode databases: storage node preflight database error: pieceinfo: expected schema does not match actual:   &dbschema.Schema{
-  Tables: []*dbschema.Table{
-  s"Name: pieceinfo_\nColumns:\n\tName: deletion_failed_at\n\tType: TIMESTAMP\n\tNullable: true\n\tDefault: \"\"\n\tReference: nil\n\tName: order_limit\n\tType: BLOB\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\n\tName: piece_creation\n\tType: TIMESTAMP\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\n\tName: piece_expiration\n\tType: TIMESTAMP\n\tNullable: true\n\tDefault: \"\"\n\tReference: nil\n\tName: piece_id\n\tType: BLOB\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\n\tName: piece_size\n\tType: BIGINT\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\n\tName: satellite_id\n\tType: BLOB\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\n\tName: uplink_cert_id\n\tType: INTEGER\n\tNullable: false\n\tDefault: \"\"\n\tReference: Reference<Table: certificate, Column: cert_id, OnDelete: , OnUpdate: >\n\tName: uplink_piece_hash\n\tType: BLOB\n\tNullable: false\n\tDefault: \"\"\n\tReference: nil\nPrimaryKey: \nUniques:\n\t",
-  },
+  Tables: nil,
-  Indexes: []*dbschema.Index{
-  s`Index<Table: pieceinfo_, Name: idx_pieceinfo__expiration, Columns: piece_expiration, Unique: false, Partial: "piece_expiration IS NOT NULL">`,
-  s`Index<Table: pieceinfo_, Name: pk_pieceinfo_, Columns: satellite_id piece_id, Unique: true, Partial: "">`,
-  },
+  Indexes: nil,
  }

storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:346
main.cmdRun:193
storj.io/private/process.cleanup.func1.4:353
storj.io/private/process.cleanup.func1:371
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:88
storj.io/private/process.ExecCustomDebug:70
main.main:323
runtime.main:203

BrightSilence · August 4, 2020, 8:43am

Let’s hope this does the trick. Though I worry with such significant damage to databases, there may be other issues.

Rename pieceinfo.db

sqlite3 pieceinfo.db

CREATE TABLE pieceinfo_ (
                                                satellite_id     BLOB      NOT NULL,
                                                piece_id         BLOB      NOT NULL,
                                                piece_size       BIGINT    NOT NULL,
                                                piece_expiration TIMESTAMP,

                                                order_limit       BLOB    NOT NULL,
                                                uplink_piece_hash BLOB    NOT NULL,
                                                uplink_cert_id    INTEGER NOT NULL,

                                                deletion_failed_at TIMESTAMP,
                                                piece_creation TIMESTAMP NOT NULL,

                                                FOREIGN KEY(uplink_cert_id) REFERENCES certificate(cert_id)
                                        );
CREATE UNIQUE INDEX pk_pieceinfo_ ON pieceinfo_(satellite_id, piece_id);
CREATE INDEX idx_pieceinfo__expiration ON pieceinfo_(piece_expiration) WHERE piece_expiration IS NOT NULL;

.quit