Delete orders files

Node Operators troubleshooting
1

My node crashed and would not start.
I tried removing the db files but the node got stuck on the orders db.
Finally, I deleted the files in orders and the dbs and then the node started.
What is going on there?
1.56.4 docker

2

Sounds like a database corruption. You may try to fix your databases https://support.storj.io/hc/en-us/articles/360029309111-How-to-fix-a-database-disk-image-is-malformed-

3

Without any logs or other info provided, you’re asking us to guess?

4

There is no orders db. Orders stored in flat text files. Append only. 2 hours later the hole batch gets submitted to the satellite.

These flat text files contain paychecks. Deleting them isn’t a big deal but if you have to do that for what ever reason maybe try to delete just the one that is corrupted and submitt the remaining order batches.

5

The first time I tried to start the node I got this

Error: Error starting master database on storagenode: database: piece_spaced_used opening file "config/storage/piece_spaced_used.db" failed: unable to open database file: no such file or directory

then later after deleting *.db

Error: Error creating tables for master database on storagenode: migrate: database: storage_usage opening file "config/storage/storage_usage.db" failed: unable to open database file: no such file or directory

Then I deleted orders// and *db and then it works

I suppose it is coincidence

6

That first one sounds more like a rights issue to me. I don’t think it’s wise to just delete all db’s. It can survive, but you delete all stats. As for the orders, you won’t get paid for the ones you deleted. But I guess that isn’t that big a deal as it’s at most about 2 hours worth. But yeah, next time take a more subtle approach. In the mean time just to be sure, run a file system check to make sure there is no further corruption. And make sure you don’t have sudden power loss events etc.

7

I finally got to the bottom of this…
I had re-used a disk that I used for chia. That disk I formatted with largefile4. On running storagenode on it, it was running out of inodes.
Deleting orders was just freeing a load of inodes.
So if a disk is out of inodes storagenode gives an incorrect error about missing db files

2022-08-24 04:01:28,004 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Error: Error starting master database on storagenode: database: piece_expiration opening file "config/storage/piece_expiration.db" failed: unable to open database file: no such file or directory
	storj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:324
	storj.io/storj/storagenode/storagenodedb.(*DB).openExistingDatabase:306
	storj.io/storj/storagenode/storagenodedb.(*DB).openDatabases:281
	storj.io/storj/storagenode/storagenodedb.OpenExisting:248
	main.cmdRun:193
	storj.io/private/process.cleanup.func1.4:372
	storj.io/private/process.cleanup.func1:390
	github.com/spf13/cobra.(*Command).execute:852
	github.com/spf13/cobra.(*Command).ExecuteC:960
	github.com/spf13/cobra.(*Command).Execute:897
	storj.io/private/process.ExecWithCustomConfigAndLogger:93
	main.main:479
	runtime.main:255
2022-08-24 04:01:28,265 INFO exited: storagenode (exit status 1; not expected)
de here

The database files are fine but I suppose the shm and wal can’t be creaetd

8

There is

-rw-r–r-- 1 root root 283758592 Aug 22 22:27 /data/data1/storage/orders.db
-rw-r–r-- 1 root root 32768 Aug 24 06:37 /data/data1/storage/orders.db-shm
-rw-r–r-- 1 root root 0 Aug 24 06:37 /data/data1/storage/orders.db-wal