Compaction Error how to fix?

Node Operators troubleshooting
,
1

Hello.

I have this compaction error how to fix it?
2026-01-14T13:51:05+02:00 ERROR hashstore compaction failed {“satellite”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “error”: “hashstore: writing into compacted log: EOF”, “errorVerbose”: “hashstore: writing into compacted log: EOF\n\tstorj.io/storj/storagenode/hashstore.(*Store).rewriteRecord:1267\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce.func7:1004\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce:1018\n\tstorj.io/storj/storagenode/hashstore.(*Store).Compact:715\n\tstorj.io/storj/storagenode/hashstore.(*DB).performPassiveCompaction:528”}

2

My best guess would be a damaged hashtable. So making a new one using write-hashtbl might be worth a try.

1 Like
3

I all the time geting this

2026-01-15T01:44:55+02:00 INFO hashstore compact once finished {satellite: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, store: s1, duration: 1m5.8668231s, completed: false, error: hashstore: writing into compacted log: EOF, errorVerbose: hashstore: writing into compacted log: EOF\n\tstorj.io/storj/storagenode/hashstore.(*Store).rewriteRecord:1267\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce.func7:1004\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce:1018\n\tstorj.io/storj/storagenode/hashstore.(*Store).Compact:715\n\tstorj.io/storj/storagenode/hashstore.(*DB).performPassiveCompaction:528}
2026-01-15T01:44:55+02:00 INFO hashstore finished compaction {satellite: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, store: s1, duration: 1m5.8668231s, error: hashstore: writing into compacted log: EOF, errorVerbose: hashstore: writing into compacted log: EOF\n\tstorj.io/storj/storagenode/hashstore.(*Store).rewriteRecord:1267\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce.func7:1004\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce:1018\n\tstorj.io/storj/storagenode/hashstore.(*Store).Compact:715\n\tstorj.io/storj/storagenode/hashstore.(*DB).performPassiveCompaction:528, stats: {NumLogs:930,LenLogs:0.9 TiB,NumLogsTTL:54,LenLogsTTL:1.3 GiB,SetPercent:0.8665437765889686,TrashPercent:0.06423300848141461,TTLPercent:0.008273894541707438,Compacting:false,Compactions:0,Today:20467,LastCompact:20467,LogsRewritten:22,DataRewritten:0.7 GiB,DataReclaimed:0 B,DataReclaimable:117.0 GiB,Table:{NumSet:2486934,LenSet:0.7 TiB,AvgSet:328133.82340826094,NumTrash:173778,LenTrash:56.3 GiB,AvgTrash:348087.33349445846,NumTTL:6889,LenTTL:7.3 GiB,AvgTTL:1131041.518943243,NumSlots:8388608,TableSize:151.8 MiB,Load:0.2964656352996826,Created:20465,Kind:1},Compaction:{Elapsed:0,Remaining:0,TotalRecords:0,ProcessedRecords:0}}}
2026-01-15T01:44:55+02:00 ERROR hashstore compaction failed {satellite: 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs, error: hashstore: writing into compacted log: EOF, errorVerbose: hashstore: writing into compacted log: EOF\n\tstorj.io/storj/storagenode/hashstore.(*Store).rewriteRecord:1267\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce.func7:1004\n\tstorj.io/storj/storagenode/hashstore.(*Store).compactOnce:1018\n\tstorj.io/storj/storagenode/hashstore.(*Store).Compact:715\n\tstorj.io/storj/storagenode/hashstore.(*DB).performPassiveCompaction:528}

is it some database error or one log file is broken?
why it try to write to compacted log?

4

The compaction happening from the old log to a new one. So, it looks like the new one is broken.
I shared with the team, so I would like to suggest to wait for an answer.

1 Like
5

I founding this error on several nodes, some of this entries are very old.