Hashstore log question

Hi!

Hashstore migration has not finished for a 1,5month , hashstore folder size is slowly decreasing, I find many entries like this in the log:

	Sor 11274788: 2026-04-08T23:56:32Z	INFO	piecemigrate:chore	all enqueued for migration; will sleep before next pooling	{"process": "storagenode", "active": {"1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE": true, "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6": true, "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S": true, "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs": true}, "interval": "10m0s"}
	Sor 11281101: 2026-04-09T00:04:43Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "DLTVQI3KOFM6ZNFHEV7SUOIRTODICM6C25E2DOXTWVAUJAJ7QVOQ"}
	Sor 11281799: 2026-04-09T00:05:36Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "DW2JFWYZNLQLPWC7CC2TZR6L65RSONMT73O22CP4OSJ7UQXE3VYQ"}
	Sor 11288208: 2026-04-09T00:11:04Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=d8f1853bc3ec9797639ded881de3b018adc48fce990aed0c857936eb6298abf7,destination=1edc350226c36a9cc6f8f7264ae10d4c8782375152140c18fbb898214fac58f1", "errorVerbose": "hash mismatch: source=d8f1853bc3ec9797639ded881de3b018adc48fce990aed0c857936eb6298abf7,destination=1edc350226c36a9cc6f8f7264ae10d4c8782375152140c18fbb898214fac58f1\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "GEBSFHX2U4Z27DNCWLWE2OE77M63ED6A5PTX5DQ2PHGV3FR6KR4Q"}
	Sor 11291731: 2026-04-09T00:15:12Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=204671cba659244cd1ac4e5ea646ee01593db180080f05ce3c3e3e00a71b2d14,destination=e50318766377ebfe39fd4746d310aaab2df0da1734b762e48bc41c30811cefb9", "errorVerbose": "hash mismatch: source=204671cba659244cd1ac4e5ea646ee01593db180080f05ce3c3e3e00a71b2d14,destination=e50318766377ebfe39fd4746d310aaab2df0da1734b762e48bc41c30811cefb9\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "H743AMQPVB2FAYRWMWSALCOOKQWY34MI47ZCJBMECNQW7YVIREGA"}
	Sor 11293722: 2026-04-09T00:17:26Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "I2NRU6WL3RPJXS63OSDSMKCW5OBG4AKFPJNIFZ4D6YFKSJGF5UBA"}
	Sor 11303136: 2026-04-09T00:26:38Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "MMYOGOY5J46Y76DBCKB7XHLSFGVPO2AMIEUFSS7Z5LC7GCRPBW5Q"}
	Sor 11307838: 2026-04-09T00:31:35Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "OKFQSNARKCGXRR2CWGABGQULVPU352DHZXCFTT6J7VNMWDDHDDGA"}
	Sor 11312696: 2026-04-09T00:36:23Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "QJ4EHD2IAF56WT7LLMCOMQJPA6PC2T75HIDHOV6ELHLCLS7TVBOA"}
	Sor 11329099: 2026-04-09T00:45:34Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "URWW6RMZV2HHAENLSI6C6RLL73P3DRU6N3MBFAI3ZYU525PXUZKA"}
	Sor 11334946: 2026-04-09T00:50:03Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=ff1c497519e24371d5188c4b2e4a00fbf4788b742748714a2c0a8d518288d319,destination=180fffbb84a89c9b4ac9fbe4ef83f06e98541818c809d2f781c68cf6835098d6", "errorVerbose": "hash mismatch: source=ff1c497519e24371d5188c4b2e4a00fbf4788b742748714a2c0a8d518288d319,destination=180fffbb84a89c9b4ac9fbe4ef83f06e98541818c809d2f781c68cf6835098d6\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "WX6PN76VZPZ3T7KMLJSKHCMCIZZUIQCGSE3DRFCRBX46H2S2CZIA"}
	Sor 11336482: 2026-04-09T00:51:14Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "XMESLOI6VQPWUFQJHUFBJWQGC73PP2N7LWLMLYVC7YCKD2PEGNFA"}
	Sor 11340849: 2026-04-09T00:55:12Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=876e15de882adcd1782d606da76f659e5976318070c168c6a3e573538479601d,destination=6a415110ee4c2f2ff04cb670502c315943af2888e41abe77d268bbe07c55c1d4", "errorVerbose": "hash mismatch: source=876e15de882adcd1782d606da76f659e5976318070c168c6a3e573538479601d,destination=6a415110ee4c2f2ff04cb670502c315943af2888e41abe77d268bbe07c55c1d4\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "ZQ3ZTT3LFUDENSHUEU6V265GXOUN3MJWHPZHPMJ7ZGCIRO3DOKJQ"}
	Sor 11343403: 2026-04-09T00:57:48Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=70019a0be2b189fdedcc4b082b5177ef9d0381c3b7ec5264736b5b0ad340ba74,destination=781da68ca6d1a4822d85c1895b1db50ddac9e9ab68fa7c504984a457e1a0ab5f", "errorVerbose": "hash mismatch: source=70019a0be2b189fdedcc4b082b5177ef9d0381c3b7ec5264736b5b0ad340ba74,destination=781da68ca6d1a4822d85c1895b1db50ddac9e9ab68fa7c504984a457e1a0ab5f\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "2T5AHCCI7TEWGSIZHR2535GE36Q56TRSMJSOR7UEKVL4KBX27ADQ"}
	Sor 11350306: 2026-04-09T01:04:33Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "id": "4WCDS4MFR4QDCU5V2C5NQZOS3JXI2PXHNNBQIHZ742KQNOWFQ6UA"}
	Sor 11358788: 2026-04-09T01:13:12Z	INFO	piecemigrate:chore	enqueued for migration	{"process": "storagenode", "sat": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Sor 11359790: 2026-04-09T01:14:15Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "DPPEWRRGWMXTL66ZDBBW67HFIGTGLZTI32W6GTUZ5Z7YXCYFWGFQ"}
	Sor 11359857: 2026-04-09T01:14:20Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=1c71fc0cdc62524856e166216205137b282092a69cd729192af8eb4cb709fd78,destination=536c8be172b88cd5808190b931eaee062b4059f1be6491f34dcbbd7403b3c1d3", "errorVerbose": "hash mismatch: source=1c71fc0cdc62524856e166216205137b282092a69cd729192af8eb4cb709fd78,destination=536c8be172b88cd5808190b931eaee062b4059f1be6491f34dcbbd7403b3c1d3\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "DZR4BNZTOWSW4X5SMZNMNYGVCEQDJFT6ELLEIJWNQNYIH5KAKSSA"}
	Sor 11360309: 2026-04-09T01:14:50Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=1256fc8901608d94300a3535d3339bc57107f26ba75498ab7009d2fd9800765a,destination=05aa3abc51733f9c5d6b9950748017761e1921c76e346b9c556de2a189830785", "errorVerbose": "hash mismatch: source=1256fc8901608d94300a3535d3339bc57107f26ba75498ab7009d2fd9800765a,destination=05aa3abc51733f9c5d6b9950748017761e1921c76e346b9c556de2a189830785\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "FCNDLTSL72HUOGDU2XRFMZ7ELQWJBNU5NHL22233JW65PHEFLBAQ"}
	Sor 11360545: 2026-04-09T01:15:05Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "F2JR635O23MHH6LDJVFQVUAG7ELJNXJEQJJN274I7D3MAFHODKGQ"}
	Sor 11360884: 2026-04-09T01:15:29Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=67a13a169c750c356cf0e0ba6f3cde7fb9f6733f6f3d20193f3c5afe229802a8,destination=0efd2e425d049b31873534010f11897f33768e3cfbc294cf1df5ba94aea3ed36", "errorVerbose": "hash mismatch: source=67a13a169c750c356cf0e0ba6f3cde7fb9f6733f6f3d20193f3c5afe229802a8,destination=0efd2e425d049b31873534010f11897f33768e3cfbc294cf1df5ba94aea3ed36\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "HGP3AODXNKKPNYTDNIBFX2EYXO4R6HG44JO7VAX7ULOIZSMNHCGQ"}
	Sor 11361281: 2026-04-09T01:15:56Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=05f12a859327977c31185faf50d004d5f01ae6ae0d8e71573057023df424b325,destination=2d5d7ed6fcf1299817e893e94c7ec11bcdd75f07370078e85eeee86535b6e64c", "errorVerbose": "hash mismatch: source=05f12a859327977c31185faf50d004d5f01ae6ae0d8e71573057023df424b325,destination=2d5d7ed6fcf1299817e893e94c7ec11bcdd75f07370078e85eeee86535b6e64c\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "I326WON2VU24LER6O4JTDU7BGVNYC6PCMKPSUEBPOQDAIKIKFV4Q"}
	Sor 11361600: 2026-04-09T01:16:18Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "KAIE3CX3UAF26PSOLE2JAJEQUZTMAZVRX7XZ2NNRS66QQX6TVB6A"}
	Sor 11361724: 2026-04-09T01:16:28Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "KIO6NQZHB56JDCZPUCYIP7D67ROPZWJ7NGBDNHXZXRQ5NQCDHZDA"}
	Sor 11362081: 2026-04-09T01:16:51Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "LXJAV2KXTHKJBDJJZBRZVBLHQ42LZMU4VSNTBMFTBAHEBTN6ZKSA"}
	Sor 11362382: 2026-04-09T01:17:12Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "NK55ALJUH2LJMVBO5JBUXAR74JJQGZ35K53U5YY6OPFD4KIQKUVQ"}
	Sor 11362847: 2026-04-09T01:17:42Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "PLG6BAVOBZCYHDWIT3GY5SG6TXUHHDFJ44MX24BRZHAQHJJM264Q"}
	Sor 11364360: 2026-04-09T01:19:16Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "U6UNGZASTBHSXALJNE27VWFYOYECSW5RBMVUB3OZUMJOGZO4434A"}
	Sor 11364846: 2026-04-09T01:19:51Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "XH4MOAPXUWWH6D3SKCV7CSSACKSFHSGLOQSXUMCZZTC6WRQ4JLPA"}
	Sor 11366092: 2026-04-09T01:21:07Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "3RNYLNTYQNOHBBTRPEOOTBLKCRB35XR2T4TF4AYEG2YBWGSINWHQ"}
	Sor 11366630: 2026-04-09T01:21:30Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)", "errorVerbose": "opening the old reader: pieces error: invalid piece file for storage format version 1: too small for header (0 < 512)\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:335\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "id": "4USB65642LKTBCPDEME6FSQEAWPIH5MGC6ZIH4XPYLUEDFH4XQGQ"}
	Sor 11367923: 2026-04-09T01:22:45Z	INFO	piecemigrate:chore	enqueued for migration	{"process": "storagenode", "sat": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Sor 11606022: 2026-04-09T06:25:56Z	INFO	piecemigrate:chore	enqueued for migration	{"process": "storagenode", "sat": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Sor 11606218: 2026-04-09T06:26:12Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=d0e634cc42f7e22ca6f12f6b41e13900e9ef6d151c14ce15ccded3bf721150e5,destination=251f89d1eb374924f47abe428c0f2ec2d5e60f479807251c03152bb538e55a01", "errorVerbose": "hash mismatch: source=d0e634cc42f7e22ca6f12f6b41e13900e9ef6d151c14ce15ccded3bf721150e5,destination=251f89d1eb374924f47abe428c0f2ec2d5e60f479807251c03152bb538e55a01\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "id": "DCMASKFPA7KAGO7HO7BX3NOENFVYRK2DXGFG5IXXMFLA4RDJ5DSQ"}
	Sor 11608185: 2026-04-09T06:27:53Z	INFO	piecemigrate:chore	enqueued for migration	{"process": "storagenode", "sat": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Sor 11608186: 2026-04-09T06:27:53Z	INFO	piecemigrate:chore	all enqueued for migration; will sleep before next pooling	{"process": "storagenode", "active": {"12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S": true, "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs": true, "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE": true, "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6": true}, "interval": "10m0s"}
	Sor 11608189: 2026-04-09T06:27:54Z	INFO	piecemigrate:chore	couldn't migrate	{"process": "storagenode", "error": "hash mismatch: source=d0e634cc42f7e22ca6f12f6b41e13900e9ef6d151c14ce15ccded3bf721150e5,destination=251f89d1eb374924f47abe428c0f2ec2d5e60f479807251c03152bb538e55a01", "errorVerbose": "hash mismatch: source=d0e634cc42f7e22ca6f12f6b41e13900e9ef6d151c14ce15ccded3bf721150e5,destination=251f89d1eb374924f47abe428c0f2ec2d5e60f479807251c03152bb538e55a01\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).copyPiece:403\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).migrateOne:359\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).processQueue:277\n\tstorj.io/storj/storagenode/piecemigrate.(*Chore).Run.func2:184\n\tstorj.io/common/errs2.(*Group).Go.func1:23", "sat": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "id": "DCMASKFPA7KAGO7HO7BX3NOENFVYRK2DXGFG5IXXMFLA4RDJ5DSQ"}
	Sor 11608192: 2026-04-09T06:27:54Z	INFO	piecemigrate:chore	enqueued for migration	{"process": "storagenode", "sat": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}

as if something was in a loop…

It seems like your migration is finished. Having a number of corrupted files left is quite normal. Those files can not be migrated but the loop will try again and again if you don’t stop migration or delete files manually.

1 Like

However, disk 2, which has this node, is still 100% loaded. If it had already finished, it wouldn’t be so loaded, right? (disk2 is dark blue)

How many files are left in blobs folder? I have seen a small number of damaged files left for most of my 100+ nodes when migration was finished.

I think I found something interesting:

I ran find -type f -size +0 command on the blobs folder and it found about 20 files that it couldn’t read.

I searched for these files and none of them have an owner or group, but I can’t modify them as sudo/root.

Probably a few years ago when I switched from btrfs to ext4 on a new hard drive, something happened to these files during fsyc.

I’m going to rename these folders to original_name_bad now and create an original empty folder instead.

This might be causing the endless trial and error that I’m seeing with the disk usage.

It is safe to delete anything within blobs folder when migration is finished. Only the empty blobs folder is still required.

1 Like

Update: I detached the volume belonging to the node and ran an fsck, after which the permissions on the corrupted files were restored. After this, the active migration was completed and the disk load was eliminated. Thanks for the help!

1 Like