Weird that i was running so flawless (no errors in logs etc), till i forced this migration and EVERYTHING was, apparently corrupted! I feel silly for not just leaving it alone!
Anyway, now that this migration is in progress, how do i do a proper graceful exit so i can start over…or just quit? If i remove the “true” from the hashstore
What difference would that make? Data was lost. Node was disqualified. This is a correct outcome.
It’s best to learn about this sooner rather than later and fix the underlying issue. So I’m not sure why are you wishing for staying in the dark for longer.
Delete the node, start over. There is no point in wasting another month on graceful exit on remaining satellites.
The key is to find and fix root cause of data loss
Only 4% or more is enough to be disqualified.
I would run memtest at this point. EVERYTHING cannot be corrupted for no reason. Also check logs for crc failures communicating with drives.
I wonder how hashstore migration could even matter for audit? I mean if migration fails for a specific piece it should still exist in original place? 
Yep, that would be my guess here too. The scenario would go as follows: The pieces might have been good on disk and being sent correctly in a scenario of low memory use. A mass piece migration probably started using your RAM to the fullest, even if just as cache, and started hitting some bad places in transit—leading to wrong data being written to hashstore.
Had exactly this happening to me long time ago—a large data copy on a machine with faulty RAM led to corruption of a lot of files. Curiously, the same computer seemed to operate correctly for years. RAM errors are sneaky.
Yes, it will remain in the piecestore, but it won’t appear in the hashstore. When auditing, the node will first check the hashstore, then the piecestore.
However, I can assume that @miicar204 could have deleted all the remaining pieces as instructed, so now it’s simply missing from any storage.
My node2 completed the migration two weeks ago. However, the values are not real. The real utilization: 12 TB of total storage, of which 9.2 TB is occupied. The system shows a lot of overload. I waited two weeks, hoping that some task would fix it, but today I stopped the container, renamed it to used_space_per_prefix.db, but it didn’t change even by evening. There is not much traffic due to the overallocation. Do I need to enable some file mover? Thanks.
I found the error, I messed it up. I forgot that the dbs were on another ssd volume, I left them in the old location, so I got confused… 
no, i didn’t do anything but trigger the active migration, and it immediately went psyco on me! I’ll lick my wounds and start over. I did try canceling the migration yesterday, and it just made things worse where the node wouldn’t even start. So I’m about to blow it all away and build a better mousetrap…lost the 5 bux that was held (i’ve been off probation for a couple years now). Learning hurts sometimes!
Any new node will start with the hashstore already in place??
I hope thats not the case…but maybe its time to do a memtest! The machine has 300+ gb of ECC ram, and i did set up the node to use more of it, in hopes the migrations would be faster. Maybe that unearthed some shenanigans in the RAM! Thanks for the warning!
Yes, but it’s a passive migration, i.e. all new pieces will go to hashstore, all requested pieces will be moved to hashstore.
I have another question
On February 21, I began the full hashstore migration on my last 10.5 TB node.
Every few days, I check the size of the hashstore folder using WinSCP and compare it to the value displayed by the web GUI to estimate when it will finish.
The web GUI currently shows the following:
Used: 8823 GB
Used Trash: 583 GB
Used, recoverable: 672 GB
On March 24, according to WinSCP, the hashstore folder was:
9713 files, 2070 folders, 9427 GB in size.
On April 4:
9,969 files, 2,070 folders, 9,386 GB in size.
Over the past few weeks, I’ve seen a daily increase of 100–200 GB, but now the size is decreasing. Is this normal behavior, or do I need to fix something? ThankYou
I think so. We have seen big deletes end of march and now its going to be finally erased from disk by compaction.
Thanks. It seemed to be very low performance over two weeks, only 41GB and ~250 file changes. Would disabling badger cache improve anything?
can i still use these in config.yaml after migration?
filestore.write-buffer-size: 4.0 MiB
pieces.write-prealloc-size: 4.0 MiB
or is it not necessary?
Badger cache is not used for hashstore AFAIK.
Its amazing to see how fast the hashstore is on a UGreenDXP4800+
Overview
2026-4-3 reclaimed: 14.80 GiB | In Trash: 1.70 GiB | deleted: 1.30 GiB | time: 00:18:09 h:m:s
2026-4-4 reclaimed: 428.78 GiB | In Trash: 0.01 GiB | deleted: 77.49 GiB | time: 21:42:05
2026-4-5 reclaimed: 1086.57 GiB | In Trash: 173.70 GiB | deleted: 3.53 GiB | time: 10:02:52
2026-4-6 reclaimed: 799.80 GiB | In Trash: 2.02 GiB | deleted: 240.37 GiB | time: 01:38:04
2026-4-7 reclaimed: 768.50 GiB | In Trash: 25.21 GiB | deleted: 191.14 GiB | time: 01:14:31
I change to this o 4th. april
environment:
- STORJ_HASHSTORE_STORE_OPEN_FILE_CACHE=500
- STORJ_HASHSTORE_COMPACTION_ALIVE_FRACTION=0.80
cpu_shares: 4096
blkio_config:
weight: 1000
mem_limit: 48g
before there are nearly 2TB reclaimable
Ether in compaction no Download error.
1TB SSD Cache
echo “net.core.rmem_max=33554432” >> /etc/sysctl.d/udp_buffer.conf
sysctl -w net.core.wmem_max=33554432
This heavily depends on the definition of “fast”. Hashstore trades metadata work for actual data churn. If your metadata access was not accelerated then, yes, hashstore may seem like an improvement.
Just confirming again that I don’t need to do anything with my compose that’s several years old at this point. Just continue to set it and ferget it… YES?
Hello @holocronology,
Welcome back!
Basically - yes.
