# Hashstore rollout commencing! **URL:** \ **Category:** Announcements\ **Tags:** storagenode, hashstore\ **Created:** [August 21, 2025, 2:15pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589 "2025-08-21T14:15:19Z")\ **Posts on this page:** 20\ **Page:** 10
**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/5_5575768a8748004e209b776fc1b2916d.png) [@snorkel](https://forum.storj.io/u/snorkel)\ **Post date:** [September 4, 2025, 8:43pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/182 "2025-09-04T20:43:09Z")
So until now, I didn’t see anyone reporting better success rates or more data stored on nodes with hashstore, only suppositions that it is better… is there any proof? All I see is reports of bad success rates and less data stored. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [September 4, 2025, 8:53pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/183 "2025-09-04T20:53:30Z")
I’ve fully converted half of my nodes so far. My average download/upload rate has doubled. I don’t track success rates. Disk usage has started climbing slowly back up. ---
**Author:** ![jammerdan](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jammerdan/32/1121_2.png) [@jammerdan](https://forum.storj.io/u/jammerdan)\ **Post date:** [September 5, 2025, 1:22am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/184 "2025-09-05T01:22:08Z")
> [@snorkel](#): > > is there any proof? The proof so far seems to be the Select network where they implemented this: > [@jtolio](#): > > The practical upshot of all of this is that our Storj Select storage nodes, which have all been on one or the other form of hashstore for the better part of a year now, practically absorb all load thrown at them. However it is probably safe to assume that Select and global network have different characteristics: > [@jtolio](#): > > In the case of Storj Select, Storj engineers currently directly help manage many of these nodes. We often run these configurations much closer to full capacity than the global network seems to get, and so resource use optimization is especially valuable. > > When we looked closely at these storage nodes, we discovered that: (a) they were using 100% of their CPU and not 100% of their disk IOPS, and (b) the CPU use was operating system inode management. We were not getting the most throughput possible out of our disks! Bummer! > [@Mitsos](#): > > I’ve fully converted half of my nodes so far. My average download/upload rate has doubled. I don’t track success rates. I am not yet seeing such effects. But I have nodes that are still migrating. But I think I can say that at this point currently none has a better upload success rate than before. Maybe more pieces are thrown at them now? I don’t know. But hashstore should “absorb” that load according to what was posted. I’ll have to wait some more days till migrations have completed to restart nodes and see. ---
**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/5_5575768a8748004e209b776fc1b2916d.png) [@snorkel](https://forum.storj.io/u/snorkel)\ **Post date:** [September 5, 2025, 5:36am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/185 "2025-09-05T05:36:05Z")
Can’t compare select and global. I don’t believe tha race is the same for storing a piece. Success rate seems to be somehow strange, and maybe less representative for node performance, so let’s ignore that. All that matters is stored data at the end of the month. I will switch 2 more nodes to hashstore, that are not bottlenecked by hardware, and I will have a better comparition in my own farm. I see many sno that are switching all their nodes, instead of just a few, and don’t have a way to compare them. ---
**Author:** ![alpharabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alpharabbit/32/1129_2.png) [@alpharabbit](https://forum.storj.io/u/alpharabbit)\ **Post date:** [September 5, 2025, 12:12pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/186 "2025-09-05T12:12:25Z")
Almost all of my nodes finished migration. Performance has not changed. ---
**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/5_5575768a8748004e209b776fc1b2916d.png) [@snorkel](https://forum.storj.io/u/snorkel)\ **Post date:** [September 5, 2025, 8:44pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/188 "2025-09-05T20:44:49Z")
Regardin rollout schedule, maybe you should take the IPs into account, not to activate the migration on more than one node at a time for a particular IP. Many multinodes are on the same system, and starting 2 or more migrations at the same time maybe will make the system too busy…? Or this could be operators job, to opt-out of migration on any other node, and wait for the first one to finish… ---
**Author:** ![alpharabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alpharabbit/32/1129_2.png) [@alpharabbit](https://forum.storj.io/u/alpharabbit)\ **Post date:** [September 5, 2025, 8:51pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/189 "2025-09-05T20:51:21Z")
My ancient 6700K with 32 GB RAM did 24 migrations in parallel. No problems. ---
**Author:** ![elek](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/elek/32/9189_2.png) [@elek](https://forum.storj.io/u/elek)\ **Post date:** [September 6, 2025, 9:27am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/190 "2025-09-06T09:27:10Z")
> [@snorkel](#): > > Many multinodes are on the same system, and starting 2 or more migrations at the same time maybe will make the system too busy…? Should be fine, as the bottleneck is the disk IO. We recommend to use one storagenode instance per physical hard disk. It’s not recommended to start multiple instances on the same disk. (With dedicated disk, it’s even easier: you can use a faster disk usage calculation). We tested the migration on nodes with 36 disks (36 storagenode instances) without any problem. But true, if somebody over-provisioned disks, it’s better to closely monitor the system state. ---
**Author:** ![arrogantrabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/arrogantrabbit/32/11200_2.png) [@arrogantrabbit](https://forum.storj.io/u/arrogantrabbit)\ **Post date:** [September 6, 2025, 8:10pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/191 "2025-09-06T20:10:09Z")
> [@elek](#): > > Should be fine, as the bottleneck is the disk IO. Is it? It seems the disk IO during migration does not exceed 60 IOPs, and cpu is not bottlenecked either. It can go much faster, but does not. I assumed it’s a conscious decision? It is possible to have it go as fast as possible? ---
**Author:** ![Toyoo](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/toyoo/32/1126_2.png) [@Toyoo](https://forum.storj.io/u/Toyoo)\ **Post date:** [September 6, 2025, 10:19pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/192 "2025-09-06T22:19:06Z")
> [@arrogantrabbit](#): > > Is it? It seems the disk IO during migration does not exceed 60 IOPs, and cpu is not bottlenecked either. It can go much faster, but does not. I assumed it’s a conscious decision? > > [@arrogantrabbit](#): > > It is possible to have it go as fast as possible? > Defaults suggest going full speed though… ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [September 8, 2025, 9:17am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/193 "2025-09-08T09:17:34Z")
I’m going to setup few new nodes.. I’m lazy and I’m not going to modify meta files for hashstore.I’m counting on you to automate everything quickly 🙂 ---
**Author:** ![EasyRhino](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/easyrhino/32/1133_2.png) [@EasyRhino](https://forum.storj.io/u/EasyRhino)\ **Post date:** [September 8, 2025, 3:49pm UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/194 "2025-09-08T15:49:06Z")
That’s the sort of spirit I can get behind! (but truthfully, I also just set up a new node, and I went ahead and set it up as full hashstore just to avoid later migration). ---
**Author:** ![jammerdan](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jammerdan/32/1121_2.png) [@jammerdan](https://forum.storj.io/u/jammerdan)\ **Post date:** [September 10, 2025, 5:57am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/195 "2025-09-10T05:57:00Z")
> [@Poor Success rate on hashstore](https://forum.storj.io/t/poor-success-rate-on-hashstore/30675/13): > > I also observe successrate degradation after migration to hashstore completed. While my results are not as dramatic as ones reported by OP – there is some regression indeed and it needs to be looked at. If anything – the issue seems to be amplified: the crappier the success rate was to begin with – the more it regresses. Biggish node in Washington: PIECESTORE HASHSTORE ========== AUDIT ============== ========== AUDIT ============== Critically failed: … ---
**Author:** ![gingerbread233](https://storj.bcdn.literatehosting.com/letter_avatar/gingerbread233/32/5_5575768a8748004e209b776fc1b2916d.png) [@gingerbread233](https://forum.storj.io/u/gingerbread233)\ **Post date:** [September 10, 2025, 6:07am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/196 "2025-09-10T06:07:40Z")
Do I get this right? When all data is stored in 1GB files, basically I don’t need a special device on ZFS anymore (for the very small files)? ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [September 10, 2025, 7:09am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/197 "2025-09-10T07:09:31Z")
Sounds about right. For hashstore SSD doesn’t add too much, but RAM is. ---
**Author:** ![HGPlays](https://storj.bcdn.literatehosting.com/letter_avatar/hgplays/32/5_5575768a8748004e209b776fc1b2916d.png) [@HGPlays](https://forum.storj.io/u/HGPlays)\ **Post date:** [September 10, 2025, 7:26am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/198 "2025-09-10T07:26:37Z")
For hashstore - is more ram preferable? I just happend to get 256GB for cheap on hand for my server. Do you know if i need to do anything for nodes on windows to allow them to use more ram? So when hashstore comes out, more ram is better? ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [September 10, 2025, 7:37am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/199 "2025-09-10T07:37:53Z")
It should work with any amount of RAM, but from what I have read - yes, and not only for hashstore. It’s also useful for almost any filesystem operations, independently of the used backend. You may migrate one of the nodes to hashstore, but measure a memory usage before and after. This could shed more light on this topic. ---
**Author:** ![elek](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/elek/32/9189_2.png) [@elek](https://forum.storj.io/u/elek)\ **Post date:** [September 10, 2025, 7:50am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/200 "2025-09-10T07:50:54Z")
> [@HGPlays](#): > > So when hashstore comes out, more ram is better? More RAM is always better. If nothing else, it’s a better feeling to have it 😃 In addition what Alexey wrote (OS usually utilize RAM for read/write cache), you can directly utilize it with Hashstore. 1. Either you can use memtbl (keeping an index of the table records in memory). This would require 1.3 x TB of RAM. 2. Or, you can enable MMAP (map the hashtbl to memory) We usually use SSD for the hashtbl (metadata) and HDD for the logs (raw binary) on servers. We use memtbl only on selected instances (we don’t have 1.3x memory everywhere) ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [September 10, 2025, 8:02am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/201 "2025-09-10T08:02:26Z")
> [@elek](#): > > Either you can use memtbl (keeping an index of the table records in memory). This would require 1.3 x TB of RAM. What happens if I enable `memtbl` without having 1.3 GB of RAM per TB? Could this cause issues if someone makes a mistake? ---
**Author:** ![seanr22a](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/seanr22a/32/68_2.png) [@seanr22a](https://forum.storj.io/u/seanr22a)\ **Post date:** [September 10, 2025, 10:39am UTC](https://forum.storj.io/t/hashstore-rollout-commencing/30589/202 "2025-09-10T10:39:51Z")
I have the STORJ\_HASHSTORE\_TABLE\_DEFAULT\_KIND=memtbl. I’m not sure how STORJ\_HASHSTORE\_MEMTBL\_MMAP fits in to this. Should the MMAP variable be set to true or false for best performance with memtbl? What is the default ? [Previous page](https://forum.storj.io/t/hashstore-rollout-commencing/30589.md?page=9) [Next page](https://forum.storj.io/t/hashstore-rollout-commencing/30589.md?page=11)