# And a bit of my observations report as a sno and not as a user

**URL:** <https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823>\
**Category:** Node Operators\
**Tags:** hashstore\
**Created:** [May 31, 2026, 4:54am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823 "2026-05-31T04:54:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 4:54am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/1 "2026-05-31T04:54:18Z")

</div>

from what copilot told me, it turns out that my node has egress of around 2.5-3x per tb stored compared to the overall average.  
and i have only 100mbps net link for the node and on a not-very-fast nas-oriented mechanical hdd not ssd!  
i was a bit of impressed so i tried to remind myself what i did to have this.

so the biggest contribution to this is using hashstore with its quick in-ram lookups instead of the old piecestore, or at least it is so that i think.  
of course there are also a lot of small optimisations like having a 10g quick in-ram hdd cache with primocache, and many storage tweaks like using ReFS on a storage spaces -formatted volume and not on a raw partition, and some sata-related windows tweaks that i myself already dont remember exactly.  
but i still think the biggest contribution is from the hashstore.

storj node is in a vm with dynamic memory that currently uses 4784mb ram for the whole vm, with around 430mb going to storagenode.exe  
all this for 1.85tb currently stored.

from other observations i can tell that storagenode can have a very delayed start if the computer was abruptly turned off after a boo-boo of a thermal shutdown which i may sometimes have when experimenting with intense tasks in other virtual machines.  
this is probably because it checks the integrity of the store, correct?

there were no other hiccups with hashstore, at least not what i may have noticed.

so hashstore doesnt eat very much ram for its lookup tables, at least not as much as i expected it would.  
but the performance gain is really noticeable.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [May 31, 2026, 6:20am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/2 "2026-05-31T06:20:05Z")

</div>

ReFS can be a problem: #refs  
Why are you using a virtual machine, especially one running Windows? It’s a waste of resources, and optimizing the file system on the host won’t help the OS in the virtual machine, because the virtual machine doesn’t have access to the host. The host’s filesystem manager also doesn’t have access to services in the VM. They are independent.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 8:06am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/3 "2026-05-31T08:06:25Z")

</div>

> [@Alexey](#):
>
> Why are you using a virtual machine

first of all because i prefer to have stuff separately for one isp link and for another, or else i have to do very complicated routing rules.  
second, it eases the inspection of things when there are surprise spikes in activity.

and refs does not seem to be a problem anymore.  
although it was a very big problem when i enabled defer-writes in primocache and lost the volume data several times after sudden power-offs.  
not using defer-writes anymore and all is okay.  
a bit of a pity, but c’est la vie.

> [@Alexey](#):
>
> optimizing the file system on the host won’t help the OS in the virtual machine

to sum up, i am already quite satisfied with the 2.5-3x egress compared to the overall average node, despite my not-very-fancy hardware and this configuration.  
of course if the datastore would be local, it might be even faster.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [May 31, 2026, 8:12am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/4 "2026-05-31T08:12:12Z")

</div>

But why Windows? If you want to use VM, it’s more logical to use Linux, it at least uses less resources. However I still do not understand why not run docker directly on the host, the networking can be configured for it. Or is your host run Windows too?

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 8:14am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/5 "2026-05-31T08:14:54Z")

</div>

> [@Alexey](#):
>
> Or is your host run Windows too

host is on windows because i use GPU-P in its hyper-v.  
and the default host’s link is my newer and faster isp, but which does not permit static ipv4, so the old 100mbps link remained only to this vm.  
and linux uses less resources only if it is without a desktop environment, but i have enough resources and this is not critical for me personally.

the surprise was in the hashstore performance compared to the default piecestore and this is what i advise other snos to consider in the first place.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [May 31, 2026, 8:27am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/6 "2026-05-31T08:27:06Z")

</div>

Then it’s still a waste of resources to solve such a simple problem, especially when it comes to resources required by Windows in this virtual machine.  
Since your Windows host is Hyper-V, you can install Docker; it still uses a Linux virtual machine—either Hyper-V or WSL2. Or simply install a Linux virtual machine and install the Docker version of storagenode or a binary version with two services—one for storagenode and one for storagenode-updater. This will help free up at least 50% of the virtual machine’s resources currently being used by Windows.

> [@vv75](#):
>
> the surprise was in the hashstore performance compared to the default piecestore and this is what i advise other snos to consider in the first place.

yes, so you can imagine how faster it would be if you remove additional useless level 🙂

The solution for the networking is very simple: you need just provide the IP of your second network adapter in this config option:

```auto
server.address: 192.168.1.11:28967

```

So you can install storagenode directly on your host, no VMs, no waste of resources.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 8:33am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/7 "2026-05-31T08:33:30Z")

</div>

> [@Alexey](#):
>
> The solution for the networking is very simple

not so simple, because that cannot be solved on windows host to have the reply packets go to the same interface the requests came from.  
solvable on linux but not on a windows host.  
this is that so far my only problem with windows host, all the rest doesnt matter much and is solvable.

and if the incoming comes from the old isp link with static ipv4, and the replies go to the new isp, it drops the reply packets because they dont seem to originate from its network and is considered spoof/abuse.

so i cannot just put storagenode directly on the host, unfortunately.

the only solution to put storagenode directly on host would probably be proxifying the tcp and udp from the vm that has that static ipv4.  
i know how to do it for tcp but not for udp.  
any insight on this and is it at all possible, given that udp does not have the notion of connection?

> [@Alexey](#):
>
> yes, so you can imagine how faster it would be if you remove additional useless level 🙂

is this an issue to worth the hassle, given that even in this configuration it already performs much above average? 🙂

and if to analyze it even more coldly, what would i cut?  
just giving filesystem operations closer to storagenode.exe and eliminating the networked layer between vm and host?  
that is microseconds, not even milliseconds!  
oh do i have such a load with storagenode to worth it?  
does it make even a dent given that hardware is hdd and not ssd?

but switching from piecestore to hashstore is considerable and i advocate for it.  
the only question that remains for me is if this is what rpi/nas nodes could afford…

---

<div class="post-metadata">

**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/6_449c533f586939b6e5f246632a46421f.png) [@Roxor](https://forum.storj.io/u/Roxor)\
**Post date:** [May 31, 2026, 11:33am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/8 "2026-05-31T11:33:48Z")

</div>

I understand hashstore benefits… in theory. But I’ve also read enough posts about the hashtable/datalog corruptions and rebuilds that I’ll wait for Storj to decide when to migrate. Asia is half-done and I guess Europe will be next?

Every node release seems to have some hashstore tweaks: so I’ll wait for as many of them as possible 🙂

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 11:41am UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/9 "2026-05-31T11:41:28Z")

</div>

> [@Roxor](#):
>
> enough posts about the hashtable/datalog corruptions and rebuilds

that probably is also influenced by the underlying fs.

and i also read enough about data reported and actually used, which sometimes dont match.  
but last ive checked recently it seems close to all okay.  
well maybe for some gaps in data files from data that was deleted, but that already seems in the reasonable limits.

> [@Roxor](#):
>
> Asia is half-done and I guess Europe will be next?

well that already depends whether you have lots of data or not so much (like me) and can afford to risk losing that then waiting for it to replentish.

> [@Roxor](#):
>
> I’ll wait for as many of them as possible

i experimented with hashstore early since its launch and seen many bumps along the road.  
but now it seems all peaceful.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 12:39pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/10 "2026-05-31T12:39:50Z")

</div>

lost my node many times, but mostly not because of hashstore but because of my attempts to force-cache writes to the refs volume.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 1:25pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/11 "2026-05-31T13:25:58Z")

</div>

pretendably resilient filesystem, but when some data misses being actually written due to some hardware glitch (or VRM overheat and thermal shutdown as in my case) it is poof the entire volume all gone!

---

<div class="post-metadata">

**Author:** ![jammerdan](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jammerdan/32/1121_2.png) [@jammerdan](https://forum.storj.io/u/jammerdan)\
**Post date:** [May 31, 2026, 1:57pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/12 "2026-05-31T13:57:47Z")

</div>

> [@vv75](#):
>
> it is poof the entire volume all gone!

Not exactly resilient, right?

ReFS is terrible at least that was my experience:

> [@Big Windows Node file system latency and 100% activity (NTFS vs ReFS)](https://forum.storj.io/t/big-windows-node-file-system-latency-and-100-activity-ntfs-vs-refs/8994/8):
>
> crazy_face ReFS is a typical Microsoft product. They started it with lots of enthusiasm but didn’t think it to the end and finally built a useless product. It happened to me too: It was running like a charm at the beginning and I could not believe the warnings I had read on the internet. Then one day BANG HDD went into RAW mode shortly after I had consolidated Terabytes onto it and retired the old drives. Where there are ways to recover a raw disk with Ntfs with ReFS there was no easy solut…

Since then I can live without it.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 2:06pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/13 "2026-05-31T14:06:39Z")

</div>

> [@jammerdan](#):
>
> Not exactly resilient, right?

the most resilient fs in my experience was HPFS on os/2.  
btw, it was also made by a microsoft employee (named Gordon Letwin) back in times of os/2 1.x when msft and ibm still worked together on that.  
(wink to Alexey: correct?) 🙂

> [@jammerdan](#):
>
> Since then I can live without it.

of course there may be life without it. 🙂  
in my case, all seems normal after i renounced to defer-write with primocache.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 2:34pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/14 "2026-05-31T14:34:25Z")

</div>

btw, what i personally gain from refs is the deduplication.  
ntfs dedupe works miserably and very very slowly so then i decided to try refs dedupe.  
thats not for storj, but for my vm vhdx files given that i have many similar vms on the host.  
it successfully deduped over a terabyte.

of course storjdata folder is excluded from dedupe and also several other ones.

but vms live on an ssd (also refs formatted), hdd is used only for temporary vm data export prior to archiving and uploading onto storj.  
decided that this job will go to the hdd because i want to avoid ssd write wearing; i backup vms quite often.

will observe for some time how this config behaves and will draw conclusions after that.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 2:43pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/15 "2026-05-31T14:43:32Z")

</div>

most likely is that i will migrate all dedupable stuff to the ssd and if refs will belly-up on hdd again, i will reformat the hdd to ntfs.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 3:01pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/16 "2026-05-31T15:01:23Z")

</div>

of course life would be easy if there would be only storj on the hdd, but here we are, there are some archives on it too, including some system recovery things that i might need even when i dont have access to cloud storage so i must keep this locally accessible at all times.

---

<div class="post-metadata">

**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/6_449c533f586939b6e5f246632a46421f.png) [@Roxor](https://forum.storj.io/u/Roxor)\
**Post date:** [May 31, 2026, 3:16pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/17 "2026-05-31T15:16:27Z")

</div>

With the improvements in hashstore… most of the network will probably just slap their nodes on ext4… and then ignore it and go live their lives. The slowest part is normally a persons Internet connection: the storage setup doesn’t need to be fancy.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 3:20pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/18 "2026-05-31T15:20:45Z")

</div>

> [@Roxor](#):
>
> the storage setup doesn’t need to be fancy

i dunno for others, but in my particular case the lazy-crazy piecestore walker made hdd heads move almost all the time.  
of course no such artifact with hashstore.

---

<div class="post-metadata">

**Author:** ![jammerdan](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/jammerdan/32/1121_2.png) [@jammerdan](https://forum.storj.io/u/jammerdan)\
**Post date:** [May 31, 2026, 4:07pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/19 "2026-05-31T16:07:53Z")

</div>

> [@vv75](#):
>
> what i personally gain from refs is the deduplication

A great feature. Normally. But if ReFS leaves nothing to deduplicate then it does not help much.

---

<div class="post-metadata">

**Author:** ![vv75](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/vv75/32/1135_2.png) [@vv75](https://forum.storj.io/u/vv75)\
**Post date:** [May 31, 2026, 4:09pm UTC](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823/20 "2026-05-31T16:09:40Z")

</div>

> [@jammerdan](#):
>
> But if ReFS leaves nothing to deduplicate

maybe you wanted to say storj, not refs?  
storj has nothing to dedupe, but my puter’s life ain’t only storj. 🙂

[Next page](https://forum.storj.io/t/and-a-bit-of-my-observations-report-as-a-sno-and-not-as-a-user/31823.md?page=2)
