got this idea, will need refinement… but base concept goes like this… post this. check below. the full graph will make it easier to spot trends over vast periods, and the daily fan will help with more accurate numbers to try and determine node performance across many different node setups, since s…

[image] Mitsos: it would still not be enough to fully saturate a decent 2Gbps connection It would still saturate I/O throughput of nodes configured in less than optimal way. We’ve got quite a lot of improvements since then, for example the hashtable-based storage. Hashtable was explicitly im…

[image] Toyoo: It would still saturate I/O throughput of nodes configured in less than optimal way. We’ve got quite a lot of improvements since then, for example the hashtable-based storage. Hashtable was explicitly implemented for the select network . Guess why? Remember the customer with suppo…

[image] Mitsos: Let’s actually see at least one disk being bottlenecked by incoming data, then we can work on ideas to add the second node on the same IP. I have seen mine. Ended up adding the SSD cache because of that.

[image] Toyoo: I have seen mine. Ended up adding the SSD cache because of that. [image] Toyoo: It would still saturate I/O throughput of nodes configured in less than optimal way. No disks here showed wait shooting up, even during the tests.

The only “load” the average node sees… is when used-space-filewalker runs after automated upgrades every couple weeks. Customer IO is basically nothing.

And GC/trash emptying. But yes, I agree 100% that customer load is basically non-existent.

[image] Mitsos: customer load is basically non-existent. That’s why we need more customers and not staking for SNOs.

[image] Toyoo: My hypothesis is that the select network also underperformed I wouldn’t say underperformed. The select network was mirroring the same issues we have seen in the public network. In parts it was entertaining to see a similar conversation internally. It took the devs a moment to u…

[image] littleskunk: the problem scaled exponentially with the amount of pieces stored on disk Why exponentially? Where did exponent come from? I’d expect linear to logarithmic scaling, not exponential, especially if slightly deeper directory hierarchy would be used to avoid millions of files…

[image] arrogantrabbit: Why exponentially? Where did exponent come from? I’d expect linear to logarithmic scaling, not exponential, especially if slightly deeper directory hierarchy would be used to avoid millions of files in a single folder (b trees size optimization). I believe the root cau…

Bandwidth utilization comparison thread

Node Operators FAQ

snorkel August 3, 2025, 6:47am 1818

The lucky pieces still on… same 2 nodes last month:

Hashstore rollout commencing!