i tried jfs only hobbyishly and on a not very performant hdd.
what i noticed primarily was a lot of heads movement between the journal zone and the data zone, so i decided that for that industrial automation project i was working on back then, i will skip it.
I donât use memtable or other gimmicks, just hashstore with default config, so the RAM is not an issue. I used to run 2 piecestore 5TB nodes on a NAS with only 1GB RAM, and it performed like the other ones with 18GB RAM. Yes, the system was laggish, but I didnât use it for anything else.
My internet is a FTTH, with like 500 Mbps upload.
There are a bunch of âhiddenâ settings regarding hashstore. Others played with them. Donât ask me more. Also, I read about primo cash, ram drives, all sorts of tunnings; in my view, for Storj they are a waste of time. It is built to run on your spare capacity in systems tunned for daily stuff, not tuned for Storj. I know I use dedicated hardware, but all my systems are âtunnedâ for usual stuff. The most important tunning you can do is to prevent sudden power loss.
if only for storj then probably yes, but i tuned my system for my rutinal tasks and storj just happened to side-benefit quite noticeably.
what concerns particuarly storj, basically the only improvement i made was the switch to hashstore and avoided lots of hdd heads movement that i had with the lazy-crazy piecestore walker.
heh.
i have an ups, but when power goes off for the block, so do the switches that connect me to the net, and the ups remains more about graceful shutdowns and not much of uptime keeping for net services. (wanted to put an emoticon but couldnt decide which one)
and btw, your egress/tbs ratio is also above the average that copilot told me.
not as much as mine, but still.
so it looks like hashstore does its job well enough.
I run 18 nodes. I didnât see any difference between piecestore and hashstore, maybe a worse performance for hashstore. But I compared the 2 and migrated all to hashstore last year, just to get one more task out of my mind. Maybe is better now, with all the impruvements made by devs. But yes, 2 points, at least, are better now: we got rid of the startup walker and the migration to another drive is way faster.
gets interesting.
worse by what parameters/metric?
If I remember corectly, stored data from month to month was leass than piecestore. I have similar setups in all locations and I can compare them pretty reliable.
So around 200GB on around 1TB stored, so a ratio of egress to storage of around 0.2
Four nodes are on the same IP. Their ratios are 0.05, 0.11, .09, 0.05
Three nodes on separate IPs, they are 0.21, 0.21, 0.14.
All are in California.
I noticed last month that a few of my nodes has a big spike on a single day of 100GB+ traffic. Just one day. Just some nodes. Maybe you had that spike. And on a smaller node that spike would be more significant for your ratio.
now copilot says 5-15% is the average, not the numbers it told me when i started the topic.
sorta hehe.
new stats published since then?
havent had spikes that big in may; my oscillations were more steady, varying 5-20gb/day.
yes, some time ago i also observed discrepancies in data usage.
particularly in my case, the actual file size sum on the filesystem was way bigger than the reported storage usage in the nodeâs web interface.
i attribute that to the gaps in storage blobs due to deleted data, and more recently the difference isnt that much of a concern as it was back then.
btw, are those blobs sparse files or not?
I donât think hashstore data logs ever have data deleted out of them: itâs only ever appended. But hashstore tracks what percentage of each log could be deleted⌠and once itâs high enough the file is compacted. Basically a fresh file is written sequentially with only the active data in it⌠then the entire old log is just deleted.
If I have that wrong, somebody will correct me 
hehe.
and if you are right, could someone point how much free space should be kept on the filesystem for these compaction operations?
Doesnât every node stop uploads when it detects only 5GB free? If so, and hashstore data logs are 1GB each⌠then it should always have enough space to compact logs.
You are correct. The only way to delete data from your hdd is a compaction which means writing alive data from existing log file to a new one and deleting the old one. How often this happens depends on STORJ_HASHSTORE_COMPACTION_ALIVE_FRACTION setting. The default is 0.25 while I am using 0.6. My stats showing 3.5 percent of reclaimable space on average for all nodes.
great.
where can i look for that for my node?
1GB x STORJ_HASHSTORE_COMPACTION_REWRITE_MULTIPLE. Default is 10.
so it wont crash or hang.
so far so good.
I am using debug endpoint.
where can i rtfm about it?