Yeah, does this all go back to how big nodes can really get now? Would a fair guess be something like 10TB per /24, over maybe three years?
So long-term a 30TB HDD would comfortably handle three nodes?
I wish 
I dont think it will handle, as more data more bandwidth, deletes, compaction, and they all corelated in time. but IO stay same.
My rule-of-thumb is six. Like one HDD has the IO to easily handle six “busy” nodes. I’m sure it can go higher… but six means the drive still has spare read+write IOPS: it’s not stuck 100%-busy.
how much TB of data do you have there on HDD?
It’s not really related to size: but you can assume 10TB/node. A “busy” node (lots of egress) seems to average ~15 read IOPS. Writes are async: which means they go to memory (and filesystems like ZFS will batch them into sequential updates about every 5 sec): so write IOPS can almost be ignored.
And modern drives seem comfortable with (6x15=) 90 sustained read IOPs and some extra writes, and still stay responsive.
Do you run more?
My biggest node is like 10tb of data, I see data usage is low. I personally don’t like idea to have several node on one HDD, but it looks like I pay to much to operate according to ToS.
There is more going on than just the total amount of data stored and network growth. Even when the total data stored does not increase, new pieces are being created and older pieces deleted. Nearly everything works this way: backup programs, storing camera video for the past 30 days, collaborating on a media project, etc. So there is always an incentive to cheat to capture as many new pieces as possible, while old pieces are removed.
The purpose of cheating isn’t to increase capacity of the network, it’s to bleed data away from other nodes by:
- making 1 HDD appear to be many independent nodes
- using unique /24 IPs to make nodes appear independent
- using VPNs to make nodes appear to be someplace geographically desirable when they aren’t in reality and would be excluded by customers requesting a specific region
- being statistically more likely to enter races; you have to enter before you can win
The other day I uploaded a file with uplink using --upload-log-file, made a list of the 54 node IPs that won, then asked Chatgpt to classify them by country. I’m in the US Midwest, but only 3 of the winning nodes were in the US and Chatgpt said they were likely VPN addresses. Most were in Europe. That seems really broken to me.
The only suggestion I’ve seen that actually addresses cheating is:
Sounds like a good idea.
witch node will be the first, if several node on same IP this is understandable, but several node on same /24 doesn’t mean they belong to same operator.
if storj makes first 500gb free for every node, we will not need ip filtering at all.
with 30k nodes, chanses that several pieces from 54 will end up in one location is almost nothing, but repair could be /24, if audit will find several pieces in same IP.
and this will be much cheaper to operate than today.
BUT first of all we need some tests for it and accumulate data, just because think twice before jump.
Everyone is arguing to make themself most profit while storj only needs lower payment. The path to achieve this is removing all restrictions from SNOs while cutting payment on a monthly base. The more nodes the more likely pure random node selection is good enough for piece distribution.
first need some calculation how it will go. because storj use not only amount of nodes, but some kind of faster is better. and this faster node amount can be much lower, the probability to send piece to same location go much higher.
They always use choice of n on top for selecting fastest nodes. What we are discussing here is pre-selection for choice of n.
What is it “fastest nodes” ?
Where can I find a description of such a concept?
Ask AI bot of your choice to explain it from source code. Way better than anything you will get here.
We know most free-space is in Europe: as are most nodes in general. It would be broken if most of the winning IPs weren’t from there, no?
It depends on the extent of cheating. I’d agree that most IP addresses are in Europe according to the databases used to map IP addresses to locations.
Thanks for the link; I didn’t know about it.
If someone uploads something from the US, it should ideally land on US nodes, no? To provide better performance? I’m sitting here with TBs free + many more drives on standby.
I’m not sure anyone can say how much free space is available in the Storj network because as I understand it, that’s just a number that SNOs enter for the size of their drive, and often times (so I’ve read) it is duplicated when a drive is divided into multiple nodes with volume managers.
Unless Storj filled the entire drive with data and then manually released it as needed, I don’t think the node software can verify that the free disk space a node says it has is truly available.
My laptop says this:
Filesystem 1024-blocks Used Available Capacity iused ifree %iused Mounted on
/dev/disk1s5s1 244912536 15058276 83053612 16% 502388 830536120 0% /
devfs 186 186 0 100% 644 0 100% /dev
/dev/disk1s4 244912536 2098196 83053612 3% 3 830536120 0% /System/Volumes/VM
/dev/disk1s2 244912536 402432 83053612 1% 2098 830536120 0% /System/Volumes/Preboot
/dev/disk1s6 244912536 3364 83053612 1% 20 830536120 0% /System/Volumes/Update
/dev/disk1s1 244912536 143065044 83053612 64% 418317 830536120 0% /System/Volumes/Data
map auto_home 0 0 0 100% 0 0 100% /System/Volumes/Data/home
All of the partitions say they have 83GB available but you can’t write 83GB to all of them.
Edit: it just occurred to me, if free space is not accurate, how can it be used to continue justifying a decrease in SNO payouts?
They might be using a different metric - such as how many nodes are not accepting uploads (because they are full). There is this full_nodes metric on http://stats.storjshare.io/, and you can turn this into a percentage of the network for example.
SNOs are free to set this number but node software will cap it at what OS is reporting to be free.
I am starting new nodes with a 1 TB setting. If free space is going below 100 GB my monitoring software will add another 500 GB.
I guess the idea is that the wider the pieces are distributed, the less chance any type of local failure (like an ISP having an outage) could knock any data offline. And then from that broad list of possible nodes… they race and the fastest wins.
Storj was also marketing to Media and Entertainment companies with global workflows. The pitch was that data could be accessed quickly anywhere: no matter where the employee was. Parking everything in the US when you’ve got people editing videos in Romania doesn’t help you.
But I think you could always choose to geolocate if you wanted to?