IP Filtering, Deep Dive

apologies for asking so that I can understand this correctly the 2 nodes behind ip 73.138.168.xxx are considered as one and will share the traffic and the node ip 173.9.183.xxx will be a separate and independent node from the other 2.

Yes. If you do not have neighbors with storagenodes too :slight_smile:

thanks that’s what I wanted to know. subnet filtering is very complicated for me /24/29/33/1 confusing.

from 1 to 255 for the last part of the IPv4

1 Like

one final question is it necessary for 2 instances of watchtower if running 2 docker nodes sorry it’s out of topic

1 Like
1 Like

can someone give me a brief assessment of how the distribution of IP addresses is locally in general? Are the 254 addresses within a /24 network necessarily close together locally? So all my 254 direct neighbours get the extension 1…254. Or is there a certain spread across a wider area?

I would like to advertise Storj to my direct neighbours, but of course I don’t want to be put at a disadvantage.
Or is it Internet provider specifically distributed?

Or do I still get it wrong: Although my direct neighbors might all be in the same /24 network the IP filtering’s job is only to guarantee that not THE SAME data piece is stored there. It does not mean perse that this network gets less data assigned?

I understand that for ONE single IP-adresse, there is no advantage running mltiple node to receive more data. This is rigth and fair for everybody. But does this refer to one EXACT single IP-addresse or to that /24 IP-address-range. That would mean that 254 nodes always share 1 ammount of data? Lucky for that one guy being alone in his /24-IP-range?

I might have gotten an idea via the webpage
https://db-ip.com/as41997
There you see how many IP-adresses your privider runs in total and what range he administers

Technically when Storj uses /24 ranges they are not using actual subnets (in actual network subnets ‘.0’ is the network address and ‘.255’ is the broadcast address. This does not apply in this case. It doesn’t matter what IP address your ISP gives you, Storj looks at the ranges where each unique group is the first three numbers separated by decimals. So there are a possible 256 IPs in a range where you might get a .0 or .255 which is legitimate for an ISP with a larger block than /24 as most of them have.

I think the whole issue is overblown in the long run and practically speaking. As my nodes on the same public single IP fill up, this issue is not important. It only affects you as you are filling new nodes and have to split traffic with other filling node within a range. The chances of a neighbor having the same 256 IP range is probably next to nil. I know we would like to fill our nodes quickly and this hampers that but we are really in this for the long run not a quick profit.

2 Likes

thanks for the answer @stuberman but the filling speed indeed is relevant for the decision of using raid vs. non-raid (see respective thread for understanding!).
Moreover a profitability analysis is sensitive of the filling speed taking potential drive-failure into account. Buying a 16TB disk that dies after 3 years and by then is not filled up completely would probably not be profitable.

But back to IP-Filtering:
My question aimed to understand if it brings me an advantage to place a node (that I (!) run) in the neighbor house instead of placing it in my own house. Running more nodes in my single ISP-given IP in my own house doesn’t bring me an advantage, clear on that. But what if I run another node in the same /24 Range.
From my understanding the IP-Filtering treats that node as if it was on the same IP, so both nodes share the data that they are filled with. Filtering makes sure that no data is put doubled. For egress it then is clear: Only the TB-amount counts.

But if it is as you say “the chance that my neighbor gets the same IP-/24 range IP from his or our same ISP is near zero” than the problem will be none.

Thanks for explaining.

1 Like

There is no filtering on individual IP’s, just on the subnets. Basically when selecting nodes for upload, the satellite selects a set of subnets and then picks a random node within that subnet to upload to.

In my experiences neighbors don’t usually get close IPs, but that may differ per ISP. A different ISP will definitely have a different /24 range though. So if you want to be sure, pick someone with a different ISP. :wink:

1 Like

That’s the problem. I live in the middle of nowhere and they now supply us with Glasfiber (FTTH). So potentially everyone here will get the same ISP. :stuck_out_tongue:

But to get this clear. The IP Filtering only exists to ensure not to spread all 80 file-pieces to the same “physical” position (due to redundancy).

So if I personally run multiple nodes in different subnets I really could receive more data on “my” nodes during one “file-putting-process” done by the satellite. While running multiple nodes within one subnet, only -let’s say- one single node receives data during that putting-round. I think I got this now.

So, running multiple nodes on ONE single IP has the same effect?? The Filter picks a set of subnets and then -let’s say- one node within that subnet and since my multiple nodes all are behind one single IP only one gets picked and the other won’t receive data during that putting-process… That is -as you say @BrightSilence - the IP-Filter is not filtering IPs, but IP-subnets only. okay, nice. Understood.

Conslusion: Running multiple nodes behind one IP address has the same effect as running them in the same subnet: Only the filling process takes longer.

Back to my original question to get a feeling of where to place my nodes strategically clever to gain data quicker: Within a street? Within a city? Within a county? Within a state?

2 Likes

Like for me, I receive my IP from my ISP in the metropole that is more than 200km away from where I live. I may share the same /24 subnet with someone that is 400km away.

My main question is if a node is full, it is excluded from the selection? Because if not it is a bit of a problem.

EDIT: Nevermind, in the code, it look like it do the check for free space before doing the filtered query.

1 Like

I listened around here at my direct neighbors. The IPs look like this:

93.118.29.XXX
5.42.141.XXX
5.42.139.XXX

Same ISP, direct house to house.
I checked on some other IPs a can look into. Those are approx. 20 and 70 km away. Totally different IP-Subnet.

I’m thinking the Sat could do a traceroute to a SNO periodically. Take the first 60% of the hops from the satellite to the SNO. Create a SHA256 hash out of the IPs collected along those hops.
-comparing this to the current implementation would be the /24 mask.

Say another node is along a similar route and the first 60% of it’s hops from the satellite match the first hash. Well, then you’re inside the ‘/24 subnet’, and are treated as a single node.

this may be a lot harder to defeat then the current workaround of purchasing more IPs outside a /24, or resetting your modem to get a better dynamic IP.

That can be defeated by most of the cheaper ways to get another IP… Like anything VPN based.

As is the current implementation.

My point is that it’s more resistant. I’ll take it to the extreme as an example. Say you wish to classify a single VPN provider’s region as a single node. They own multiple /24s in the same region, so you can have ‘muliple single nodes’ within that provider.

If you pull back the prefix of the traceroute sampled, the whole region is classed as one node, even having multiple subnets.

This would be more flexible, more resistant to purposeful bypass. But is by no means foolproof.

I understand that the current implementation doesn’t prevent that either. I’m just saying that if you’re going to add complexity, this would be one of the most important scenarios to prevent. Although you may have a point that it might cause diminishing returns of several SNOs try to use the same VPN. For now though, there are so many VPN providers with so many locations. It’s still be fairly easy to work around this method.

That’s the question, the added complexity - does it account for enough of the resistance to that threat to be worth it? Probably not right now.

There’s only 16.7million combinations of unique ‘single nodes’ right now :stuck_out_tongue: