Muliple nodes, a total of 16 nodes on one IP

Terms of Service are out of date. What the new terms are going to be, I don’t know. I’m also not going to provide legal advice if you should or shouldn’t follow the ToS as they are written as I’m a contractor and not an executive of Storj Labs to make that kind of decision.

Heunland and I agree that bypassing the /24 rule to run more nodes in pursuit of additional data to gain profit is bad for the network because it can cause a centralization of the data and put that data at risk if that location is taken offline for an extended period. That is why the rule exists. And I think anyone that is clear headed understands that. Whether that is written into the ToS or not, the rule is there for a reason. Saying you are just following the existing ToS would mean you are only running 1 node at your location. And that is cherry picking the parts you like and taking forum comments that work for your side (Such as JG saying you can run more than one node) and ignoring forum comments that don’t. (Heunland saying you shouldn’t run multiple nodes on multiple IP’s at a single location)

I understand the need for clear Terms of Service. At the very least it will make discussions in this forum better for everyone involved.

4 Likes

I can prove you wrong by reality.
That 10Mbps is referring only to load balancers. But the virtual NIC is up to 2.5Gbps. Some days I got over 30GB increase of my nodes over it, so ingress over that. See for example also https://techoverflow.net/2022/01/16/oracle-cloud-always-free-tier-arm-network-speedtest/

I think Storj could better to focus on sales, than enforcements, because if Ingress will go from all doors and windows there will be no need in multi ip itself. I remember when EU satellite provided havy ingress all low power PCs screamed from overload.

7 Likes

ToS isn’t saying that, ToS is only saying you’re running one node per IP. That’s quite different. So please, read section 5 very carefully: Node Operator Terms & Conditions
These mishaps, are actually clouding the discussion in my opinion.

I think we all can agree on this, but I can’t understand why it takes that long to even put only this in the ToS in place of statement 5.4.

There’s being asked for over a year…

So again, perhaps hurry things a little bit and keep things up-to-date along with significant changes?

In general we’re not taking a different stance and I think almost everyone sees the benefit of this so-called “rules”, but as long these are not real rules you have no back-up in enforcing them. And I think it’s downright overzealous -of even people related to Storj- acting like these are already real rules instead of just writing them down. So the bolds aren’t favoured by fortune and an equal play field is being created for everyone?

I appreciate the length of time since it has been out of date. However the company is small and resources are better spent focusing on sales and the technology itself. Node operators have been getting on fine without an updated ToS document. However, we’ve made significant changes in the past few years so it does make sense to update the document so it isn’t that far out of date.

As of right now, the legal team has the document and is reviewing it. How long that takes likely depends on if it needs revisions and approvals. Hopefully it is completed soon. What lawyers do, I don’t know, but I’m sure posting things in this forum to try and speed them up is not likely to result in any progress there. I’d ask that we all just be patient and allow them to do their job. I assure you that it is being worked on and will be posted when it is done.

5 Likes

I’m fine with it, than we’ve got an acceptance stance towards any behaviour on the matter. I’ll refer to this thread every time when anything is being referred as being against the ToS which isn’t explicitly written in it that way. Or when people are being told to abstain from certain behavior because being against the spirit of Storj but not being mentioned in the ToS :wink: .

I liked the discussion, but also the clarity it gave on enforcement and to what extent it’s being prioritized (almost none).

10Mbps is 3.25 TB/month. Are your nodes pumping that much data? Mine don’t. But yes, 10Mbps violates tos, but hey.

No neighbors.

Latency increase is insignificant, few ms, datacenter is in the same city.

As a test I connected that node via another datacenter, two states over, with additional 20ms latency, and did not see much difference in performance.

And then — slow node is better than no node, no? Those locations are behind double NAT. I can’t host the node there any other way.

I even have over 20ms increase in latency, but over 20MB/s (=160Mbps) bandwidth and got over 1TB increase of disk use in two weeks on my best node using Oracle Cloud. Also no neighbors, different /24 on all computing instances of which two are being used for Storj purposes.

Only drawback is that after each update, QUIC seems to be misconfigured.

Also others report significant higher throughput then 10Mbps, like oracle cloud infrastructure - The total amount of free network bandwidth an always free compute can use for a month or some period of time - Stack Overflow

Also Oracle is talking about 10TB outbound traffic a month, which is an average of 32Mbps.

This is likely due to start up sequencing. You need to ensure that wireguard tunnel starts first, and then the node. Otherwise I also see the TCP working but quick shows misconfigured. I’m pretty sure it still works, it just node only checks it on startup, and if routing changes later — it gets confused.

I’m not sure if you can bind node to a specific interface — that would be the best.

2 Likes

Right, let’s go through these.

You can comply with both by just running a single node. Using a trick to circumvent one clause doesn’t mean you can then break another. So 5.4 even if it was still relevant, doesn’t allow you to break 4.1.3. So this argument is moot.

Not relevant. It is part of the storage services now, so it shouldn’t be “disrupted” as defined in 4.1.3.

Agreed, but 4.1.3 doesn’t address the matter in 5.4 at all. You can use multiple nodes behind one IP because of written statements by Storj Labs that explicitly allowed that since writing the ToS. Not because of 4.1.3.

Also see:

  • 17.1.1. Entire Agreement; Severability; Waiver. This Agreement sets forth the complete and final agreement of the parties concerning the subject matter hereof, and supersedes, replaces all prior agreements, written and oral, between them concerning the subject matter hereof. If a term of this Agreement to be invalid or unenforceable, the remaining provisions will continue in full force and effect. A party’s consent to, or waiver of, enforcement of this Agreement on one occasion will not be deemed a waiver of any other provision or such provision on any other occasion.

Meaning that the invalidation of 5.4 doesn’t mean any other terms are also invalidated.

Incorrect. See the above clause as well as consider what an arbitrator would decide when presented with the overwhelming amounts of explicit statements from Storj Labs allowing running multiple nodes behind a single IP. They have given that waiver as discussed in clause 17.1.1.
And no, it doesn’t LEAD to a broader interpretation of 4.1.3, because 4.1.3 has always used broad language. It has 2 relevant components. I think we can both agree that the purpose of the discussed circumvention of the /24 system is to get more data, so that’s a check on that part. Which would leave Storj Labs to prove in a hypothetical arbitration that using a VPS/VPN to circumvent this system constitutes disruption of the storage node services. I would agree that that’s not necessarily a given, but probably also not a very hard case to make. And that’s all there is to it really. The other clauses simply aren’t relevant.

Prepare to lose in binding arbitration if these are the arguments you plan to bring. :wink:

Yeah, I’m not burning my hands on this stunt. Good luck with that, mate.

I’m not counting on it. The truth is that it’s not a big enough problem at the moment. And with broad clauses giving them backing already, they don’t need to specifically mention this in the new terms. Alternative systems could have negative impact on legitimate node operators as well. So it might do more harm than good to go after it right now. Of course that is as long as this doesn’t become a widespread used thing, hence why we shouldn’t post instructions on doing this.

Using VPN for this purpose has been suggested and supported by Storj Labs many times. They could change that with the new terms, but I highly doubt that. If you’re also using it to circumvent the /24 system though, you’re on shakier ground already. The new terms won’t make that any better, but possibly will make that worse.

Not really. VPN/VPS close by might add only single digit Ms of latency. While legitimate connections can easily have double digit latency to begin with. There is a massive overlap between the two. Latency is a useless measure and either would let a lot of VPN/VPS users survive or would kick out a lot of legitimate slower connections as well.

2 Likes

4.1.3 is explicitly talking about “You will not modify or attempt to modify the Storage Node Software for any purpose including but not limited to attempting to circumvent the audit, bypass security, manipulate the performance of, or otherwise disrupt the Storage Services for any reason, including but not limited to attempting to increase the amount of data stored or bandwidth utilized or the amount of Storage Node Fees, as defined herein, and you will not otherwise interfere with the operation of the Storage Services.”

Using a VPN isn’t modifying the Storage Node Software, since Storage Node Software only pertains to the source code and thereof created binaries that can be used to participate in the network made available by Storj (see 1.5 and 2.2). So the remainder of your post on the 4.1.3 vs 5.4 subject can be disregarded, because it doesn’t imply anything for the environment or network setup it’s running in.

So any disruption or evasion of so-called rules like /24-“rule”, isn’t from modification of the SN-software.

In the remainder, we’re thinking alike. Indeed 17.1.1 indeed implies 4.1.3 and 5.4 are still in effect, but not relevant because the first doesn’t pertain to the current situation and the latter isn’t enforced on purpose. Or you can see the manual as a kind of waiver, difference is not relevant since having the same effect.

Case closed again, as I don’t see a new point of view arising from your arguments.

So doesn’t, as explained up here.

Or you can just comply to both, by creating a new VPN the way outlined above for every node you start: all nodes behind different IPs, without changing a single piece of the SN-software.

And it’s a real pity in my opinion, latency doesn’t help in this matter. So I hope a brilliant brain will come forth with a better means to differentiate between VPN and non-VPN-connections.

I’m running two systems behind GC-NAT, using two different VPNs. Side effect of having more ingress due to different IPs is much appreciated although not the primary purpose. But the shakier ground isn’t forthcoming from current ToS, as explained quite extensively in this thread. We’ll see whether future ToS-es will change this.

So, when the Storj-service would start after wireguard it should work in your opinion?
I actually doubt it to be a solution for Docker, because I explicitly stop Wireguard before starting Docker, and I restart it afterwards.

So probably I will take some time to write a script polling QUIC-config every X minutes and restarting the service as soon as it turns out to be misconfigured.

You’re running it as a service as I remember. I’m running it in Docker. In Docker you theoretically can specify -p IP:hostport:dockerport, meaning you have to run all VPNs on a different subnet. I never tried, because I’m running them all in different virtual machines.

You’re right. I thought at first that the “or disrupt the storage services” part was an or to that initial start of the sentence, but it seems to instead be part of a comma separated list. Why do lawyers insist on writing single mile long sentences? Ambiguity would not work in their favor here as ambiguity always gets interpreted o favor of the party who didn’t draft the agreement.
So yeah, I guess they have some work to do (unless there is another clause I couldn’t find that covers it).

Hrmm, yeah, I guess you don’t have much of a choice in that either then.

I think the point is that it works either way. It just doesn’t display it working, because it didn’t work when it checked on startup and that indicator is then never updated.

So we’re finally on the same page, concluding:

  • ToS essentially doesn’t say any physical location should be bound to no more than one IP (actually rather the opposite, if we choose to comply to article 5.4 and also decided to run more than one storage node); although it would be more in the Storj-decentralization-spirit if it was in the ToS?
  • So there is also no evasion of a /24-“rule” using VPN or anything alike, because this rule actually doesn’t exist in the ToS. And the ToS doesn’t explicitly demand anything with respect to the environment the unmodified storage node software runs in? Although, it would also be more in the Storj-spirit if it was and also creates an unequal play field in comparison to them who aren’t apt enough to work out the same or comparable solution.

I had the option to route all nodes over the same VPN, but it would take some more time to configure. Besides I read the ToS to the letter, concluding it didn’t forbid using multiple VPNs and I did like the side effect of more ingress. So there was a choice, which turned out to be quite simple in the end.

Although appreciating their tons of work for the Storj-project and honoring their credits, in the end I hope people like @littleskunk and @heunland will revisit their expressions on people said being cheating or being bold. And @Alexey revisiting the threat of banning nodes due to so-called /24-“rule” evasion, which isn’t backed-up by the real ToS; which is overzealous being mentioned in this matter anyway as far as I can see. Feeling with @mihalko , a lot of people are being treated unfair that way. Because the ToS has been neglected for a while in my opinion (intention to change ToS apparently dates back to 2020) and isn’t aligned with current demands and spirit of the Storj-project, while people are often judged by whether they act in the spirit of Storj even if that’s not in their best interest.

So please, abstain from overzealous ToS-references and make sure it’s updated and keep it up-to-date; in which I don’t care whether VPN’s or anything alike (VPS + iSCSI, whatever) are being condoned or condemned. But create clarity on the matter.

1 Like

I think the part about it being bold was actually something I said. And I stand by it. Regardless of what the ToS says, someone was still asking to circumvent a system for their benefit and the detriment of the network and other node operators. It’s pretty bold to expect a forum full of people who would only be hurt by this to help you with this. While I don’t agree with some of the wording used, I absolutely think neither Storj, nor the community should encourage that.

I think in the end the reference to ToS has been a distraction. It’s not the intended way for SNOs to run nodes and it has an overall negative impact. Allowing this on large scale turns it into a game theory scenario where in the end everyone is worse of. ToS changes would be useful to give comments here more backing, but in the end this needs a technical solution. It’s just that I don’t know what that solution could be. Maybe Storj Labs doesn’t either. Time will tell, but until then, I will not be encouraging or helping anyone set up circumventions.

1 Like

I actually doubt whether it’s boldness, or just ignorance. Also given the fact @eddy89 doesn’t seem to understand DNS in his question and one of his first replies, especially the relationship between domain name and IP. @heunland is the one interpreting it as a question with the aim to be able to circumvent the /24-“rule”; which is being called bold by you. After which the full discussion quite soon derailed…

Again, I doubt whether it’s a fair interpretation. But that’s something only directly involved people can answer, especially @eddy89 .

Node operators have been getting on fine without an updated ToS document. However, we’ve made significant changes in the past few years so it does make sense to update the document so it isn’t that far out of date.

This statement is not true, IMO. SNOs can be just fine without ToS, but with consistent statements from Storj company representatives. And (part of) this thread is pretty much result of the inconsistency which people, including me, wrote about here.

I am sure, you will be consistent in the future about this, after this. Meanwhile, I will wait, while storj will actually need more supply to get to the exa byte, so you will publish a official solution to scale for SNOs, if needed.

2 Likes

… Deafening silence BTW …

1 Like

And that’s it.
Storj is aware of it and will deal with it when it is a problem. I think this is what @jtolio has mentioned sometimes.
By the way it is always worth to repeat that using VPN etc. might be required and legit to run nodes behind CGNAT.
So forum moderators could allow such questions if they are about CGNAT or so and dismiss discussions if the intention is clearly to bypass this rule.

1 Like