I’m running my node now for a couple month and it’s running fine so far. What question came into my mind recently: Is there a real technical need for having my node available under a (dynamic) DNS name?
I mean, I don’t have a clear picture by now how the communication between the customers, satellites and storage nodes is organized. In case that the satellites are the network coordinators, wouldn’t it be enough that my storage node knocks on their door and tell “Hey I’m node #ID and this is my IPv4/IPv6 address”? In other words, this could eliminate for many private SNOs a (single) point of failure named dynDNS provider.
If you wanted to restart your node every time your IP changed, I don’t see why that wouldn’t work (as it’s an env-variable or config entry). But if you do have a setup where your IP changes… I think most SNOs find it easier to leave the node running 24x7 with a domain name instead… and allow something like a dyndns plugin to tell the world about IP changes when they happen.
Plus dyndns is often something running on a router… so changing that domain-name-lookup handles every other service you may be running in your homelab too… not just Storj.
I fully agree with @karsten. I think configuring DNS and/or needing to put external IP into configuration file is a fools errand.
Who initiates connection? Node does, because they know where satellites are, and satellites don’t yet know where nodes are.
Right after connection satellite knows damn well which IP is the incoming connecting is originating from. So, why do we still have to specify it explicitly?
There are cases of asymmetric routing, but those are rare, and for those who do have such need there should be a way to override automatic IP detection. However, for vast majority of node operators this is an unnecessary and unproductive, error prone (see DuckDNS shenanigans) burden.
I’d go a step further and have node attempt to use PCP and/or NAT-PMP, or what’s the heck, fall back to uPNP to manage DNAT to itself in the lan.
There’s definately some good automated firewall-traversal tech out there now that a node could use.
Perhaps at this point it’s a support-load issue? Having a specific IP:port/domain:port in a config file is straightforward to debug… but if dynamic traversal isn’t working that’s a whole other can of worms to figure out.
Failures shall be logged, and I’d argue “did you put correct FQDN?, did you specify the correct port? Is this the same port that you forwarded on the routed? Is your LAN IP correct in the forward rule?” is much more support overhead than "show me your 20 lines from the node log, that will contain all that information, responses from the gateway, and any evidence of success or failure of the automatic configuration.
So I think it would reduce support volume, because no configuration beats any “simple” configuration.
I’d go even further and have node do NAT traversal if everything else fails, via STUN. It will add some latency, but losing some races is better than not running node. It can show yellow warning in the dashboard to nudge the node operator to maybe configure stuff manually, but it will work. And help improve node geo diversity of nodes distribution. Because not everyone has desire to install VPNs on VPSes and all that nonsense, and a lot of residential customers don’t have public IP (nor they need one)
Basically, there are a bunch of improvement some Storj intern shall implement. Or AI.
This aligns with KISS – it shall be S for all Storage node operators, not for a few software developers.
Wanted to chime in and support the sentiment here - if our nodes report our IP/DNS name on startup, then the server can easily see where the connection came from. Why not have this on some frequency (heartbeat?) to ensure the satellites have our current IP? Sure, they need to, in theory, have 5-digit nodes updating their IPs every so often, but that really shouldn’t be a real load to a server…right?
The problem most often is outgoing IP ≠ incoming IP
My server for example has 3 IPS, but outgoing is the default IP and incoming connection are separated on the 3 different ips.
The same is with my server at home, while incoming connection are handled by a VPN server, if my server would initiate the first connection, then it would send the IP wich is a cgnat IP from my provider and so not usable
I guess a simpler solution is leave existing implementation but make it optional? So if you provide a static ip great it will use that like in your setup. And if you don’t it would use incoming/outgoing as outlined above?
The check-in on the satellites is happening once a hour by default, if you would increase this frequency, the satellite may limit the node. If IP would change before that the satellite will not have a valid address to request the node, if its cached IP is not responding. At the moment the IP is cached, but if it doesn’t respond, the satellite will resolve the provided external address by the node.
In the case of DDNS domain it maybe updated that time and the satellite will be able to connect the node and can give it to the customer, otherwise the node will be considered offline, which will result in a bad reputation.
I got your point, but the world of DynDNS providers is wild. Some are free, others have to be renewed after 30 days, IPv4/v6 only and other limitations, reliability & uptime etc.
What would be if Storj creates its own DynDNS service for storage nodes? So if no external IP or FQDN is specified in the node config, the storage node will contact Storj DNS Service to get an address like nodeID.storj.whatever. This would not affect the satellites performance, but it would require an additional service on your side. Maybe it could be even hosted on our nodes as well as a kind of “distributed DynDNS”?
At the end of the day setting up a node should be as easy as possible, and we should avoid extra hurdles. Not everyone here is an IT expert. Keep it as simple as possible.
It is additional expenses for storj, why they need to do it, if it is your responsibilities as node operator provide valid connection IP for your node.
DynDNS is a cheap, boring, and easy-to-use commodity service you can get from many providers already: Storj shouldn’t be trying to compete with them. It’s not what Storj is good at.
Tens of thousands of nodes are already providing more than double the space customers want to pay for. And it has been that way for years. It’s not clear to me anything needs to be done to make running a node easier: when we have a mountain of idle capacity.
Maybe next year when Storj becomes cashflow-positive, and can maybe attract investors, and expand… then perhaps they’ll need to attract new SNOs. Then they can make node management easier too.
The solution of the overpopulation is to discourage new nodes from joining, by reducing payout rates. Not reducing reliability of existing nodes by forcing them to rely on third party for the essential service. Storj is inserting an unnecessary point of failure they cannot control into data path. Why?
The node can check-in as soon as it detects its external IP changed. It’s a non-problem.
Nobody is forcing anybody to “rely on third party for the essential service”. It’s not essential: DynDNS is a convenience, and optional, and not even a consideration for those with static IPs.
Hmmm: software that acts on detected change to an external IP address? Like how dynamic DNS can make sure the entire world points to your current correct IP (and can handle all your services: not just Storj)?
Storj certainly could add dyndns-functional-equivalent code to the node. But why reinvent that wheel?
Not reinvent. Reimplement. And I just explained why: because otherwise node operators insert DuckDNS into the stack.
It’s mandatory for vast majority of operators, the same majority who has dynamic ip. Static IPv4 is very expensive and rarely needed. It’s never needed on residential connections that actually prohibit hosting services.
We (even OP) were talking about dynamic DNS providers in general: and you skipped to one particular one (of the dozens available) that had some recent problems. SNOs make many choices: hardware, ISP, OS, filesystem etc: not all work perfectly all the time. Storj can’t be expected to “reimplement” any part of the stack some SNOs have extremely rare issues with.
Dynamic DNS is an old, boring, widely-adopted service. It works. Storj reimplementing it is a waste of time they could better spend elsewhere. If you argue that “It’s mandatory for vast majority of operators…” … then… OK… just use it. Use what exists. Use what a hundred thousand homelabbers use every day. Pick the provider you like. Don’t reimplement it for the Nth time.
I agree with you on principle, in ideal world separation of responsibilities is great. We are not in ideal world.
Of course – I’m illustrating that there is a possibility to introduce issues. It would be weird if I quoted a provider that has stellar performance history :). I use cloudflare as my DDNS for years (decades?). Did not have a single issue. It’s not a problem for me personally. Every my node runs in jail along with inadyn that updated DNS record even for nodes that are on the static IP. Because why not.
Ha.. hash… hashstore, anyone? They reimplemented the core of how node stores data! Why? Because some potato nodes were somewhat slow.. And this was not data integrity or availability problem, just performance. Flaky connectivity affects the performance just the same. It also involves extra support work. All for something that can be solved in two lines of code.
The tradeof seems worthwhile for me: “reimplement” the bicycle, but overall improve reliability and reduce cost of support.
The whole topic exists because it does not always work: Because users are allowed to select the service, because users will pick the shittiest one possible. But this is just one side of the problem.
But I don’t need it. It does not “exist” for me. My homelabbig does not involve having a FQDN pointing to my house. Storj needs it.
This is another side: You are arguing that it’s better for 2000 operations to have to configure DDNS manually, just to spare one storj developer from spending two hours building in the functionality that would relieve those 2000 people from this mundane chore, (let alone improve resilience of the network). I don’t consider this a fair or productive tradeoff.
Implementing IP tracking would be the start. That will pave the way for further improvements – for example, removing the need to forward ports or have a public IP in the first place. Yes, I know, now the quantity of node operators with public IP is more than enough. But if storj is to continue to grow, they would want to let more people join. Most people are going to be behind CGNAT. Most of those people would not be arsed to setup VPN, let alone have access to one with unlimited or cheap traffic.
You would argue “but ensuring node connectivity is node operators job” and you would be right. But I would come back and argue that – making is not their job will help improve perofmranc and resilience of the network, so lets reimplement this bicycle too.
Taking control of all parts of connectivity now is more important that some purist view that “its’ not our job”. It’s very much is.
This is great to see being heard - I hope it actually gains some traction because ultimately Storj most likely (?) wants to have node maintenance as hands-off as possible and this would be a great step forward.
