Could you please show the available options?
In case you are operating a STORJ node on a QNAP device, you would have to update the docker run command in the following file:
/share/CACHEDEV1_DATA/.qpkg/STORJ/web/public/scripts/storagenodestart.sh
and replace
-p ${PORTADDR}:28967 with
-p ${PORTADDR}:28967/tcp -p ${PORTADDR}:28967/udp
1 Like
Thanks @jtolio for that extensive response! I hope this will help alleviate some of the concerns around the use of QUIC and specifically the choices Storj might make in the future.
I’m happy to see the ultimate goal is not to use a one size fits all implementation, but actually optimizing it for this specific use case with both customer and storagenode experience in mind.
Specifically happy to see the mention of LEDBAT here. I remember looking into that a while ago and this specifically stood out to me
LEDBAT is designed for use by background bulk-transfer applications to be no more aggressive than standard TCP congestion control (as specified in RFC 5681) and to yield in the presence of competing flows, thus limiting interference with the network performance of competing flows.
Source: RFC 6817 - Low Extra Delay Background Transport (LEDBAT)
From a node operators perspective, this would be a really welcome prioritization. When I need my network for something, I would want my use to get priority as there will always be other nodes more responsive enough to pick up my slack.
Either way, I’m in on helping test whatever you plan to try on production satellites, but also on the 2 testnet satellites. I don’t think my network would be among the first to buckle (I get the feeling that I have been really lucky with my ISP when I hear some of the stories others report).
Maybe a subset of SNOs can help pave the way for more hesitant SNOs to take the plunge after the first rounds of optimizations have been implemented.
Either way, thanks for taking time out of your busy schedule to add a lot more information and your thinking on this subject specifically!
5 Likes
sQUIC
I very much appreciate the detailed response.
I haven’t explored the official documentation in a while. If the information in your post isn’t included in the documentation, it might be useful to put the QUIC plan in there…
Having run several p2p experiments over the years, Storj does a lot of things right that many other projects have gotten slightly not so right.
My LAN is fairly complex. UDP servers break it at the moment. My ISP is UDP unfriendly. That’s the situation. It’s really good to hear that participation in Storj can continue without enabling UDP at the moment. I understand that you need SNOs to enable it for live network tests. Unfortunately, I will not be able to do that at this time. I will continue doing my own tests from time to time, perhaps I will be able to participate in UDP packetized sQUIC in the future.
Perhaps, sQUIC will replace gQUIC for p2p networks everywhere
Almost pay day. Time to gear up for Polygon payouts.
4 Likes
I have sucessfully set up my node, but I see that my QUIC is misconfigured.
I’m behind CGNAT, so I rent a VPS which is connected to my homelab via Wireguard.
I do receive Ingress and Engress traffic, so I’m correctly connected to the network.
I’m unsure how to troubleshoot this problem. So I’d like somebody to point out what could be misconfigured on my homelab or VPS.
These are my iptables settings on the VPS.
iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 to:192.168.1.54
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 to:192.168.1.54
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:5432 to:192.168.1.54
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:28967 to:192.168.1.54
DNAT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:28967 to:192.168.1.54
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
SNAT tcp -- 0.0.0.0/0 192.168.1.54 tcp dpt:80 to:192.168.1.55
SNAT tcp -- 0.0.0.0/0 192.168.1.54 tcp dpt:443 to:192.168.1.55
SNAT tcp -- 0.0.0.0/0 192.168.1.54 tcp dpt:5432 to:192.168.1.55
SNAT tcp -- 0.0.0.0/0 192.168.1.54 tcp dpt:28967 to:192.168.1.55
SNAT udp -- 0.0.0.0/0 192.168.1.54 udp dpt:28967 to:192.168.1.55
iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-A FORWARD -i eth0 -o wg0 -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o wg0 -p tcp -m tcp --dport 443 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i wg0 -o eth0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o wg0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o wg0 -p tcp -m tcp --dport 5432 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o wg0 -p tcp -m tcp --dport 28967 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
The CGNAT is probably not going to allow UDP traffic to traverse the NAT to your node.
So, you’ll need to tunnel the UDP QUIC through TCP. This can be accomplished using various methods, including ssh as long as you’re not using Debian Buster which has a bug in allowing tunnels via the PermitTunnel yes setting in sshd_config.
Here’s some information regarding tunneling UDP through TCP:
RFC 5128: State of Peer-to-Peer (P2P) Communication across Network Address Translators (NATs)
1 Like
I got the message “Misconfigured” on 4 nodes. This is what my Docker command looks like: sudo docker run -d --restart unless-stopped -p 28967:28967/tcp -p 28967:28967/udp -p :14002:14002 -e WALLET=“xxxxx” -e EMAIL=“xxxxx " -e ADDRESS=“xxxxx:28967” -e BANDWIDTH=“20TB” -e STORAGE=“7.5TB” --mount type=bind,source=”/volume1/docker/storj",destination=/app/identity - -mount type=bind,source="/volume1/docker/storj/share",destination=/app/config --name storagenode storjlabs/storagenode:latest
The ports are released for tcp and udp according to each node in the Fritzbox. On my Raspberry pi everything runs with the same settings without any problems. Have already deleted the nodes and started again manually, without success.
Said nodes run on 2 different Synologies.The firewall was switched off as a test
Welches Fritzbox-Modell? Sind die UDP-Portfreigaben explizit als IPv4 und IPv6 eingestellt? Bei 2 von 4 meiner Fritz 7590 kann ich nur IPv4 freigeben, weshalb kein QUIC funktioniert.
English: Which Fritzbox model are you using? Did you explicitly forward UDP for both IPv4 and IPv6? On 2 of my Fritzbox 7590, I can only forward IPv4 which causes QUIC to not work there.
It is a 6951. The raspberry pi is behinde the same router and works fine while the synology isn’t
Isn’t it that the Synology has lost the IP? It happened to me with Qnap
I use Synology ds920+ and I had to update command line to -p xxx.xxx.xxx.xxx/tcp:port -p xxx.xxx.xxx.xxx/udp:port -p 14002 . After a restart of the docket container. Now is says ok.
I see. This router has a bug. Then you always will see this warning on your dashboard until the vendor fix their bug in the router firmware.
I have enabled QUIC options in configuration of all nodes since first topic about it appeared on forum. No problem with testing it from my side.
@Sierra - How did you fix this on your Synology? From my port settings in docker, it looks I have port 28967 open for TCP and UDP traffic.
1 Like
That fixed it more me @lex-1 . Thank you! Some have suggested a simple restart of the node but that didn’t do it for me. I had to add in the new seperate udp/tcp settings.
Hey, That looks good to me, but that’s just the docker container. You also need to go into the the port settings in the actual synology.
OKAy how about i cant forward the port via UDP to enable QUIC.
because i go with VPN, which alows only TCP. I wont have it.
I have QUIC: Misconfigured on dashboard, and thats fine.
BUT
i got email 10 hours ago from storj that this node is suspended on a satelite because of that!
thats not fine at all :
"Your Storage Node on the europe-north-1 Satellite was suspended because it produced errors too often during audits.
You were suspended on 2022-02-02 at 18:18 UTC."
And thats not true, because my dashboard says 100% audit everywhere.
its 1.47.3. its Windows Gui.
EDIT: okay that might be due some internet problems in this node as well, it got disconnection problems i fixed so its not nessesary beccause the new 1.47.3 version and QUIC BUT Please confirm, i can have QUIC Misconfigured and still have this Node?
Thank You