Need help with wireguard on vps for storj node

gingerbread233 · November 16, 2023, 7:46pm

Hello, I’m trying hard to get my stuff to run and couldn’t get any useful information so far. I hope somebody can help me out.

I’m trying to run a node through wireguard. On my system. I run debian with docker/portainer. I installed gluetun on my system as vpn client. On my vps I installed wireguard (pivpn), when running gluetun, the logs show me the public ip of the vps, so it seem to work. I also connected my phone through wireguard on my vps to test it. Everything seem to work so far (I get basic internet connection for websites etc). But when trying to access a specific port through the public ip of the vps, I can’t establish a connect. Even though the port is opened on my vps (like I did with wireguard).

The wireguard clients can ping each other, the server can ping the wireguard clients, and the clients can ping the server, so connecting to each other internally shouldn’t be the problem.

For testing reasons I deployed jdownloader with gluetun. Downloading worked with jdownloader (through gluetun/wireguard, showing the vps ip adress, but I couldn’t get any connect to the gui via the public ip and port of my vps. On my local network I’m able to access the container via the port.

Even though I forwarded the ports on my vps and the gluetun container (via yaml), there is no way accessing the container through the vps via the public ip address and port. The vps is running Ubuntu with disabled firewall, since it’s managed through the providers interface.

Is there anybody who run storj through wireguard/gluetun? What am I doing wrong? Thank you in advance.

arrogantrabbit · November 17, 2023, 12:02am

It appears the issue is routing between your completely separate docker and wireguard networks, exacerbated by the whole gluetun being a separate container. Do you really want to become an expert in configuring fragile static routes all over the place and fighting windmills with TTLs and what not…

Instead of complicating the setup even further do this:

configure container with its own network stack (that’s the whole point docker exists)
Install wireguard client inside the container, and prevent container from auto-updating.
Enable packet forwarding net.ipv4.ip_forward=1
The end.

If you want to get fancier – create your own container from the original container as a base image + wireguard with your config. Deploy that container instead.

That’s on the client side. On the server side you need to ensure your iptables is setup for correct routing and masquerading. There was a guide in one of the responses to you in the other thread. Setup a simple http server and try reaching it from outside, through VPS, through the tunnel, into the container.

gingerbread233 · November 17, 2023, 5:47am

That’s basically what is did, I deployed gluetun and storj in the same stack, opened the ports on gluetun and the vps, did “network_mode = service:gluetun” on the storj section in my yaml, so the traffic flows from the storj container to the gluetun container, then the gluetun container routes the traffic to the wireguard server. But for some treason incoming traffic can’t reach my container through the vps to my gluetun/storj stack on docker. It’s strange because basic things like downloading stuff with jdownloader through gluetun ist working, but I can’t get a connection to the ui via vps, so something is not right jet. Do I have to tell the vps server (Ubuntu) that incoming traffic from port “xxxxx” needs to go to the wireguard client? Because wireguard clients use other ip addresses inside the OS.

JWvdV · November 17, 2023, 3:03pm

Jup, see previous post. There were some iptable-rules in it. So, what’s your current server Wireguard-config?

gingerbread233 · November 18, 2023, 12:55am

Sorry, I opened another post, because the other one was full of discussions about other stuff. Today I unfortunately deployed a container with daemon rights, which deleted all of my containers, which I needed to get running again. But now I saw that there is a wg0.conf in the wireguard folder, which isn’t for the client as I thought, but for the server, in this file there were some iptables. Client and server files were both conf filess, which confused me. So I have to add the iptables to the server side “wg0.conf”, so it gets redirected to the client?

JWvdV · November 18, 2023, 6:16am

Although I can see that the other post was cluttered with a dirty discussion about what is allowed and isn’t from perspective of Oracle and Storj, it would be nice if you sticked a little bit to the questions asked and the advices given.

In that other post I even added a working config you could use, on the server (and not the client to re-iterate):

[Interface]
Address = 10.66.66.1/24,fd42:42:42::1/64
ListenPort = 59435
PrivateKey = [~TheKey~]

# Input from outside, mind the ingress-rule from Oracle-cloud!
# In my case the Wireguard-port is 59435 
PostUp = iptables -I INPUT -p udp --dport 59435 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 59435 -j ACCEPT

# Forwarding bidirectional, %i is substituted for the name of the wireguard-adapter
PostUp = iptables -I FORWARD -o %i -j ACCEPT
PostUp = iptables -I FORWARD -i %i -j ACCEPT
PostUp = ip6tables -I FORWARD -i %i -j ACCEPT
PostDown = iptables -D FORWARD -o %i -j ACCEPT
PostDown = iptables -D FORWARD -i %i -j ACCEPT
PostDown = ip6tables -D FORWARD -i %i -j ACCEPT

# In my case the ethernet adapter is enp0s6, change according to your situation
PostUp = iptables -t nat -A POSTROUTING -o enp0s6 -j MASQUERADE
PostUp = ip6tables -t nat -A POSTROUTING -o enp0s6 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o enp0s6 -j MASQUERADE
PostDown = ip6tables -t nat -D POSTROUTING -o enp0s6 -j MASQUERADE

# STORJ, in my case it's the client with IP 10.66.66.2.
PostUp = iptables -I INPUT -p udp --dport 28967 -j ACCEPT
PostUp = iptables -I INPUT -p tcp --dport 28967 -j ACCEPT
PostUp = iptables -t nat -A PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.66.66.2
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.66.66.2
PostDown = iptables -D INPUT -p udp --dport 28967 -j ACCEPT
PostDown = iptables -D INPUT -p tcp --dport 28967 -j ACCEPT
PostDown = iptables -t nat -D PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.66.66.2
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.66.66.2

# The STORJ-client
[Peer]
PublicKey = [~PublicKey~]
PresharedKey = [~PSK~]
AllowedIPs = 10.66.66.2/32,fd42:42:42::2/128

You only need to adapt it to your situation.
Did you use the install script from angristan?

If you want further help, we need to see your server Wireguard-config. So the contents of /etc/wireguard/*.conf ( mind to cut out the keys ).

gingerbread233 · November 18, 2023, 4:28pm

Even though I use wireguard pretty often, I never digged very deep, because everything was working fine when using it with my phone etc. I used the install script from pivpn.io.
For testing purposes, I tried to route jdownloader instead of storj, because I don’t want to turn the container on and off several times. I adjusted the ip adress, and the port (in this cas 5801 for jdownloader gui), but unfortunately I can’t get any connection. Downloading through the vpn still seem to work. Please take a look at the attached pictures. In the wg0.conf file, my client is listed with public and preshared key, but when opening the file for the client, the keys dont match, is this right? Connection etc. is working for some reason.

I hope you can help me out.

JWvdV · November 18, 2023, 7:03pm

Server config looks fine, although you should check with ip a whether your device also is named enp0s6.

Since you installed it using PiVPN, I actually don’t know whether the configs are compatible with each other. Because as far as I can see, that script uses iptable-persistent which uses other config files. That’s of course fine, but I prefer to keep everything as much in one file as possible. As this script really adapts a lot in your system as well if which I can’t overseer the consequences in a short glance. And it even installs openvpn, which is quite less efficient (lower bandwidth) than Wireguard and unnecessary in this case.

I never used that glueton-docker. I also doubt it’s gonna work, because that docker has to be configured again that all incoming traffic from STORJ (not related to any outgoing traffic), needs to be forwarded to your STORJ-docker, which actually is not the idea of these containers because they are meant to be agnostic as possible besides the fact docker containers usually don’t have a fixed I-address. But this is most probably your current culprit: traffic from STORJ like upload and download requests are being forwarded to your VPN-docker, but then they are being dropped because that docker didn’t know how to handle them > fix that.
This is something that has to be persistent, so is something that needs to be done using the docker compose file or scripted. Because every other change will be lost after every upgrade of the image.

To be honest with you, also to save you some previous time, as far as I can see you don’t seem to understand the basic logic of networking. Therefore you’re making it yourself very hard to find your own path here. So, just follow the manual from @arrogantrabbit (I posted in previous topic) or just use my the angristan-install script. If you really want your host networking not to be influenced, just create lxc-containers in which you install Wireguard and run your STORJ-docker as I also showed in previous topic.

I actually am finding myself considering, whether to help you, or leaving stew you a little bit more in your own juice. Because I think you’re trying to attain something that’s out of your league also being stubborn to follow your own path. And yeah, this is being bold Dutch. But I would really appreciate it, if you proofed me to be wrong.

gingerbread233 · November 18, 2023, 9:10pm

Thank you for your help, I managed to get it working. There are two seperate “port” flags for docker compose on gluetun: “PORT” (for local portmapping) and “FIREWALL_VPN_INPUT_PORTS” (for getting traffic through the vpn). The last one did the trick. And yes, my vps also had the same adapter than yours, I checked it

Just because I don’t understand “basic” networking yet, doesn’t mean I can learn it. The best way to do it, is via lerning by doing, and help from others. I appreciate your help and work you did for me, in the end you helped me alot, with iptables etc.

JWvdV · November 18, 2023, 10:06pm

Then I’m wondering what your config looks like now, especially because I’m wondering how the inbound traffic is being redirected to the right Storj-docker.

gingerbread233 · November 18, 2023, 10:34pm

The wg0.conf looks like the one you sent me, only with adjusted IP adress and port.
It looks something like this:

I forwarded port 14002 to access it locally, and the port 28967 via vpn, so I have something like a split tunnel. The inbound traffic gets redirected to the right docker via “network_mode”, this would work also if both containers aren’t in ine stack, you hast have to add it to the right container/yaml. The traffic gets in via the the iptables, then to the gluetun client, the “FIREWALL_VPN_INPUT_PORTS” allows the inbound traffic, and forwards it to the connected client (via “network_mode”).

JWvdV · November 19, 2023, 6:27am

Well done, so in the end you proofed me wrong on the point it seemed out of your league!

One point though: what happens if the Wireguard server isn’t reachable?

snorkel · November 19, 2023, 11:05am

I thaught about this too. I don’t use wireguard for nodes, only for remote access my networks. The wireguard client shows connected on pc and iphone, even if there is a problem with the server, or it’s wan ip changed.
So what can you do to check if the wireguard is working on server, has access to wan and restart the client-server connection untill all is fine, and you can see the light on the end of the vpn tunnel?

gingerbread233 · November 20, 2023, 6:35pm

@snorkel @JWvdV I tested it, while the client was connected to the server, I restarted the server, after that, the client connected automatically, you could see in the logs, that gluetun was trying to reconnect, as the vps and wireguard were back online, the client reconnected and everything was working like usual.

Walter1 · October 21, 2024, 3:50pm

Hello together,

when I try to run a node on a wireguard-conncection I’m failing to ping the storage node, the connection is not working.

Here is the server-conf0:

[Interface]
Address = 10.0.0.1/24
ListenPort = 51820
PrivateKey = PRK

PostUp = iptables -I INPUT -p udp --dport 51820 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT

PostUp = iptables -I FORWARD -o %i -j ACCEPT
PostUp = iptables -I FORWARD -i %i -j ACCEPT
PostUp = ip6tables -I FORWARD -i %i -j ACCEPT
PostDown = iptables -D FORWARD -o %i -j ACCEPT
PostDown = iptables -D FORWARD -i %i -j ACCEPT
PostDown = ip6tables -D FORWARD -i %i -j ACCEPT

PostUp = iptables -t nat -A POSTROUTING -o ens6 -j MASQUERADE
PostUp = ip6tables -t nat -A POSTROUTING -o ens6 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o ens6 -j MASQUERADE
PostDown = ip6tables -t nat -D POSTROUTING -o ens6 -j MASQUERADE

PostUp = iptables -I INPUT -p udp --dport 28967 -j ACCEPT
PostUp = iptables -I INPUT -p tcp --dport 28967 -j ACCEPT
PostUp = iptables -t nat -A PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.0.0.2
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.0.0.2
PostDown = iptables -D INPUT -p udp --dport 28967 -j ACCEPT
PostDown = iptables -D INPUT -p tcp --dport 28967 -j ACCEPT
PostDown = iptables -t nat -D PREROUTING -p udp --dport 28967 -j DNAT --to-destination 10.0.0.2
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 28967 -j DNAT --to-destination 10.0.0.2

[Peer]
PublicKey = PUK
AllowedIPs = 10.0.0.2/32

Here the client:

[Interface]
PrivateKey = PRK
Address = 10.0.0.2/32

[Peer]
PublicKey = PUK
Endpoint = ADRESS:51820
AllowedIPs = 0.0.0.0/0

Here the wireguard run command:

docker run -d --restart unless-stopped --stop-timeout 300 --name=wgc --cap-add=NET_ADMIN --dns=8.8.8.8 --sysctl net.ipv6.conf.all.disable_ipv6=0 --device /dev/net/tun --log-opt max-size=4m -v /root/wgc/wgc.conf:/etc/wireguard/wg0.conf linuxserver/wireguard

And here the STORJ node run command:

docker run -d --restart=unless-stopped --stop-timeout 300 -e WALLET=“wallet” -e EMAIL=“email” -e ADRESS=“ADRESS:28967” -e STORAGE=“7.6TB” --name STORJ --net=container:wgc --mount type=bind,source=…

Can somebody help and find the mistake?

Thanks and kind regards,

arrogantrabbit · October 21, 2024, 4:10pm

You can’t dump incomplete set ofconfig files of an overly complicated setup and ask people to magically debug it in their heads for you.

You need to provide more information. Where is the server running? Is there a gateway firewall? Does your tunnel work in the first place? Can you ping the server from the client host? How about from node container? How do other iptables rules look like?

Describe all your troubleshooting steps up to this point and ask specific questions. “What’s wrong with my blob of config files” is not a specific question.

Alexey · October 22, 2024, 7:57am

Hello @Walter1,
Welcome back!

Do you have a public IP? If yes, stop right here. Please, use your public IP instead.

snorkel · October 22, 2024, 8:09am

Walter1 · October 22, 2024, 8:10am

It’s the one of ADRESS, I just removed it for data privacy. Did you find the mistake? Are the parameteres right in the linuxserver-Docker?

Alexey · October 22, 2024, 8:16am

Why do you need a VPN?