VPN port forwarding for Pi nodes?

MarkFeuer · July 9, 2022, 8:20am

Hey all,

Recently moved cross-country into a new apartment with community internet. I just spent all night trying to get my node back up and running with a new Raspberry Pi setup, only to discover that the ISP blocks port forwarding on the community internet.

I came across this thread where Alexey mentions options to get around port forwarding blocks with VPN. PIA, portmap.io , purevpn, ngrok all come up as options.

Has anyone running a Pi node/Linux setup tested one of these options, and is there a good process to setting it up?
I’m leaning towards Portmap, but Ngrok seems like it would be more compatible with the Pi. Eyeballing a setup guide here. Can anyone confirm if this would work?
Would I still need my No-IP DDNS hostname to use NGrok or Portmap?

Thank you in advance for any help available!

Bivvo · July 9, 2022, 12:00pm

Try VPS + proxy forwarding instead of a VPN.

I wanted to try that, but not yet invested time into it.

It might be cheaper and hopefully has a better performance on the long run.

MarkFeuer · July 9, 2022, 7:05pm

@Bivvo do you have a good software recommendation or guide link that I could use to attempt that?

Portmap does have a free option, too

Bivvo · July 9, 2022, 7:11pm

I’ve received the following tips, but have not tried myself:

£1 at Ionos.co.uk

Install simpleproxy directly on the VPS

Then set it to proxy port 28967 to your local server which is running a SN on IP 11.22.33.44 port 10001, and remember to put the VPS IP into the SN config file on the SN running on 11.22.33.44.
simpleproxy -L 28967 -R 11.22.33.44:10001

Don’t know if that works with static IPs only or with dyndns, too.

I’ve tried portmap, but there was some limitation - can’t remember exactly, if it was QUICK, TCP+UDP (you need both) or sth else.

Knowledge · July 9, 2022, 7:20pm

We do have a guide for PIA VPN

Bivvo · July 9, 2022, 7:26pm

VPS might be cheaper than VPN (min 1.8€, up to 3-5€) plus DYNDNS (min 1.99€) monthly fees.

But as mentioned, I’ve not yet tried the VPS solution described above

Knowledge · July 9, 2022, 7:36pm

Yes, I was just providing an answer to the original question. In the meantime I will setup one of my nodes with your suggestion Bivvo and if it works, I’ll write up some documentation for it and include it with the rest. Much appreciated!

Bivvo · July 9, 2022, 7:48pm

That would be really great and helpful.

Likewise.

lyoth · July 9, 2022, 10:15pm

I have this setup working on my setup all on docker.
I created my own docker to run my openvpn client inside with duckdns.org as my ddns and it updates every 5 minutes in case the vpn disconnect and reconnect to a different server.

You need to make sure your vpn provider support port forwarding.

I also tested this on a friend asus router that can run an openvpn server and that works as well.

When the vpn docker is up, you connect your storj docker container using

--net=container:storagenode-vpn

MarkFeuer · July 11, 2022, 2:07am

@lyoth does your setup require a paid service or are you running it all on free software?

I’m sort of hell-bent on avoiding paying for another service for this, to ensure it stays economically feasible, so I was trying to think of using the free version of Portmap for UDP and the free version of NGrok for TCP.

Alexey · July 11, 2022, 8:19am

This will not work. You need to have the same external IP for both - TCP and UDP. In case of using different VPN providers you will have different external IPs.
So, if you want to use free versions, you will be forced to stick only to TCP, the UDP will remain misconfigured then.

MarkFeuer · July 11, 2022, 2:49pm

Ah, darn - I thought my No-IP DDNS would have been able to alleviate using the different clients.

In that case, I may go the route of trying the PIA walkthrough that was attached above. Since it’s an 8-terabyte drive, I think it will be able to make up the cost.

Are there instructions for installing the PIA client on a Pi as well? It looks like the guide focuses mainly on running the Windows version.

Bivvo · July 11, 2022, 4:13pm

You could try or wait for the VPS solution, too. Just wanted to throw it up. Might be cheaper in the long run. Up to you.

lyoth · July 12, 2022, 5:32pm

I recommend you use a PIA trial to see all the IP that PIA have.
Use this tool to check how many nodes are on those subnet Neighbors
No point in using PIA if there are like 4+ nodes on their subnet.
You just need to install an openvpn client on your PI, and use PIA openvpn file on it.

About your point on having a free solution, if you know a friend who has a public IP, and they let you host your vpn, you can do it that way.
I have one of my nodes connected to a friend Asus router that can run an openvpn server.

MarkFeuer · July 26, 2022, 6:41pm

Did anyone have any luck with the VPS option? I am leaning toward trying the public IP option at this point.

Knowledge · July 28, 2022, 2:14am

Not yet. I setup a VPS with IONOS but am still working out the connectivity. Need to have the node connect to the VPS and then forward the port down to the node. Working on it.

Krawi · July 28, 2022, 9:20am

Here is my setup with Wireguard and VPS for two nodes on ports 28867 & 28868

It works in general but has a small issue when the container starts, it got stuck when trying to connect to github while the VPN is active.

Workaround:

Stop & Delete container
Stop VPN
Start container
Wait for finished startup and downloded binaries
Start VPN

From Google research
https://www.cyberciti.biz/faq/debian-10-set-up-wireguard-vpn-server/
https://www.cyberciti.biz/faq/how-to-set-up-wireguard-firewall-rules-in-linux/
https://golb.hplar.ch/2019/01/expose-server-vpn.html

Client Config
File: /etc/wireguard/wg0.conf

[Interface]
## This Desktop/client's private key ##
PrivateKey          = <Key>

## Client private IP address ##
Address             = <Client private VPN IP>/24

#DNS                 = 1.1.1.1
#DNS                 = 8.8.8.8
DNS                 = <my local DNS server>

[Peer]
## server public key ##
PublicKey           = <Server public key>
## Preshared Key from server ##
PresharedKey        = <Preshared key>

## set ACL ##
#################################################
## Allow remote server as gateway
## Otherwise client won't show server's IP
#################################################
AllowedIPs          = 0.0.0.0/0

## server's public IPv4/IPv6 address and port ##
Endpoint            = <Server public IP>:51820

##  Key connection alive ##
PersistentKeepalive = 20

Server Config

UFW Firewall Entries (Not sure if they are really needed after the iptables entries were made in the script for startup VPN)

# Node 1
ufw route allow proto tcp to <Client private VPN IP> port 28867
ufw route allow proto udp to <Client private VPN IP> port 28867
# Node 2
ufw route allow proto tcp to <Client private VPN IP> port 28868
ufw route allow proto udp to <Client private VPN IP> port 28868

Wireguard Config
File: /etc/wireguard/wg0.conf

[Interface]
Address = <Server private VPN IP>/24
SaveConfig = true
PostUp = /etc/wireguard/helper/add-nat-routing.sh
PostDown = /etc/wireguard/helper/remove-nat-routing.sh
ListenPort = 51820
PrivateKey = <Server private key>

[Peer]
PublicKey = <Client public key>
PresharedKey = <Preshared key>
AllowedIPs = <Client private VPN IP>/32

File: /etc/wireguard/helper/add-nat-routing.sh

#!/bin/bash
IPT="/sbin/iptables"
IPT6="/sbin/ip6tables"

IN_IF="eth0"                 # NIC connected to the internet
WG_IF="wg0"                  # WG NIC
WG_SUBNET="<Private Subnet>.0/24" # WG IPv4 sub/net aka CIDR
WG_GW="<Server private VPN IP>"        # WG IPv4 Gateway, Server private VPN IP
WG_PORT="51820"              # WG udp port
#WG_SUBNET_6="fd42:42:42:42::/112" # WG IPv6 sub/net
HOST01="<Client private VPN IP>"      # Host pi-vpn IPv4
HOST01_PORT01="28867"        # Host pi-vpn Storj Port 1
HOST01_PORT02="28868"        # Host pi-vpn Storj Port 2

## IPv4 Rules ##
$IPT -t nat -I POSTROUTING 1 -s $WG_SUBNET -o $IN_IF -j MASQUERADE
$IPT -I INPUT 1 -i $WG_IF -j ACCEPT
$IPT -I FORWARD 1 -i $IN_IF -o $WG_IF -j ACCEPT
$IPT -I FORWARD 1 -i $WG_IF -o $IN_IF -j ACCEPT
$IPT -I INPUT 1 -i $IN_IF -p udp --dport $WG_PORT -j ACCEPT

# Node 1
$IPT -A FORWARD -i $IN_IF -o $WG_IF -p tcp --syn --dport $HOST01_PORT01 -m conntrack --ctstate NEW -j ACCEPT
$IPT -A FORWARD -i $IN_IF -o $WG_IF -p udp       --dport $HOST01_PORT01 -m conntrack --ctstate NEW -j ACCEPT
# Node 2
$IPT -A FORWARD -i $IN_IF -o $WG_IF -p tcp --syn --dport $HOST01_PORT02 -m conntrack --ctstate NEW -j ACCEPT
$IPT -A FORWARD -i $IN_IF -o $WG_IF -p udp       --dport $HOST01_PORT02 -m conntrack --ctstate NEW -j ACCEPT

$IPT -A FORWARD -i $IN_IF -o $WG_IF -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
$IPT -A FORWARD -i $WG_IF -o $IN_IF -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Node 1
$IPT -t nat -A PREROUTING -i $IN_IF -p tcp --dport $HOST01_PORT01 -j DNAT --to-destination $HOST01
$IPT -t nat -A PREROUTING -i $IN_IF -p udp --dport $HOST01_PORT01 -j DNAT --to-destination $HOST01
# Node 2
$IPT -t nat -A PREROUTING -i $IN_IF -p tcp --dport $HOST01_PORT02 -j DNAT --to-destination $HOST01
$IPT -t nat -A PREROUTING -i $IN_IF -p udp --dport $HOST01_PORT02 -j DNAT --to-destination $HOST01

# Node 1
$IPT -t nat -A POSTROUTING -o $WG_IF -p tcp --dport $HOST01_PORT01 -d $HOST01 -j SNAT --to-source $WG_GW
$IPT -t nat -A POSTROUTING -o $WG_IF -p udp --dport $HOST01_PORT01 -d $HOST01 -j SNAT --to-source $WG_GW
# Node 2
$IPT -t nat -A POSTROUTING -o $WG_IF -p tcp --dport $HOST01_PORT02 -d $HOST01 -j SNAT --to-source $WG_GW
$IPT -t nat -A POSTROUTING -o $WG_IF -p udp --dport $HOST01_PORT02 -d $HOST01 -j SNAT --to-source $WG_GW

## IPv6 (Uncomment) ##
## $IPT6 -t nat -I POSTROUTING 1 -s $WG_SUBNET_6 -o $IN_IF -j MASQUERADE
## $IPT6 -I INPUT 1 -i $WG_IF -j ACCEPT
## $IPT6 -I FORWARD 1 -i $IN_IF -o $WG_IF -j ACCEPT
## $IPT6 -I FORWARD 1 -i $WG_IF -o $IN_IF -j ACCEPT

File: /etc/wireguard/helper/remove-nat-routing.sh

#!/bin/bash
IPT="/sbin/iptables"
IPT6="/sbin/ip6tables"

IN_IF="eth0"                 # NIC connected to the internet
WG_IF="wg0"                  # WG NIC
WG_SUBNET="<Private Subnet>.0/24" # WG IPv4 sub/net aka CIDR
WG_GW="<Server private VPN IP>"        # WG IPv4 Gateway, Server private VPN IP
WG_PORT="51820"              # WG udp port
#WG_SUBNET_6="fd42:42:42:42::/112" # WG IPv6 sub/net
HOST01="<Client private VPN IP>"      # Host pi-vpn IPv4
HOST01_PORT01="28867"        # Host pi-vpn Storj Port 1
HOST01_PORT02="28868"        # Host pi-vpn Storj Port 2

## IPv4 Rules ##
$IPT -t nat -D POSTROUTING -s $WG_SUBNET -o $IN_IF -j MASQUERADE
$IPT -D INPUT -i $WG_IF -j ACCEPT
$IPT -D FORWARD -i $IN_IF -o $WG_IF -j ACCEPT
$IPT -D FORWARD -i $WG_IF -o $IN_IF -j ACCEPT
$IPT -D INPUT -i $IN_IF -p udp --dport $WG_PORT -j ACCEPT

# Node 1
$IPT -D FORWARD -i $IN_IF -o $WG_IF -p tcp --syn --dport $HOST01_PORT01 -m conntrack --ctstate NEW -j ACCEPT
$IPT -D FORWARD -i $IN_IF -o $WG_IF -p udp       --dport $HOST01_PORT01 -m conntrack --ctstate NEW -j ACCEPT
# Node 2
$IPT -D FORWARD -i $IN_IF -o $WG_IF -p tcp --syn --dport $HOST01_PORT02 -m conntrack --ctstate NEW -j ACCEPT
$IPT -D FORWARD -i $IN_IF -o $WG_IF -p udp       --dport $HOST01_PORT02 -m conntrack --ctstate NEW -j ACCEPT

$IPT -D FORWARD -i $IN_IF -o $WG_IF -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
$IPT -D FORWARD -i $WG_IF -o $IN_IF -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Node 1
$IPT -t nat -D PREROUTING -i $IN_IF -p tcp --dport $HOST01_PORT01 -j DNAT --to-destination $HOST01
$IPT -t nat -D PREROUTING -i $IN_IF -p udp --dport $HOST01_PORT01 -j DNAT --to-destination $HOST01
# Node 2
$IPT -t nat -D PREROUTING -i $IN_IF -p tcp --dport $HOST01_PORT02 -j DNAT --to-destination $HOST01
$IPT -t nat -D PREROUTING -i $IN_IF -p udp --dport $HOST01_PORT02 -j DNAT --to-destination $HOST01

# Node 1
$IPT -t nat -D POSTROUTING -o $WG_IF -p tcp --dport $HOST01_PORT01 -d $HOST01 -j SNAT --to-source $WG_GW
$IPT -t nat -D POSTROUTING -o $WG_IF -p udp --dport $HOST01_PORT01 -d $HOST01 -j SNAT --to-source $WG_GW
# Node 2
$IPT -t nat -D POSTROUTING -o $WG_IF -p tcp --dport $HOST01_PORT02 -d $HOST01 -j SNAT --to-source $WG_GW
$IPT -t nat -D POSTROUTING -o $WG_IF -p udp --dport $HOST01_PORT02 -d $HOST01 -j SNAT --to-source $WG_GW

# IPv6 rules (uncomment) #
## $IPT6 -t nat -D POSTROUTING -s $WG_SUBNET_6 -o $IN_IF -j MASQUERADE
## $IPT6 -D INPUT -i $WG_IF -j ACCEPT
## $IPT6 -D FORWARD -i $IN_IF -o $WG_IF -j ACCEPT
## $IPT6 -D FORWARD -i $WG_IF -o $IN_IF -j ACCEPT

Commands
systemctl start wg-quick@wg0
systemctl stop wg-quick@wg0

wg show
=>Shows connection information

Perhaps needed if name resolution don’t work:
ln -s /usr/bin/resolvectl /usr/local/bin/resolvconf
systemctl enable systemd-resolved.service
systemctl start systemd-resolved.service

Knowledge · July 28, 2022, 2:00pm

Thanks, this will come in handy.

MarkFeuer · July 29, 2022, 11:13am

@Krawi and @Knowledge - Thank you both for continuing to talk with me on this subject. Krawi’s documentation looks very informative.

I just had a thought as well, now that you mentioned Wireguard!

I actually also run a Mysterium VPN node with a second Raspberry Pi at my current location and earn MYST crypto which allows me to use their VPN service essentially for free. Their service is built on Wireguard as well and they have some documentation on NAT traversal and port forwarding for UDP. Port forwarding

Do you think it would be possible to utilize the Myst VPN to run my Storj node through?

Krawi · July 29, 2022, 1:18pm

I didn’t find an option to enable port forwarding on the backend side of Myst VPN, so i think you can’t expose your storj node through this VPN.

The link about forwarding is for setting up your router for your Myst node, similar to the setup for a Storj node.