QUIC stop working after SYNOLOGY DSM update

Hello community,

I have properly configured my nodes (on Windows and under Docker on SYNOLOGY NAS).

I have updated SYNOLOGY DSM system. Everything went fine, docker operates properly and nodes under docker as well. Node on Virtual Machine (WINDOWS10) works properly.

Only issue which I have is QUIC looks uncofigured. I tried re-add one node but still is the same:

I did not change anything on router and this works properly for all nodes from this NAS:

This is how I am configuring or re-create node if needed:

For other nodes I have adjusted ports and paths etc…

This QUIC issue is there for all nodes running this SYNOLOGY NAS after update.

Any ideas what can be wrong? Do I have to just wait longer?

Thank you

Wigo

From Docker log:

What is your version ?
on Synology DS918+ - DSM 6.2.4-25556 Update 5
works fine

Do you have configured Firewall on Synology

HI Krystof,

Synology DS1817+ DSM 7.1-42661 Update 2 which is latest update. Works till today´s update.
Firewall is turned off on Synology. I had to play with firewall only on Windows node but there is no issue with QUIC.

This is how docker config from DSM GUI looks like:

Hi LordWigo,
that would be a big coincidence, but is it possible that another process is listening on the NAS on port 28967/UDP after the synology update?

Not sure if the docker log would write something like “can not bind/forward to 28967/UDP” when starting the container.

On my QNAP NAS (sure Synology too) I can check via SSH with the following command whether the port is open:

lsof -iUDP | grep *:28967

If the command does not provide any output, the port can be used. If an output appears you can see which process already has the port in use. For me it’s dockerd, of course.

dockerd 8244 admin 54u IPv4 34566387 0t0 UDP *:28967

best regards
Vossi

I had the same problem previously with my Synology. In the end it helped me to delete the ports in the router and create them again. It helped, but I can’t explain it

Hi Frieseba,

I have tried this and reboot router and also SYNOLOGY NAS but no change unfortunately :frowning_face:

Thanks for tip anyway :slight_smile:

Hi Vossi,

I have tried via SSH but not very advanced in this (unix?) commands:

I have run more docker cointainers and there must by configured ports like this:

So for each storage node/container port must be 28967 but local port differ for each container:

Strange is that this works perfectly. Going to search how to list assigned ports on SYNOLOGY.

Thank you for the guess.

Wigo

I found this:

Not sure if this what I am looking for?

Wigo

I have to agree with others - try to delete rule on your router for UDP, then add it back. Make sure that the destination IP is match the IP of your NAS (if your router cannot address it by name for example).
Please also try to re-create a container on the NAS.
It’s worth to check the firewall on Synology - maybe they enabled it automatically during upgrade. In this case just create an inbound rules to allow TCP and UDP ports of your nodes.

Hi Alexey,

Firewall on NAS is turned off:

I have already recreated all nodes (docker containers) but did not helped.

Strange is that node running on SYNOLOGY (alongside other nodes) has this issue:

Router settings:

But Node running on Windows behind the same router is perfectly fine:

Router settings:

I have of course tried recreate all port forwarding rules on Router for SYNOLOGY but without success :frowning:

Wigo

Please try to re-create a container on your Synology after you re-created a rule.

Hi Alexey,

I have recreated container after re-creation rule on router. Restarted router as well.

…but no luck:

You showed the rule for 28968 which you supposedly re-created, and then re-created a container with 28967.
Perhaps you confused something?

Hi Alexey,

Port 28968 in my previous posts is allocated for my node running on Windows to demonstrate that other node running different machine are running fine behind the same router.

Ports 28967 and 28969 - 28974 assigned to all nodes running on SYNOLOGY NAS (under docker) has this issue only:

So all these 7 nodes has this QUIC issue.

One node running on Windows (28968) works without issues.

I have created different VM windows machine hosted on SYNOLOGY NAS (different IP than NAS itself) just for test.

Create rule on router:

And this node running also without issues:

Hi LordWigo,
the first thing I see is, your router seems to use/link the forwarding rule with the MAC Address(The Dropdown behind the Internal Host inputfield).

Is it possible the MAC-Address of your NAS has changed after upgrade? Normaly the MAC Address is branded to your Ethernet Adapter Chip, but most NAS has two Adapters and create a virtual MAC Address or uses the MAC Address from one physical member of a network team.

Try to ping the ip from your StorJ container:

ping 10.0.0.200

after you recived your ICMP echo replies, type

arp -a

search the line with the IP 10.0.0.200, the MAC Address will be shown in the same line.
That Mac Address must match in your forwarding rule on the router.

With reference to the check for listening ports … netstat was my first choice, but not working on my QNAP NAS. :smiley:
Try

netstat -nau
-u for UDP Ports
-t for TCP Ports

I am not sure but in the Screenshot of your output many TIME_WAITS can be seen. Looks like an misconfiguration. I belive 172.17.0.1 try close Connections to 172.17.0.4 and 172.17.0.7 and waiting for the ACK to close the connection.

Hi Vossi,

My MAC match with NAS:

Router:

Network settings of Docker:

I think if there would be issue with MAC address I would not have Online status for my Nodes at all?

I am trying to run ping and other commands from container terminal but with no luck:

Tried some bash script for ping:

Ping via SSH works but this is probably just from the same IP (10.0.0.200) no from docker container:

image

Output of netstat -nau:

Here sample of output netstat -t:

Here output of netstat -u:

So do you suggest to kill somehow kill those listen/waiting connections?

Hi LordWigo,

OK, that’s what I wanted to hear. :wink:

That’s what I thought when I wrote it. But then I thought it would be possible that the node display online, if he can establish a connection to a external host.

In the image “Network settings of Docker” it seems as docker is running in bridge mode. I assume that your internetrouter has the IP 172.17.0.1. If your Docker was really running in bridge mode, shouldn’t the containers also have an IP from the network 172.17.0.0/16 and not from the network 10.0.0.0/8?

But then nothing should work. :woozy_face:

Seems, there is no ping command in the image.

Thats do your tcpdriver for you on the NAS. But this trys will never end because these tcp packets flow to your NAS.
They will keep coming because 172.17.0.1 forwards incoming tcp packets on port 28967 to 172.17.0.3, 172.17.0.6 and 172.17.0.8 (as far as I can see). But they close the connection.

Can you try to run one of these Container with a ip in the subnet 172.17.0.0/16?
Or try set the Networktype to NAT instead of bridge.
NAT Network should be 10.0.0.0/8 or /24 and the Container should have the IP 10.0.0.200

best regards and GN8
Vossi

Hi CutieePie,

Well but this works without change before and no problems.

Port 28980 is used already for node running on windows PC:

Thah one is OK.

This is how I have configured all running nodes on my NAS:

image

So what do you suggest is run every node on different port?

Example:

STORJ_07: -p 28974:28974/tcp -p 28974:28974/udp -p 14008:14002

instead of: STORJ_07: -p 28974:28967/tcp -p 28974:28967/udp -p 14008:14002

Thank you

Wigo