How to increase the UDP Receive Buffer Size in the docker

Node Operators
,
1

I also need that information.
I am running a Windows machine and I am running the StorJ node on Docker Desktop and would like to know how to increase the UDP Receive Buffer Size in the docker.

2

You need to configure it on your docker host

Since you use Docker desktop for Windows, it should be done in its VM.
If you use a docker desktop with Hyper-V, you need to get shell to this docker VM: Getting a Shell in the Docker for Windows Moby VM ¡ Bret Fisher
if you use a newest Docker desktop with wsl2 engine, you need just open a shell to the selected wsl2 distro, by default it has a name docker-desktop, so

wsl -d docker-desktop

will get you into the root shell.

3

I tried your suggestions, but i got this:

/ # echo “net.core.rmem_max=2500000” >> /etc/sysctl.conf
/bin/sh: can’t create /etc/sysctl.conf: Read-only file system

Where are these files stored, so i can modify them directly or if you know how I can get root access

4

You are on the root system. However the hack with a privileged container doesn’t work for files in the root filesystem, because it’s read-only for the DockerDesktopVM.
So this hack allows you to temporary change the value of runtime parameters, but not make them a persistent.
But if you can enable wsl2, you may upgrade the Docker Desktop for Windows and switch the docker engine to wsl2, this will allow you to change the root filesystem without using any hacks.
Or you may leave it as is, but QUIC wouldn’t be useful for these nodes.

5

thank you for the information.
can you provide me instructions on how to do the above?
and one more thing. I read somewhere in the StorJ documentation that it is not advisable to go above docker desktop version 2.1.0.5

image

If you say that this is no longer valid I can probably install the newest version. I am with Windows 10

6

It’s still valid for Hyper-V engine, but if you can enable wsl2, then better to use the latest version of the Docker desktop.

7

But is there a buffer size in relation the docker amount? When you run multiple nodes should it still be 2.500.000 or multiples of it?

grafik
grafik1898×54 4.69 KB

Thanks alot for the guide. It is working and the message disappeard. When you add the line in the /etc/sysctl.conf you can watch if with

cat /etc/sysctl.conf

and edit it with

nano /etc/sysctl.conf

Edit-1:

Can you explain why only rmem is beeing added? There is also wmem, with a similar command:

net.core.wmem_max = 2500000

Why does this not find an application?

Edit-2:

UnRAID by default has this configuration, maybe this is an inspiration as well?:

grafik
grafik847×146 3.91 KB

8

As far as I understand, it’s per process and per socket, it’s a maximum allowed value for each socket, and you should not scale it in case of multiple nodes.

because only receive buffer should be increased, not send buffer.

Could you please elaborate?

we do not offer a support for UnRaid, we support our software. But in general - each software may require own settings. For storagenode and QUIC only net.core.rmem_max should be changed.
If you would like to read more deeply about kernel options, you may Google them.

9

Thank you
it was hard but i managed to get it going
thank you for the help

1 Like
10

hi , it is working for me on a HyperV docker machine

with

docker run -it --rm --privileged --pid=host justincormack/nsenter1

then

sysctl -w net.core.rmem_max=2500000

but how can i make it persistent , i have tried many things but read-only error all over the place

1 Like
11

With Hyper-V engine it will not allow you to do so, only set it in runtime, as you did.
You can make it persistent only with wsl2 engine, since its filesystem is available for writes too, not only for read as with DockerDesktopVM (Hyper-V).

1 Like
12

Hi Again,

Today I rebooted the machine on which i am running Docker and deleted the old log file. However, now I again see the same error in the log file:

2023-03-16T09:31:32.373Z INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See UDP Receive Buffer Size · quic-go/quic-go Wiki · GitHub for details. {“Process”: “storagenode”}

And you can see that I applied the setting:

cat /etc/sysctl.conf

net.core.rmem_max=2500000

What is causing the error?

13

One additional comment. I get this as well:

sysctl -w net.core.rmem_max=2500000

/bin/sh: 12: sysctl: not found

14

In what VM you have run this command?
Is it a docker-desktop in wsl?
like

wsl -d docker-desktop
15

yes, it is a docker desktop

16

/etc/sysctl.d # sysctl -w net.core.rmem_max=2500000
sysctl: error: ‘net.core.rmem_max’ is an unknown key
/etc/sysctl.d #

17

Did you run it in a regular wsl, or after enter to the VM for docker-desktop with the command

wsl -d docker-desktop

I just tried

PS C:\Users\user> wsl -d docker-desktop
nodes:/tmp/docker-desktop-root/mnt/host/c/Users/user# sysctl -w net.core.rmem_max=2500000
net.core.rmem_max = 2500000

So, it should work

18

I tried executing this in CMD and PS and i get the same:

image

Like it is hung or something …
it doesn’t do anything. It is like that for more than 1 hour.

19

I see… Seems it become a mainstream after the latest update of the OS or the docker-desktop - wsl2 just hangs on any command, and I did not find a workaround so far except reboot, but this is not a solution.
Please try to do

wsl --update
20

Done:

image

now ?