UDP QUIC configuration guide for linux / docker

How to test if your Quic UDP network configuration is correctly.
http://storjnet.info/ping_my_node

Run the following command on the host to change network protocol settings.
makes it persistent across reboots

echo "net.core.rmem_max=2500000" >> /etc/sysctl.conf

sysctl -w net.core.rmem_max=2500000

changes the udp buffer size without reboot
and can be used together
You will also need to forward ports in your router, but i will not touch on that since that part will be unique to most users.

finally you will need to change your docker run command.

not sure if i’m leaving anything out, but i don’t think so…
wanted to put all the pieces in one place, and in what i consider the simplest and easy to understand way.

2 Likes

Have done this yet today, so here are some tips.

I wasn’t sure the sysctl -w will survive the next boot, so searched for a permanent change and found this
https://www.tecchannel.de/a/tcp-ip-tuning-fuer-linux,429773,6.

It’s a german site, so in short:

Edit /etc/sysctl.conf and add these lines:

# increase Linux TCP buffer limits
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152

# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
net.ipv4.tcp_rmem = 4096 87380 2097152
net.ipv4.tcp_wmem = 4096 65536 2097152

I changed the 2097152 (2MB) to 2500000, the other 3 parameters seems to be relating, so i added them too.

For immediately taking effect without reboot, use
sysctl -p

Another site about performance tuning of the network i discovered, but not yet tried:
https://www.ibm.com/docs/it/smpi/10.2?topic=mpi-tuning-your-linux-system

Have you checked? Is it permanent?
Do you think so? (as usual you are used to)

sysctl --help

IBM documentation usually has some excellent tips and tricks, and very well documented about how and why.

i already optimized my network setting a long time ago, but generally what i learned from that rabbit hole was that there are way to many bad suggestions about changing network setting for optimizing…

so my recommendation would be try to stick close to the defaults, if you like 10x or 100x a buffer sizes you might end up having problems emptying the buffer and stuff might sit for a long time, which can lead to all sorts of issues.

so do take all the optimizing suggestions online with a grain of salt, unless if you are learning about networking and want to learn the hard way…
the default are adopted because they work fine for most general purpose use cases.
and tho you might think you see an improvement short term from optimizing, its almost always at the cost of something else.

i’m sure what you used here is fine…

i saw somebody else said -w on sysctl would make it persistent, but i’m really unsure i did check
sysctl --help
which also seemed to indicate that to be the case… on top of that it’s a non critical thing, so even if it fails it doesn’t hurt anything really… so meh…

anyways since you doubted it i went and dug into the linux manual as suggest by the
sysctl --help
but as usual it’s futile because it has it’s own language… i mean how does this tell me if it is persistent or not…

-w, --write
Use this option when all arguments prescribe a key to be set.

if you do decided to test it out do let us know, if its persistent or not…
don’t really find the manual or --help very illuminating even if they sort of do indicate it without making it perfectly clear imo.

It is not persistent, it only lasts until shutdown.

It is just as @Krawi wrote:

enabling writing of a value to a variable

which seems again doesn’t tell you if that is persistent or temporary…
ofc since we can change variables without using -w then one can deduce that it most likely means that it’s persistent…

i just don’t think it’s written in a way that cannot be misunderstood or requires prior knowledge…
like say if i didn’t know that i could have variables without using -w…
then how would i know…
but just me mincing words lol

because i hate the linux manual, i so rarely feel helped by it.

Didn’t you feel like rebooting the system to check?

1 Like

hell no, its a soft failure even if it doesn’t work…
barely changes anything…everything will work…

the storagenode at worst gets slightly more latency.
so i can deal with that if it turns out to be a problem

… rebooting my server is a bother…

man sysctl info about the -w:

       -w, --write
          Use this option when all arguments prescribe a key to be set.

Nothing about the destination for this action, is it memory, a file, a piece of rock, wood, steel or the surface of the next SpaceX rocket?

2 Likes

But you’re writing the manual: " [UDP QUIC configuration guide for linux / docker]"

and you mislead others …

1 Like

it will work… else i can correct it
i did consider doing an echo whatever into whatever sysctl or a sysctludp.conf
or something… but just made everything so much more difficult, this was simple and i’m sure somebody will complain if it doesn’t work.

and the --help sort of says it works, other people said it would make it persistent.
and its a soft failure… can’t really go wrong

i’m not trying to mislead anyone, now since you are aware of it and ain’t testing it, that just makes you just as bad as i am… :smiley:

i believe it will work with a high degree of certainty else i wouldn’t have used it.
and it’s not like it’s on print if it was wrong.

@SGC, we’ve established that sysctl -w does not persist across reboots, regardless of how you deduced that it should. Please update the top post to include an entry in /etc/sysctl.conf or /etc/sysctl.d to prevent confusion.

4 Likes

Or change the title to “I believe this will work UDP QUIC configuration for linux / docker”

thanks for clearing that up. fixed it, kept the sysctl -w net.core.rmem_max=2500000
as it fixed the buffer issue for my system without a reboot…

you got any idea what the -w actually does then?
i mean i guess it could be why i didn’t need to do the sysctl -p or reboot after changing the /etc/sysctl.conf, sometimes i just wish that linux was so much better documented.

thanks for being a troll, having no knowledge to contribute and no intention of helping at all.
keep up the good work… lol

This is guaranteed for you :slight_smile:
Call it what you want - I achieved the goal - you corrected the nonsense you wrote in the first version.

Look what I found somwhere on internet:

Modify Kernel parameter in /etc/sysctl.conf for permanent change

After modifying the kernel parameter in the /etc/sysctl.conf, execute sysctl –p to commit the changes. The changes will still be there after the reboot.

vi /etc/sysctl.conf # sysctl –p

Modify kernel parameter temporarily

To temporarily modify a kernel parameter, execute the following command. Please note that after reboot these changes will be lost.

sysctl –w {variable-name=value}

So the -w parameter just executes it immediately, but temporarily
You did a good job making this post :+1:
At least someone inform the masses about it

2 Likes

So, should we all be making this change now?

I mean, last I heard about QUIC was that it was still being tested and we shouldn’t do anything yet?

Ah? Never read that, must have missed it ^^’
I did reconfigure my nodes and all seems fine for now.
My take is that it doesn’t hurt to make the switch: you’ll be future ready for when it starts being massively used by satellites :slight_smile:

That was said in a previous changelog indeed. But there has been more recent communication saying we should probably start thinking about enabling it. It’s definitely not essential yet, but best be prepared I guess.

To be honest, I don’t think you have to bother with increasing the UDP buffer. But it can’t hurt either.

1 Like