Write buffer size not set

I am running this command:

docker run -d --restart unless-stopped --stop-timeout 300
–sysctl net.ipv4.tcp_fastopen=3
-p 28967:28967/tcp
-p 28967:28967/udp
-p 127.0.0.1:14002:14002
-e WALLET=“WALLET ADDRESS”
-e EMAIL=“MYEMAIL”
-e ADDRESS=“SERVERIP:28967”
-e STORAGE=“3TB”
–user $(id -u):$(id -g)
–mount type=bind,source=“/root/.local/share/storj/identity/storagenode”,destination=/app/identity
–mount type=bind,source=“/var/local/storagenode”,destination=/app/config
–name storagenode storjlabs/storagenode:latest
–filestore.write-buffer-size 4MiB
–pieces.write-prealloc-size 4MiB

but when I actually check the settings in the container I still got :
cat config.yaml | grep “filestore”

filestore.write-buffer-size: 128.0 KiB

1 Like

Thanks for the command. Looks like you have invested already alot into the optimisation for STORJ-Dockers.

Can you explain what filesore and pieces do? An from what are they fetching the MiB? From the RAM?

Thanks and kind regards,

1 Like

This makes buffer in RAM 4MB for every connection you have, download piece and write in one chunk to hdd, default is 128k so it will write to temp folder till pice is written then copy it to needed location. but be carefully if you have node with 100 connections then it will be 400 MB reserved. so if you have lot of nodes make math.

But if I do in the new container now, cat config.yaml | grep “filestore” I get:

filestore.write-buffer-size: 128.0 KiB

Shouldn’t it be 4MiB?

It is your choice what it should be.

I want it to be 4MiB, and I used the command I quoted, which doesn’t seem to work, how can I set it into the config.yaml using docker run?
Thank you

doker is not my case, sorry.
I personaly would not use this one –pieces.write-prealloc-size 4MiB
need more information about it.
Write-buffer-size and write-prealloc-size explained - Node Operators - Storj Community Forum (official)
here is why, but I can be wrong.

1 Like

oh thanks Vadim, very precious information this one as well. So –filestore.write-buffer-size 4MiB can work without –pieces.write-prealloc-size?

They are independent things at all.
one is buffer size in RAM other, is how it allocated things on HDD

1 Like

ok it turns out that this command loaded config from the local permanent storagenode folder, so all I had to do is edit it directly in /var/local/storagenode, stop and remove container and re run it.

1 Like

@archimede91
What you set in docker run command, superseds the settings in config.yaml, for the running container, but dosen’t changes the config file. From the limited linux knowledge I have, I see that you check the setting in config.yaml, which is not changed, because you didn’t edited it with Notepad++ or Nano or something, and ofcourse you will get the default value. But the running container uses the corect value, 4MiB because it takes it from docker run.
BTW, you don’t have to edit anything in config.yaml for docker storagenodes. Just ignore it. Everything you want to set, put it in docker run command.

1 Like

I started a similar topic :smiley:

https://forum.storj.io/t/write-buffer-size-and-write-prealloc-size-explained/22230

as long as I use the bind in the run command it’s what is set actually as far as I understand.

It takes settings from config.yaml others than what it finds in docker run.

1 Like

There are two (technically three) ways of configuring your node for docker:

  • config.yaml
  • run parameters

If you specify something in run parameters, it will be used. If not, it will fallback to config.yaml for the value. If the value is not set in config.yaml either, some internal default will be used.

3 Likes

No, it should not. No one argument option will change the config.yaml file. It is generated only during setup (or it could be updated during migration to a new version).
The options provided as an arguments will override options from the config.yaml file during runtime only.

I found that changing write buffer size leads some sort memory leaks in 1.76.2. Containers begin to consume RAM like crazy, eating 6-8Gb each. Setting it to default solves this issue.

i think it because every connection take 4 MB, but is possible that is hdd connection is slow, is start to use ram, you use USB connection to HDD?

1 Like

Nope, I use RAID5 with BBU writeback enabled

Can the -e Parameters be removed in the docker run and then get fetched by the config.yaml?