Please enable TCP fastopen on your storage nodes

did I break it…
sudo docker logs --tail 20 storagenode
2023-08-12T04:07:22Z ERROR piecestore upload failed {“process”: “storagenode”, “Piece ID”: “Z6ABKYESDIH7TEY5E5MWFSYOBRKBEEPYWORHOESWLZVMXFLLCLGQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “error”: “context canceled”, “errorVerbose”: "context canceled\n\tstorj.io/common/rpc/rpcstatus.Wrap:75\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload.func6:500\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:506\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:243\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.

These are usual long tail cancelation errors, your node loosed the race for pieces. Please search for “tcp fast open” only.

I just tried to activate it if someone could confirm me

I have an error when I run the docker command, I don’t quite understand why…
Capture d’écran 2023-08-12 131114

And the commands below return nothing to me
$ netstat -s | grep FastOpen
$ netstat -s | grep TCPFast

Capture d’écran 2023-08-12 131537

/etc/sysctl.conf file

--sysctl option must be BEFORE the image name, since it’s a docker option, not the node.

1 Like

Correct, thank you


And aside from this line, is there any other way to confirm that the traffic is going through tcp fastopen?

If it isn’t, you will not see this line.

Yet nothing returns to me
Capture d’écran 2023-08-12 193923

1 Like

Debian with docker you have to start with 'docker run -d --restart unless-stopped --stop-timeout 300 --network host '

note the ‘–network host’ without TCPFastOpen doesn’t work. I think someone posted about the same thing for Synology.

I believe you mean the reverse - the TCPFastOpen feature doesn’t work without --network host on Debian?
Because --network host does work independently of TCPFastOpen, but you should be aware that switching to the host network exposes all ports of the container to the host.
In case of running multiple nodes you will need to change every port in the second and next nodes to unique ones, as if you were setup them as binary versions.

Same with me sorry to say. But some nodes return the message: TCPFastOpenCookieReqd: 16

when adding --network host i get this error
WARNING: Published ports are discarded when using host network mode
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: sysctl “net.ipv4.tcp_fastopen” not allowed in host network namespace: unknown.

docker run -d --restart unless-stopped --stop-timeout 300
-p 28967:28967/tcp
-p 28967:28967/udp
-p 14002:14002
-p 5999:5999
-e WALLET=“0xf82835c11e78a44C0F5176991b624D8635B8702C”
-e EMAIL=“@gmail.com"
-e ADDRESS="
*:28967”
-e STORAGE=“18.5TB”
–mount type=bind,source=“/mnt/storj2/storagenode-new”,destination=/app/identity
–mount type=bind,source=“/mnt/storj2/storagenode-new”,destination=/app/config
–log-opt max-size=10m
–log-opt max-file=5
–sysctl net.ipv4.tcp_fastopen=3
–network host
–name storagenode storjlabs/storagenode:latest
–server.address=“:28967”
–console.address=“:14002”
–debug.addr=“:5999”
–log.level=info
–filestore.write-buffer-size 4MiB
–pieces.write-prealloc-size 4MiB
–storage2.piece-scan-on-startup=true
–operator.wallet-features=zksync

You don’t need the sysctl command. Just use host network, and make sure fast tcp is enabled on the host, and use the host network.

2 Likes

so like this by removing line –sysctl net.ipv4.tcp_fastopen=3

$ netstat -s | grep FastOpen
TCPFastOpenPassive: 15
TCPFastOpenPassiveFail: 18
finally a result!

1 Like

You don’t need to specify those ports if you use network host.
Look at the last post in this topic:
https://forum.storj.io/t/my-docker-run-commands-for-multinodes-on-synology-nas/22034

Edit:

You may place the exact link to the post, this way future readers would not guess what’s post was the last on the time when you posted a link :slight_smile:

2 Likes

I tried on iPhone/Safari, but it just shows the link to the forum. The only way to link something is to long-press on the topic’s title and link the entire topic. I saw others linking the exact post, but maybe it works only on PC…

Click on
image
under the comment you want link to. And copy the string from a popup.

Then you can modify the URL manually if needed:

https://forum.storj.io/t/please-enable-tcp-fastopen-on-your-storage-nodes/22886/138?u=arrogantrabbit
                        └─── irrelevant text, can be edited or removed ──┘  │    │         │
                                                          topic ID ─────────┘    │         │
                                                          comment index ─────────┘         │
                             you, the fella who linked, for tracking and achievements ─────┘

That last /138 – is the comment index.

4 Likes

I know about that button; I don’t know what I was doing wrong, but now, after this state-of-the-art explanation, it works flowlessly. :sunglasses:
Thank you for +1 to my knowledge level.

3 Likes

After 20 days of enabling TCP fastopen, I don’t see any difference between traffic.
Synology1, TCPf off, 7.09TB - 314GB egress, 634GB ingress.
Synology2, TCPf on, 6.68TB - 303GB egress, 635GB ingress.

2 Likes