Changelog v1.22.2

For Storage Nodes

zkSync Opt-In
We hope with the next payout we can offer sending payouts over zkSync. Storage Nodes can opt in to zkSync by adding operator.wallet-features: ["zksync"] to the config file. Please enter everything in lowercase and please also double-check for spelling mistakes. This is a very basic implementation without any validations.
Warning:
With zkSync the storage node payout is cheaper but if you want to send your payout to an L1 exchange you have to pay an L2 withdrawal fee. You can withdraw directly to the exchange in one transaction. Best case you can pay the L2 withdrawal fee in STORJ token. That should be even in fees compared to an L1 transaction that requires an ETH deposit first. However, currently, it is not possible to pay the L2 fee in STORJ tokens. We are working on resolving that issue. Since we now ask you to opt into zkSync before having clear information for you we also want to make sure you understand the worst-case scenario. Worst case (current situation) you need an additional ETH L2 deposit. For now please only opt-in if you are fine with the worst-case fee. See Test zkSync with TestSTORJ token for more details.
Do not use an exchange deposit address. The exchange will not recognize the payment. Since you don’t have the keys to that address you will not be able to withdraw to L1. Your only option would be an expensive emergency withdraw. Better make a free test run in testnet first. We are happy to send you testSTORJ token

Paid vs Distributed Payout
Storage node dashboard API api/heldamount/paystubs/2021-01 has a new field now. For each satellite it will show something like "paid":13488319,"distributed":0. In this example, the satellite distributed $0 to me and still owes me $13. In the following month I would expect to see something like "paid":10000000,"distributed":23488319. That would show that the satellite distributed the $13 from the previous month plus the payout for this month. We are working on making that visible on the storage node payout dashboard. That will take a few more releases. Meanwhile, we hope this step will allow you to keep track of your payout.
Note: We are aware that we have broken the held amount overview on the storage node dashboard because it can’t deal with this new field. That bug will also be fixed with this version.

QUIC
Satellite and storage nodes now support the QUIC protocol (UDP). Currently, uplinks still use TCP. We want to run a few internal tests with QUIC first. If the first test results are looking good we are going to ask you to also add a port forward rule for UDP (same port as for TCP). Don’t worry about it yet. We will communicate it when the time comes.

Multinode Dashboard
We have a very early test version of the multinode dashboard for you to try out. Please be aware that this first version is difficult to install. If you don’t mind a little challenge here is how you can get it running: [Tech Preview] Storage Node Multinode Dashboard

11 Likes

One of my docker nodes is now updated to the new V1.22.2, and I noticed a weird INFO line in my logs after starting it:

image

“INFO failed to sufficiently increase receive buffer size (was: 176 kiB, wanted: 2048 kiB, got: 352 kiB). See UDP Receive Buffer Size · lucas-clemente/quic-go Wiki · GitHub for more details”

Initially, I saw the message about UDP and remembered the post about changing port forwarding rules to “both” for TCP and UDP, which I did, although after stopping and restarting the node after the update, I still got that message.

so went over to this page, and found this description.

Could someone at storj confirm that the recommended maximum buffer size increase on the page is something that we should do on linux systems?

Edit: I went ahead and ran that command, and stopped, and started the node and didn’t see that message again. Although, now I’m curious, should we be changing our port forwarding rules to “both” after this update?

-oops, just re-read the changelog, says don’t worry about changing port forwarding protocol. Will it hurt anything if I just leave it as is?

1 Like

We (SNO) can update rules to support both - TCP and UDP, but it’s not mandatory.

1 Like

sounds like littleskunk don’t want to get SNO’s to do the UDP change in case its rolled back because of problems…

ofc… for some it might take a long while to implement…
would there be any advantage to getting udp opened already, or would it just expose nodes to possible issues?

Just be patient :grinning:

1 Like

I was curious about this, so did some extra reading. Looks like to make the change stick across reboots, an entry must be made in the /etc/sysctl.conf file by adding the line: net.core.rmem_max=2500000
(via this page)

1 Like

What about the nodes behind the CGNAT? Using a VPN will exclude nodes over UDP …

Why? You can forward UDP just fine.

In portmap.io only TCP or UDP, not not both at once

Sure, but that’s only a weird limitation of that service.

you simply add two entries for the same port, it’s common in some NAT configurations… depends a bit on the software

Where should be visible this information?
I don’t receive a payments anymore as well there are no information on what’s going on. Is there any place where i can see this ?

The Minimum Threshold for Storage Node Operator Payouts is applied, so your earnings is accumulated to the level when they would be in 4 times greater than a fee to transfer them.

This information you can see via storagenode’s API: http://localhost:14002/api/heldamount/paystubs/2021-01
To the dashboard UI it should be added in next releases.

would be nice if the notifications on the storagenode dashboard was working again…
worked or didn’t work for a while and then just stopped… now it never shows anything relevant…
it’s a real pity because it’s the obvious way to get into contract with the storagenode operators that are basically ghosts.

and can’t really be that hard to setup some site or database that the dashboard gets its notifications from so it’s easy to change and update important information… or simply just show recent version…

just imagine on all the time saved collectively from answering repeated questions by worried SNO’s
i know it seems trivial but it adds up…

3 Likes

Did I miss a change regarding garbage collection? There was like 50GB of trash on my node 3 or 4 days ago and now it’s gone. Did I miss having so much trash before or was the 7 days changed?

It turns out it was a brilliant idea to keep the trash for 7 days. We found a bug in the new multipart upload code. Saltlake testing stallite was sending out a bad bloom filter. To solve that problem we had to call restore from trash. So that is what you have seen on your storage node.

11 Likes

Would a restore call show up in log ?

1 Like

Not on the storage node side. We have logs about the successful execution on the satellite side including every storagenodeID

3 Likes

I just updated the earnings calculator to include this information.

2 Likes

Nothing better than a little backup in the back room. :stuck_out_tongue:

2 Likes