Changelog v1.28.2 (title has already been used)

littleskunk · April 28, 2021, 10:38pm

For Storage Nodes

Move Deleted Pieces to Trash

UDP Pingback
We haven’t finished our research around QUIC yet. At some point, all storage nodes need to forward UDP traffic. In a first step, the satellite will check if the UDP port is open. The outcome is already visible in the storage node log but not on the storage node dashboard. If you haven’t already please forward TCP and UDP traffic.

For Customers

New Branding, Pricing and Free Tier
We changed our branding. Tardigrade is now Storj DCS. We also updated the pricing and last but not least we have a free tier instead of a free trial now. If you haven’t tested our product already you now have the chance to test it for a longer period for free.

Satellite UI File Browser
The satellite UI now contains a file browser. You can upload and download files directly with your web browser. No additional tools needed. Please note this solution works with server side encryption. If you want to stick with client side encryption you should not use it. You can still choose between different existing integrations with client side encryption.

Toyoo · April 28, 2021, 10:47pm

What is the message we can grep for?

littleskunk · April 28, 2021, 10:51pm

Here is an example:

2021-04-28T22:46:23.823+0200    DEBUG   contact:endpoint        contact/endpoint.go:53  pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:46228"}
2021-04-28T22:46:24.557+0200    DEBUG   contact:endpoint        contact/endpoint.go:53  pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:56648"}

It is impossible to tell which one is which. Getting 2 pingbacks from the satellite means the storage nodes has noticed the TCP and the UDP pingback.

BrightSilence · April 29, 2021, 3:44am

Looks like the log level needs to be set to debug for that.

Pentium100 · April 29, 2021, 4:49am

How do I forward UDP port with docker?

kirbah · April 29, 2021, 4:53am

See answers here Experimenting with UDP based protocols

peem · April 29, 2021, 6:52am

What happens if SNO does not want to or cannot (e.g. because of VPN) enable UDP?

kirbah · April 29, 2021, 6:57am

Currently there is no difference if you open udp or not.
But who knows what will be in several months.

I think you will have some time when there is some difference.
Udp was added several months ago.

BrightSilence · April 29, 2021, 7:13am

It will fall back to TCP, but you’ll likely lose more races to be the fastest node as a result of the failure and then retry with a slower protocol. Though I doubt this will have a big impact any time soon as there will be plenty of nodes which don’t have UDP forwarded yet for a while.

peem · April 29, 2021, 7:36am

This will eliminate nodes via VPN, the successrate is already at 75%, if it drops even more, it will not make sense to keep the node, let alone buy an external IP only for Storj.

Now the question is - is it a deliberate VPN elimination or a coincidence (side effect)…?

BrightSilence · April 29, 2021, 8:01am

I think that’s jumping the gun a little. Lets see what the impact is first.

But I would like to point out that Storj Labs themselves suggests using a VPN to work around CGN from time to time. They are definitely not intentionally blocking it.

However, that doesn’t change the fact that extra hops slow down nodes and Storj’s objective is to speed up transfers for customers. Implementing QUIC (which uses UDP) is another one of those speed optimizations.

Yes, VPN nodes that can only forward TCP will have another disadvantage. There is no way around it. But if the other option is not implementing the speed improvements then you’re just out of luck.

That said, I’m pretty sure there are VPN’s that allow you to forward UDP as well.

hoarder · April 29, 2021, 8:04am

This deserves an email announcement.

Toyoo · April 29, 2021, 9:04am

Documentation needs an update too: Port Forwarding - Node Operator

SGC · April 29, 2021, 4:51pm

i wouldn’t exactly call this blatantly obvious… but i believe this means’s its on…

grep contact:endpoint sn1-2021-04-29.log
2021-04-29T09:37:40.708Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.189.66.130:42676"}
2021-04-29T09:37:40.772Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.222.168:39858"}
2021-04-29T09:37:41.073Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.221.25.52:47560"}
2021-04-29T09:37:41.139Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:54248"}
2021-04-29T09:37:41.479Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:34206"}
2021-04-29T09:37:41.790Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:55572"}
2021-04-29T10:37:40.727Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.197.226.85:39892"}
2021-04-29T10:37:40.814Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.36.232:43632"}
2021-04-29T10:37:41.172Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:48992"}
2021-04-29T10:37:41.364Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:43038"}
2021-04-29T10:37:41.614Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.238.72.85:42310"}
2021-04-29T10:37:42.583Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "107.167.187.38:36834"}
2021-04-29T11:37:40.796Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.189.66.130:50996"}
2021-04-29T11:37:40.838Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.222.168:55424"}
2021-04-29T11:37:41.132Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.221.46.106:37102"}
2021-04-29T11:37:41.159Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.231.222.201:58338"}
2021-04-29T11:37:41.489Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:36822"}
2021-04-29T11:37:41.936Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "107.167.187.38:35042"}
2021-04-29T12:07:39.340Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.36.232:46380"}
2021-04-29T12:07:39.372Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "34.76.106.129:49032"}
2021-04-29T12:07:39.703Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:41354"}
2021-04-29T12:07:39.870Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.221.25.52:34848"}
2021-04-29T12:07:39.881Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:54062"}
2021-04-29T12:07:40.498Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.220.160.211:53762"}
2021-04-29T13:07:39.346Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "34.107.106.254:48680"}
2021-04-29T13:07:39.475Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.223.134:39584"}
2021-04-29T13:07:39.707Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.236.192.71:53486"}
2021-04-29T13:07:39.723Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:37102"}
2021-04-29T13:07:40.550Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.220.160.211:33468"}
2021-04-29T13:07:40.580Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:32910"}
2021-04-29T14:07:39.390Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.189.119.130:35018"}
2021-04-29T14:07:39.705Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.199.22.252:42968"}
2021-04-29T14:07:39.784Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "34.74.113.54:57046"}
2021-04-29T14:07:39.975Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.222.168:47908"}
2021-04-29T14:07:40.033Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:60082"}
2021-04-29T14:07:40.518Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:58380"}
2021-04-29T15:07:39.438Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.36.232:52992"}
2021-04-29T15:07:39.559Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.234.111.65:39892"}
2021-04-29T15:07:39.758Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.236.192.71:54238"}
2021-04-29T15:07:39.790Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:56952"}
2021-04-29T15:07:40.088Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:57624"}
2021-04-29T15:07:40.777Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "104.155.206.12:47726"}
2021-04-29T16:07:39.461Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "34.76.106.129:57612"}
2021-04-29T16:07:39.503Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.222.168:46226"}
2021-04-29T16:07:39.808Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "34.72.236.162:51306"}
2021-04-29T16:07:39.941Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.199.22.252:48900"}
2021-04-29T16:07:40.134Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:53680"}
2021-04-29T16:07:40.999Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:57470"}

so decided to go take a look at the log directly… not much more illuminating…
then i had a bit better luck using the ip in your example…

grep '"srcAddr": "35.243.108.21' sn1-2021-04-29.log
2021-04-29T09:37:41.790Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:55572"}
2021-04-29T14:07:40.518Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:58380"}
2021-04-29T16:07:40.999Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:57470"}

seems a little better… but there seems to be one more than there is suppose to be…
when checking the same ip you was…

maybe it isn’t on and it’s because i haven’t fixed that yet… but yeah well next up … just wanted to give some raw first try feedback.

2021-04-29T09:37:40.491Z        INFO    failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 $

littleskunk · April 29, 2021, 5:06pm

SGC:

2021-04-29T09:37:40.708Z        DEBUG   contact:endpoint        pinged  {"by": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "srcAddr": "35.189.66.130:42676"}
2021-04-29T09:37:40.772Z        DEBUG   contact:endpoint        pinged  {"by": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "srcAddr": "35.228.222.168:39858"}
2021-04-29T09:37:41.073Z        DEBUG   contact:endpoint        pinged  {"by": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "srcAddr": "35.221.25.52:47560"}
2021-04-29T09:37:41.139Z        DEBUG   contact:endpoint        pinged  {"by": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "srcAddr": "35.224.23.248:54248"}
2021-04-29T09:37:41.479Z        DEBUG   contact:endpoint        pinged  {"by": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "srcAddr": "35.236.98.215:34206"}
2021-04-29T09:37:41.790Z        DEBUG   contact:endpoint        pinged  {"by": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "srcAddr": "35.243.108.21:55572"}

This means UDP port is closed. You get one ping per satellite. You need 2 pings per satellite. Keep an eye on the timestamp.

Tony_boy_32 · April 30, 2021, 6:56am

On Windows,

In windows via Docker,
I get the following

2021-04-30T06:49:25.410Z INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See UDP Receive Buffer Size · lucas-clemente/quic-go Wiki · GitHub for details.

Does anyone know the procedure to fix this error?

I saw the link indicated, but I can’t resolve the error as the guide talks about Linux.

Best regards

Tony_boy_32 · April 30, 2021, 9:44am

Thanks for the reply, I still await some other opinion before trying.
Thanks so much.

Storgeez · April 30, 2021, 9:59am

This will probably be announced soon by Storj.

sorry2xs · April 30, 2021, 10:00am

good morning tony_boy tried it get this error, PS C:\Users\sorry> New-ItemProperty -Path $registryPath -Name $drw -Value $v_drw -PropertyType DWORD -Force | Out-Null

$v_drw = “2097152”
$drw = “DefaultReceiveWindow”
$registryPath = “HKLM:\SYSTEM\CurrentControlSet\services\AFD\Parameters”
New-ItemProperty : Cannot bind argument to parameter ‘Path’ because it is null.
At line:1 char:24

New-ItemProperty -Path $registryPath -Name $drw -Value $v_drw -Proper …
```
                   ~~~~~~~~~~~~~
```
- CategoryInfo : InvalidData: ( [New-ItemProperty], ParameterBindingValidationException
- FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.NewItemProp
  ertyCommand

PS C:\Users\sorry>

naxbc · April 30, 2021, 4:24pm

What UDP ports should be forwarded?