QUIC was working, now not - a few booboo's on the way

aardvarkl · December 20, 2022, 4:39pm

[As background: My Storj node is running as a Container on TrueNAS Scale]

So - I deleted my Storj Container. It was one of those brain fart moments when I thought I was deleting another container, but clicked on StorJ instead.
Oh Poo, time to test my backups - which needless to say didn’t work (actualy I suspected this would be the case anyway)

OK - I didn’t delete the off container storage which contains identity and node storage - so all I have to do is recreate the container. I have a note of ID etc

So I have recreated the container and its working again. Except that now its says that QUIC is misconfigured. I also noticed a new update which probably happened as part of the container recreate.

Not sure why…

Some more info. I actually have two small nodes, fairly new. 1 on a Synology NAS and 1 on TrueNAS Scale. They both use the same WAN IP address, but different ports, and different local IP addresses
Node 1 (Synology) uses 28967 → 28967
Node 2 (TrueNAS) uses 20988 → 20988 which is defined in the container config.
WAN forwards (Opensense) are TCP/UDP. DNS is correct. Firewall rules haven’t changed.

When I click on Misconfigured it pops up a message saying I must use 28967, which I obviously can’t as its in use for the other Node. The other Node is NOT complaining about QUIC

The only thing I can think of is that there has been a change in the app as I hadn’t updated which is forcing 28967 for some reason. Maybe

Any ideas?
Logs in the next message. The only errors I can see are:

2022-12-20T16:51:33.440322154Z 2022-12-20T16:51:33.437Z INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 48 kiB). See UDP Receive Buffer Size · lucas-clemente/quic-go Wiki · GitHub for details. {“Process”: “storagenode”}
I am trying to do something about this as we speak - just in case
and
2022-12-20T16:51:33.440286434Z 2022-12-20T16:51:33.436Z INFO Public server started on [::]:28967 {“Process”: “storagenode”}
Which is wrong

aardvarkl · December 20, 2022, 4:55pm

2022-12-20T16:51:27.795335872Z downloading storagenode
2022-12-20T16:51:28.122147199Z --2022-12-20 16:51:28--  https://version.storj.io/processes/storagenode/minimum/url?os=linux&arch=amd64
2022-12-20T16:51:28.133651671Z Resolving version.storj.io (version.storj.io)... 35.232.172.28
2022-12-20T16:51:28.232139392Z Connecting to version.storj.io (version.storj.io)|35.232.172.28|:443... connected.
2022-12-20T16:51:28.623491867Z HTTP request sent, awaiting response... 200 OK
2022-12-20T16:51:28.623573110Z Length: 84 [text/plain]
2022-12-20T16:51:28.623588808Z Saving to: 'STDOUT'
2022-12-20T16:51:28.623601693Z 
2022-12-20T16:51:28.623642460Z      0K                                                       100%  193M=0s
2022-12-20T16:51:28.623657705Z 
2022-12-20T16:51:28.623672574Z 2022-12-20 16:51:28 (193 MB/s) - written to stdout [84/84]
2022-12-20T16:51:28.623685640Z 
2022-12-20T16:51:28.628949746Z --2022-12-20 16:51:28--  https://github.com/storj/storj/releases/download/v1.68.2/storagenode_linux_amd64.zip
2022-12-20T16:51:28.643193408Z Resolving github.com (github.com)... 140.82.121.3
2022-12-20T16:51:28.657369854Z Connecting to github.com (github.com)|140.82.121.3|:443... connected.
2022-12-20T16:51:28.764219975Z HTTP request sent, awaiting response... 302 Found
2022-12-20T16:51:28.764429805Z Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/128089774/2f540d24-8045-4433-ae6b-40e081053990?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221220%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221220T165018Z&X-Amz-Expires=300&X-Amz-Signature=d4640f7c04a72f3d4e6cde7c541212c94a66dab4f6696d1828fc98ba54d6bce2&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=128089774&response-content-disposition=attachment%3B%20filename%3Dstoragenode_linux_amd64.zip&response-content-type=application%2Foctet-stream [following]
2022-12-20T16:51:28.764550062Z --2022-12-20 16:51:28--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/128089774/2f540d24-8045-4433-ae6b-40e081053990?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221220%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221220T165018Z&X-Amz-Expires=300&X-Amz-Signature=d4640f7c04a72f3d4e6cde7c541212c94a66dab4f6696d1828fc98ba54d6bce2&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=128089774&response-content-disposition=attachment%3B%20filename%3Dstoragenode_linux_amd64.zip&response-content-type=application%2Foctet-stream
2022-12-20T16:51:28.773288662Z Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
2022-12-20T16:51:28.776559432Z Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
2022-12-20T16:51:29.152324198Z HTTP request sent, awaiting response... 200 OK
2022-12-20T16:51:29.152414221Z Length: 16654326 (16M) [application/octet-stream]
2022-12-20T16:51:29.152524629Z Saving to: '/tmp/storagenode.zip'
2022-12-20T16:51:29.152569343Z 
2022-12-20T16:51:29.156100688Z      0K .......... .......... .......... .......... ..........  0% 13.1M 1s
2022-12-20T16:51:29.159696444Z     50K .......... .......... .......... .......... ..........  0% 13.5M 1s
2022-12-20T16:51:29.164223827Z    100K .......... .......... .......... .......... ..........  0% 11.0M 1s
2022-12-20T16:51:29.164498430Z    150K .......... .......... .......... .......... ..........  1%  124M 1s
2022-12-20T16:51:29.172059846Z    200K .......... .......... .......... .......... ..........  1% 6.43M 1s
2022-12-20T16:51:29.173980574Z    250K .......... .......... .......... .......... ..........  1% 25.3M 1s
2022-12-20T16:51:29.177189826Z    300K .......... .......... .......... .......... ..........  2% 15.3M 1s
2022-12-20T16:51:29.179640489Z    350K .......... .......... .......... .......... ..........  2% 20.1M 1s
2022-12-20T16:51:29.180665063Z    400K .......... .......... .......... .......... ..........  2% 49.2M 1s
2022-12-20T16:51:29.181473615Z    450K .......... .......... .......... .......... ..........  3% 56.1M 1s
2022-12-20T16:51:29.182442082Z    500K .......... .......... .......... .......... ..........  3% 50.0M 1s
2022-12-20T16:51:29.183023943Z    550K .......... .......... .......... .......... ..........  3% 84.3M 1s
2022-12-20T16:51:29.183710254Z    600K .......... .......... .......... .......... ..........  3% 71.7M 1s
2022-12-20T16:51:29.184380469Z    650K .......... .......... .......... .......... ..........  4% 73.9M 1s
2022-12-20T16:51:29.184925739Z    700K .......... .......... .......... .......... ..........  4% 87.5M 1s
2022-12-20T16:51:29.185580792Z    750K .......... .......... .......... .......... ..........  4% 76.5M 1s
2022-12-20T16:51:29.186124073Z    800K .......... .......... .......... .......... ..........  5% 87.0M 1s
2022-12-20T16:51:29.186830575Z    850K .......... .......... .......... .......... ..........  5% 70.4M 1s
2022-12-20T16:51:29.187360310Z    900K .......... .......... .......... .......... ..........  5% 91.2M 1s
2022-12-20T16:51:29.187785505Z    950K .......... .......... .......... .......... ..........  6%  124M 1s
2022-12-20T16:51:29.188237999Z   1000K .......... .......... .......... .......... ..........  6%  100M 1s
2022-12-20T16:51:29.188712791Z   1050K .......... .......... .......... .......... ..........  6%  101M 1s
2022-12-20T16:51:29.189152650Z   1100K .......... .......... .......... .......... ..........  7%  125M 0s
2022-12-20T16:51:29.189667706Z   1150K .......... .......... .......... .......... ..........  7% 88.3M 0s
2022-12-20T16:51:29.190128672Z   1200K .......... .......... .......... .......... ..........  7%  107M 0s
2022-12-20T16:51:29.190546778Z   1250K .......... .......... .......... .......... ..........  7%  115M 0s
2022-12-20T16:51:29.190961332Z   1300K .......... .......... .......... .......... ..........  8%  120M 0s
2022-12-20T16:51:29.191402488Z   1350K .......... .......... .......... .......... ..........  8%  106M 0s
2022-12-20T16:51:29.191876229Z   1400K .......... .......... .......... .......... ..........  8%  105M 0s
2022-12-20T16:51:29.192281034Z   1450K .......... .......... .......... .......... ..........  9%  118M 0s
2022-12-20T16:51:29.192770753Z   1500K .......... .......... .......... .......... ..........  9%  101M 0s
2022-12-20T16:51:29.193297490Z   1550K .......... .......... .......... .......... ..........  9% 93.6M 0s
2022-12-20T16:51:29.193760369Z   1600K .......... .......... .......... .......... .......... 10%  105M 0s
2022-12-20T16:51:29.222443480Z   1650K .......... .......... .......... .......... .......... 10% 1.70M 1s
2022-12-20T16:51:29.222955729Z   1700K .......... .......... .......... .......... .......... 10% 91.4M 1s
2022-12-20T16:51:29.223575155Z   1750K .......... .......... .......... .......... .......... 11% 79.7M 1s
2022-12-20T16:51:29.224147427Z   1800K .......... .......... .......... .......... .......... 11% 83.2M 1s
2022-12-20T16:51:29.224730241Z   1850K .......... .......... .......... .......... .......... 11% 85.6M 1s
2022-12-20T16:51:29.225958847Z   1900K .......... .......... .......... .......... .......... 11% 43.2M 1s
2022-12-20T16:51:29.226448641Z   1950K .......... .......... .......... .......... .......... 12% 86.4M 1s
2022-12-20T16:51:29.297205291Z   2000K .......... .......... .......... .......... .......... 12%  708K 1s
2022-12-20T16:51:29.297902169Z   2050K .......... .......... .......... .......... .......... 12% 62.2M 1s
2022-12-20T16:51:29.298739777Z   2100K .......... .......... .......... .......... .......... 13% 57.3M 1s
2022-12-20T16:51:29.299406340Z   2150K .......... .......... .......... .......... .......... 13% 71.9M 1s
2022-12-20T16:51:29.300095406Z   2200K .......... .......... .......... .......... .......... 13% 71.7M 1s
2022-12-20T16:51:29.300707799Z   2250K .......... .......... .......... .......... .......... 14% 81.8M 1s
2022-12-20T16:51:29.301365653Z   2300K .......... .......... .......... .......... .......... 14% 74.2M 1s
2022-12-20T16:51:29.302016946Z   2350K .......... .......... .......... .......... .......... 14% 73.3M 1s
2022-12-20T16:51:29.302734535Z   2400K .......... .......... .......... .......... .......... 15% 70.1M 1s
2022-12-20T16:51:29.303298745Z   2450K .......... .......... .......... .......... .......... 15% 80.8M 1s
2022-12-20T16:51:29.303881538Z   2500K .......... .......... .......... .......... .......... 15% 87.8M 1s
2022-12-20T16:51:29.304657923Z   2550K .......... .......... .......... .......... .......... 15% 68.4M 1s
2022-12-20T16:51:29.305170352Z   2600K .......... .......... .......... .......... .......... 16% 83.6M 1s
2022-12-20T16:51:29.305813766Z   2650K .......... .......... .......... .......... .......... 16% 76.3M 1s
2022-12-20T16:51:29.306320567Z   2700K .......... .......... .......... .......... .......... 16% 88.9M 1s
2022-12-20T16:51:29.307009465Z   2750K .......... .......... .......... .......... .......... 17% 83.2M 1s
2022-12-20T16:51:29.307796726Z   2800K .......... .......... .......... .......... .......... 17% 58.2M 1s
2022-12-20T16:51:29.308428370Z   2850K .......... .......... .......... .......... .......... 17% 75.3M 1s
2022-12-20T16:51:29.309156049Z   2900K .......... .......... .......... .......... .......... 18% 77.9M 1s
2022-12-20T16:51:29.310137542Z   2950K .......... .......... .......... .......... .......... 18% 52.0M 1s
2022-12-20T16:51:29.310726333Z   3000K .......... .......... .......... .......... .......... 18% 68.6M 1s
2022-12-20T16:51:29.311193354Z   3050K .......... .......... .......... .......... .......... 19% 91.9M 1s
2022-12-20T16:51:29.311411902Z   3100K .......... .......... .......... .......... .......... 19%  226M 1s
2022-12-20T16:51:29.311717081Z   3150K .......... .......... .......... .......... .......... 19%  178M 1s
2022-12-20T16:51:29.311959346Z   3200K .......... .......... .......... .......... .......... 19%  181M 1s
2022-12-20T16:51:29.372066689Z   3250K .......... .......... .......... .......... .......... 20%  834K 1s
2022-12-20T16:51:29.373484526Z   3300K .......... .......... .......... .......... .......... 20% 34.7M 1s
2022-12-20T16:51:29.374896553Z   3350K .......... .......... .......... .......... .......... 20% 34.3M 1s
2022-12-20T16:51:29.376133604Z   3400K .......... .......... .......... .......... .......... 21% 36.0M 1s
2022-12-20T16:51:29.376402536Z   3450K .......... .......... .......... .......... .......... 21%  147M 1s
2022-12-20T16:51:29.376732827Z   3500K .......... .......... .......... .......... .......... 21%  155M 1s
2022-12-20T16:51:29.377098942Z   3550K .......... .......... .......... .......... .......... 22%  129M 1s
2022-12-20T16:51:29.377395589Z   3600K .......... .......... .......... .......... .......... 22%  163M 1s
2022-12-20T16:51:29.377725460Z   3650K .......... .......... .......... .......... .......... 22%  149M 1s
2022-12-20T16:51:29.378084621Z   3700K .......... .......... .......... .......... .......... 23%  142M 1s
2022-12-20T16:51:29.378424530Z   3750K .......... .......... .......... .......... .......... 23%  135M 1s
2022-12-20T16:51:29.378805815Z   3800K .......... .......... .......... .......... .......... 23%  130M 1s
2022-12-20T16:51:29.379150159Z   3850K .......... .......... .......... .......... .......... 23%  141M 1s
2022-12-20T16:51:29.379531926Z   3900K .......... .......... .......... .......... .......... 24%  131M 1s
2022-12-20T16:51:29.380051597Z   3950K .......... .......... .......... .......... .......... 24%  111M 1s
2022-12-20T16:51:29.380368061Z   4000K .......... .......... .......... .......... .......... 24%  121M 1s
2022-12-20T16:51:29.460821191Z   4050K .......... .......... .......... .......... .......... 25%  626K 1s
2022-12-20T16:51:29.461344600Z   4100K .......... .......... .......... .......... .......... 25% 46.9M 1s
2022-12-20T16:51:29.462686381Z   4150K .......... .......... .......... .......... .......... 25% 44.5M 1s
2022-12-20T16:51:29.464079146Z   4200K .......... .......... .......... .......... .......... 26% 30.2M 1s
2022-12-20T16:51:29.465354916Z   4250K .......... .......... .......... .......... .......... 26% 37.5M 1s
2022-12-20T16:51:29.466647099Z   4300K .......... .......... .......... .......... .......... 26% 43.0M 1s
2022-12-20T16:51:29.468015875Z   4350K .......... .......... .......... .......... .......... 27% 35.8M 1s
2022-12-20T16:51:29.469255837Z   4400K .......... .......... .......... .......... .......... 27% 35.0M 1s
2022-12-20T16:51:29.469623382Z   4450K .......... .......... .......... .......... .......... 27%  104M 1s
2022-12-20T16:51:29.469916585Z   4500K .......... .......... .......... .......... .......... 27%  168M 1s
2022-12-20T16:51:29.470326937Z   4550K .......... .......... .......... .......... .......... 28%  167M 1s
2022-12-20T16:51:29.470606570Z   4600K .......... .......... .......... .......... .......... 28%  125M 1s
2022-12-20T16:51:29.470891337Z   4650K .......... .......... .......... .......... .......... 28%  169M 1s
2022-12-20T16:51:29.471233882Z   4700K .......... .......... .......... .......... .......... 29%  143M 1s
2022-12-20T16:51:29.471555929Z   4750K .......... .......... .......... .......... .......... 29%  170M 1s
2022-12-20T16:51:29.471835401Z   4800K .......... .......... .......... .......... .......... 29%  152M 1s
2022-12-20T16:51:29.474658747Z   4850K .......... .......... .......... .......... .......... 30% 17.4M 1s
2022-12-20T16:51:29.474972611Z   4900K .......... .......... .......... .......... .......... 30%  154M 1s
2022-12-20T16:51:29.533356885Z   4950K .......... .......... .......... .......... .......... 30%  868K 1s
2022-12-20T16:51:29.534695623Z   5000K .......... .......... .......... .......... .......... 31% 23.8M 1s
2022-12-20T16:51:29.535848361Z   5050K .......... .......... .......... .......... .......... 31% 43.3M 1s
2022-12-20T16:51:29.541338672Z   5100K .......... .......... .......... .......... .......... 31% 8.82M 1s
2022-12-20T16:51:29.541879455Z   5150K .......... .......... .......... .......... .......... 31% 82.4M 1s
2022-12-20T16:51:29.542193239Z   5200K .......... .......... .......... .......... .......... 32%  164M 1s
2022-12-20T16:51:29.542468280Z   5250K .......... .......... .......... .......... .......... 32%  152M 1s
2022-12-20T16:51:29.542736811Z   5300K .......... .......... .......... .......... .......... 32%  191M 1s
2022-12-20T16:51:29.544466828Z   5350K .......... .......... .......... .......... .......... 33%  157M 1s
2022-12-20T16:51:29.544519671Z   5400K .......... .......... .......... .......... .......... 33%  163M 1s
2022-12-20T16:51:29.544532698Z   5450K .......... .......... .......... .......... .......... 33%  197M 1s
2022-12-20T16:51:29.544543568Z   5500K .......... .......... .......... .......... .......... 34%  160M 1s
2022-12-20T16:51:29.547528729Z   5550K .......... .......... .......... .......... .......... 34% 30.5M 1s
2022-12-20T16:51:29.559997123Z   5600K .......... .......... .......... .......... .......... 34% 3.81M 1s
2022-12-20T16:51:29.560129864Z   5650K .......... .......... .......... .......... .......... 35% 27.0M 1s
2022-12-20T16:51:29.560424946Z   5700K .......... .......... .......... .......... .......... 35%  194M 1s
2022-12-20T16:51:29.561851843Z   5750K .......... .......... .......... .......... .......... 35% 33.8M 1s
2022-12-20T16:51:29.562164640Z   5800K .......... .......... .......... .......... .......... 35%  141M 1s
2022-12-20T16:51:29.562522670Z   5850K .......... .......... .......... .......... .......... 36%  154M 1s
2022-12-20T16:51:29.607041735Z   5900K .......... .......... .......... .......... .......... 36% 1.10M 1s
2022-12-20T16:51:29.607684572Z   5950K .......... .......... .......... .......... .......... 36% 76.7M 1s
2022-12-20T16:51:29.608262054Z   6000K .......... .......... .......... .......... .......... 37% 80.2M 1s
2022-12-20T16:51:29.608872021Z   6050K .......... .......... .......... .......... .......... 37% 80.7M 1s
2022-12-20T16:51:29.609564918Z   6100K .......... .......... .......... .......... .......... 37% 73.4M 1s
2022-12-20T16:51:29.610251217Z   6150K .......... .......... .......... .......... .......... 38% 95.2M 1s
2022-12-20T16:51:29.610659863Z   6200K .......... .......... .......... .......... .......... 38% 79.8M 1s
2022-12-20T16:51:29.611338351Z   6250K .......... .......... .......... .......... .......... 38% 71.4M 1s
2022-12-20T16:51:29.611918888Z   6300K .......... .......... .......... .......... .......... 39% 85.2M 1s
2022-12-20T16:51:29.612587248Z   6350K .......... .......... .......... .......... .......... 39% 85.6M 1s
2022-12-20T16:51:29.613145592Z   6400K .......... .......... .......... .......... .......... 39% 74.6M 1s
2022-12-20T16:51:29.613813881Z   6450K .......... .......... .......... .......... .......... 39% 73.2M 1s
2022-12-20T16:51:29.614327738Z   6500K .......... .......... .......... .......... .......... 40% 96.0M 1s
2022-12-20T16:51:29.615002434Z   6550K .......... .......... .......... .......... .......... 40% 74.5M 1s
2022-12-20T16:51:29.615557609Z   6600K .......... .......... .......... .......... .......... 40% 84.5M 1s
2022-12-20T16:51:29.616168209Z   6650K .......... .......... .......... .......... .......... 41% 81.1M 1s
2022-12-20T16:51:29.616769428Z   6700K .......... .......... .......... .......... .......... 41% 80.4M 1s
2022-12-20T16:51:29.681390651Z   6750K .......... .......... .......... .......... .......... 41% 85.2M 1s
2022-12-20T16:51:29.681792740Z   6800K .......... .......... .......... .......... .......... 42%  776K 1s
2022-12-20T16:51:29.682422872Z   6850K .......... .......... .......... .......... .......... 42% 78.2M 1s
2022-12-20T16:51:29.683196825Z   6900K .......... .......... .......... .......... .......... 42% 62.9M 1s
2022-12-20T16:51:29.683766106Z   6950K .......... .......... .......... .......... .......... 43% 90.9M 1s
2022-12-20T16:51:29.684750750Z   7000K .......... .......... .......... .......... .......... 43% 47.8M 1s
2022-12-20T16:51:29.685277270Z   7050K .......... .......... .......... .......... .......... 43% 92.6M 1s
2022-12-20T16:51:29.685816879Z   7100K .......... .......... .......... .......... .......... 43% 90.0M 1s
2022-12-20T16:51:29.686839723Z   7150K .......... .......... .......... .......... .......... 44% 53.9M 1s
2022-12-20T16:51:29.687381154Z   7200K .......... .......... .......... .......... .......... 44% 74.6M 1s
2022-12-20T16:51:29.688126112Z   7250K .......... .......... .......... .......... .......... 44% 64.6M 1s
2022-12-20T16:51:29.688614292Z   7300K .......... .......... .......... .......... .......... 45%  103M 1s
2022-12-20T16:51:29.689254819Z   7350K .......... .......... .......... .......... .......... 45% 80.0M 1s
2022-12-20T16:51:29.689833686Z   7400K .......... .......... .......... .......... .......... 45% 80.7M 1s
2022-12-20T16:51:29.690359005Z   7450K .......... .......... .......... .......... .......... 46% 92.4M 1s
2022-12-20T16:51:29.690946853Z   7500K .......... .......... .......... .......... .......... 46% 90.3M 1s
2022-12-20T16:51:29.691534646Z   7550K .......... .......... .......... .......... .......... 46%  102M 1s
2022-12-20T16:51:29.692085954Z   7600K .......... .......... .......... .......... .......... 47% 68.5M 1s
2022-12-20T16:51:29.692618127Z   7650K .......... .......... .......... .......... .......... 47% 92.6M 1s
2022-12-20T16:51:29.756560951Z   7700K .......... .......... .......... .......... .......... 47%  782K 1s
2022-12-20T16:51:29.756975874Z   7750K .......... .......... .......... .......... .......... 47%  125M 1s
2022-12-20T16:51:29.757487301Z   7800K .......... .......... .......... .......... .......... 48% 89.5M 1s
2022-12-20T16:51:29.757917709Z   7850K .......... .......... .......... .......... .......... 48%  113M 1s
2022-12-20T16:51:29.761738851Z   7900K .......... .......... .......... .......... .......... 48% 12.8M 1s
2022-12-20T16:51:29.762145382Z   7950K .......... .......... .......... .......... .......... 49%  128M 1s
2022-12-20T16:51:29.762492063Z   8000K .......... .......... .......... .......... .......... 49%  129M 1s
2022-12-20T16:51:29.762835098Z   8050K .......... .......... .......... .......... .......... 49%  139M 1s
2022-12-20T16:51:29.763122689Z   8100K .......... .......... .......... .......... .......... 50%  168M 1s
2022-12-20T16:51:29.949266120Z   8150K .......... .......... .......... .......... .......... 50%  270K 1s
2022-12-20T16:51:29.954149985Z   8200K .......... .......... .......... .......... .......... 50% 19.5M 1s
2022-12-20T16:51:29.954190130Z   8250K .......... .......... .......... .......... .......... 51% 30.8M 1s
2022-12-20T16:51:29.954200311Z   8300K .......... .......... .......... .......... .......... 51% 54.1M 1s
2022-12-20T16:51:29.954209966Z   8350K .......... .......... .......... .......... .......... 51%  182M 1s
2022-12-20T16:51:29.954243536Z   8400K .......... .......... .......... .......... .......... 51%  153M 1s
2022-12-20T16:51:29.954284171Z   8450K .......... .......... .......... .......... .......... 52%  154M 1s
2022-12-20T16:51:29.954739625Z   8500K .......... .......... .......... .......... .......... 52%  173M 1s
2022-12-20T16:51:29.954874908Z   8550K .......... .......... .......... .......... .......... 52%  155M 1s
2022-12-20T16:51:29.955163079Z   8600K .......... .......... .......... .......... .......... 53%  149M 1s
2022-12-20T16:51:29.957218460Z   8650K .......... .......... .......... .......... .......... 53%  163M 1s
2022-12-20T16:51:29.957254093Z   8700K .......... .......... .......... .......... .......... 53%  144M 1s
2022-12-20T16:51:29.957282404Z   8750K .......... .......... .......... .......... .......... 54% 79.1M 1s
2022-12-20T16:51:29.957292830Z   8800K .......... .......... .......... .......... .......... 54%  127M 1s
2022-12-20T16:51:29.957312902Z   8850K .......... .......... .......... .......... .......... 54%  160M 1s
2022-12-20T16:51:29.957435290Z   8900K .......... .......... .......... .......... .......... 55%  180M 1s
2022-12-20T16:51:29.957747230Z   8950K .......... .......... .......... .......... .......... 55%  150M 1s
2022-12-20T16:51:29.959681318Z   9000K .......... .......... .......... .......... .......... 55%  151M 1s
2022-12-20T16:51:30.019087738Z   9050K .......... .......... .......... .......... .......... 55%  819K 1s
2022-12-20T16:51:30.019554052Z   9100K .......... .......... .......... .......... .......... 56% 98.5M 1s
2022-12-20T16:51:30.020204049Z   9150K .......... .......... .......... .......... .......... 56% 79.9M 1s
2022-12-20T16:51:30.020746436Z   9200K .......... .......... .......... .......... .......... 56% 82.2M 1s
2022-12-20T16:51:30.021408325Z   9250K .......... .......... .......... .......... .......... 57% 76.0M 1s
2022-12-20T16:51:30.021934154Z   9300K .......... .......... .......... .......... .......... 57% 90.4M 1s
2022-12-20T16:51:30.022573455Z   9350K .......... .......... .......... .......... .......... 57% 79.2M 1s
2022-12-20T16:51:30.023155222Z   9400K .......... .......... .......... .......... .......... 58% 79.5M 1s
2022-12-20T16:51:30.023726100Z   9450K .......... .......... .......... .......... .......... 58% 85.9M 1s
2022-12-20T16:51:30.024306204Z   9500K .......... .......... .......... .......... .......... 58% 84.8M 1s
2022-12-20T16:51:30.025025621Z   9550K .......... .......... .......... .......... .......... 59% 86.4M 1s
2022-12-20T16:51:30.033978153Z   9600K .......... .......... .......... .......... .......... 59% 5.37M 1s
2022-12-20T16:51:30.044810689Z   9650K .......... .......... .......... .......... .......... 59%  175M 1s
2022-12-20T16:51:30.044873611Z   9700K .......... .......... .......... .......... .......... 59%  157M 1s
2022-12-20T16:51:30.044886857Z   9750K .......... .......... .......... .......... .......... 60%  204M 1s
2022-12-20T16:51:30.044928729Z   9800K .......... .......... .......... .......... .......... 60%  166M 1s
2022-12-20T16:51:30.044940325Z   9850K .......... .......... .......... .......... .......... 60%  174M 1s
2022-12-20T16:51:30.044950409Z   9900K .......... .......... .......... .......... .......... 61%  183M 1s
2022-12-20T16:51:30.093500373Z   9950K .......... .......... .......... .......... .......... 61%  162M 1s
2022-12-20T16:51:30.093832413Z  10000K .......... .......... .......... .......... .......... 61%  864K 1s
2022-12-20T16:51:30.094461159Z  10050K .......... .......... .......... .......... .......... 62% 77.5M 1s
2022-12-20T16:51:30.094984378Z  10100K .......... .......... .......... .......... .......... 62% 90.8M 1s
2022-12-20T16:51:30.095622436Z  10150K .......... .......... .......... .......... .......... 62% 79.3M 1s
2022-12-20T16:51:30.096207581Z  10200K .......... .......... .......... .......... .......... 63% 81.0M 1s
2022-12-20T16:51:30.096801838Z  10250K .......... .......... .......... .......... .......... 63% 82.5M 1s
2022-12-20T16:51:30.097410657Z  10300K .......... .......... .......... .......... .......... 63% 79.5M 1s
2022-12-20T16:51:30.098178604Z  10350K .......... .......... .......... .......... .......... 63% 83.7M 1s
2022-12-20T16:51:30.098635683Z  10400K .......... .......... .......... .......... .......... 64% 77.8M 1s
2022-12-20T16:51:30.099338396Z  10450K .......... .......... .......... .......... .......... 64% 72.9M 1s
2022-12-20T16:51:30.099863488Z  10500K .......... .......... .......... .......... .......... 64% 87.6M 1s
2022-12-20T16:51:30.100433223Z  10550K .......... .......... .......... .......... .......... 65% 87.9M 1s
2022-12-20T16:51:30.100960899Z  10600K .......... .......... .......... .......... .......... 65% 88.8M 1s
2022-12-20T16:51:30.101640549Z  10650K .......... .......... .......... .......... .......... 65% 71.8M 0s
2022-12-20T16:51:30.102114077Z  10700K .......... .......... .......... .......... .......... 66%  104M 0s
2022-12-20T16:51:30.102838602Z  10750K .......... .......... .......... .......... .......... 66% 85.7M 0s
2022-12-20T16:51:30.103417677Z  10800K .......... .......... .......... .......... .......... 66% 66.6M 0s
2022-12-20T16:51:30.171116464Z  10850K .......... .......... .......... .......... .......... 67%  741K 1s
2022-12-20T16:51:30.171467038Z  10900K .......... .......... .......... .......... .......... 67% 96.4M 0s
2022-12-20T16:51:30.171922517Z  10950K .......... .......... .......... .......... .......... 67%  160M 0s
2022-12-20T16:51:30.172175440Z  11000K .......... .......... .......... .......... .......... 67%  114M 0s
2022-12-20T16:51:30.172387770Z  11050K .......... .......... .......... .......... .......... 68%  228M 0s
2022-12-20T16:51:30.172692978Z  11100K .......... .......... .......... .......... .......... 68%  163M 0s
2022-12-20T16:51:30.172942527Z  11150K .......... .......... .......... .......... .......... 68%  220M 0s
2022-12-20T16:51:30.173160715Z  11200K .......... .......... .......... .......... .......... 69%  196M 0s
2022-12-20T16:51:30.173411358Z  11250K .......... .......... .......... .......... .......... 69%  192M 0s
2022-12-20T16:51:30.173644359Z  11300K .......... .......... .......... .......... .......... 69%  217M 0s
2022-12-20T16:51:30.173951319Z  11350K .......... .......... .......... .......... .......... 70%  194M 0s
2022-12-20T16:51:30.174167085Z  11400K .......... .......... .......... .......... .......... 70%  181M 0s
2022-12-20T16:51:30.174380480Z  11450K .......... .......... .......... .......... .......... 70%  230M 0s
2022-12-20T16:51:30.176023543Z  11500K .......... .......... .......... .......... .......... 71% 30.2M 0s
2022-12-20T16:51:30.176305376Z  11550K .......... .......... .......... .......... .......... 71%  202M 0s
2022-12-20T16:51:30.176586304Z  11600K .......... .......... .......... .......... .......... 71%  164M 0s
2022-12-20T16:51:30.176784627Z  11650K .......... .......... .......... .......... .......... 71%  189M 0s
2022-12-20T16:51:30.177069191Z  11700K .......... .......... .......... .......... .......... 72%  182M 0s
2022-12-20T16:51:30.177636197Z  11750K .......... .......... .......... .......... .......... 72% 88.0M 0s
2022-12-20T16:51:30.245242054Z  11800K .......... .......... .......... .......... .......... 72%  740K 0s
2022-12-20T16:51:30.246317215Z  11850K .......... .......... .......... .......... .......... 73% 51.2M 0s
2022-12-20T16:51:30.247089677Z  11900K .......... .......... .......... .......... .......... 73% 55.3M 0s
2022-12-20T16:51:30.247446437Z  11950K .......... .......... .......... .......... .......... 73%  123M 0s
2022-12-20T16:51:30.247792448Z  12000K .......... .......... .......... .......... .......... 74%  129M 0s
2022-12-20T16:51:30.257229432Z  12050K .......... .......... .......... .......... .......... 74% 8.30M 0s
2022-12-20T16:51:30.257340482Z  12100K .......... .......... .......... .......... .......... 74%  129M 0s
2022-12-20T16:51:30.257354564Z  12150K .......... .......... .......... .......... .......... 75%  128M 0s
2022-12-20T16:51:30.257364740Z  12200K .......... .......... .......... .......... .......... 75%  154M 0s
2022-12-20T16:51:30.327703859Z  12250K .......... .......... .......... .......... .......... 75%  686K 0s
2022-12-20T16:51:30.328118116Z  12300K .......... .......... .......... .......... .......... 75%  101M 0s
2022-12-20T16:51:30.331859476Z  12350K .......... .......... .......... .......... .......... 76% 13.5M 0s
2022-12-20T16:51:30.332323958Z  12400K .......... .......... .......... .......... .......... 76% 85.2M 0s
2022-12-20T16:51:30.332703844Z  12450K .......... .......... .......... .......... .......... 76%  127M 0s
2022-12-20T16:51:30.333126451Z  12500K .......... .......... .......... .......... .......... 77%  128M 0s
2022-12-20T16:51:30.333533977Z  12550K .......... .......... .......... .......... .......... 77%  116M 0s
2022-12-20T16:51:30.333901367Z  12600K .......... .......... .......... .......... .......... 77%  119M 0s
2022-12-20T16:51:30.334322815Z  12650K .......... .......... .......... .......... .......... 78%  116M 0s
2022-12-20T16:51:30.334688777Z  12700K .......... .......... .......... .......... .......... 78%  133M 0s
2022-12-20T16:51:30.335190910Z  12750K .......... .......... .......... .......... .......... 78%  133M 0s
2022-12-20T16:51:30.335567302Z  12800K .......... .......... .......... .......... .......... 79% 98.5M 0s
2022-12-20T16:51:30.335977273Z  12850K .......... .......... .......... .......... .......... 79%  126M 0s
2022-12-20T16:51:30.339791414Z  12900K .......... .......... .......... .......... .......... 79% 12.7M 0s
2022-12-20T16:51:30.340073049Z  12950K .......... .......... .......... .......... .......... 79%  180M 0s
2022-12-20T16:51:30.340335078Z  13000K .......... .......... .......... .......... .......... 80%  164M 0s
2022-12-20T16:51:30.340628316Z  13050K .......... .......... .......... .......... .......... 80%  166M 0s
2022-12-20T16:51:30.340887298Z  13100K .......... .......... .......... .......... .......... 80%  188M 0s
2022-12-20T16:51:30.396551375Z  13150K .......... .......... .......... .......... .......... 81%  900K 0s
2022-12-20T16:51:30.397022800Z  13200K .......... .......... .......... .......... .......... 81% 84.8M 0s
2022-12-20T16:51:30.397620624Z  13250K .......... .......... .......... .......... .......... 81% 80.9M 0s
2022-12-20T16:51:30.398166295Z  13300K .......... .......... .......... .......... .......... 82% 90.3M 0s
2022-12-20T16:51:30.398983065Z  13350K .......... .......... .......... .......... .......... 82% 74.9M 0s
2022-12-20T16:51:30.399461431Z  13400K .......... .......... .......... .......... .......... 82% 75.3M 0s
2022-12-20T16:51:30.400054947Z  13450K .......... .......... .......... .......... .......... 83% 81.6M 0s
2022-12-20T16:51:30.400612093Z  13500K .......... .......... .......... .......... .......... 83% 88.2M 0s
2022-12-20T16:51:30.401281024Z  13550K .......... .......... .......... .......... .......... 83% 75.5M 0s
2022-12-20T16:51:30.401951488Z  13600K .......... .......... .......... .......... .......... 83% 71.2M 0s
2022-12-20T16:51:30.402540685Z  13650K .......... .......... .......... .......... .......... 84% 86.8M 0s
2022-12-20T16:51:30.403078722Z  13700K .......... .......... .......... .......... .......... 84% 86.5M 0s
2022-12-20T16:51:30.403580187Z  13750K .......... .......... .......... .......... .......... 84%  101M 0s
2022-12-20T16:51:30.404331860Z  13800K .......... .......... .......... .......... .......... 85% 80.4M 0s
2022-12-20T16:51:30.404786857Z  13850K .......... .......... .......... .......... .......... 85% 77.5M 0s
2022-12-20T16:51:30.405498191Z  13900K .......... .......... .......... .......... .......... 85% 73.2M 0s
2022-12-20T16:51:30.406070087Z  13950K .......... .......... .......... .......... .......... 86%  107M 0s
2022-12-20T16:51:30.406647075Z  14000K .......... .......... .......... .......... .......... 86% 67.2M 0s
2022-12-20T16:51:30.470864068Z  14050K .......... .......... .......... .......... .......... 86%  781K 0s
2022-12-20T16:51:30.471216549Z  14100K .......... .......... .......... .......... .......... 87% 94.1M 0s
2022-12-20T16:51:30.471828728Z  14150K .......... .......... .......... .......... .......... 87% 82.7M 0s
2022-12-20T16:51:30.472377107Z  14200K .......... .......... .......... .......... .......... 87% 81.4M 0s
2022-12-20T16:51:30.472968852Z  14250K .......... .......... .......... .......... .......... 87% 82.2M 0s
2022-12-20T16:51:30.473589813Z  14300K .......... .......... .......... .......... .......... 88% 80.7M 0s
2022-12-20T16:51:30.474319597Z  14350K .......... .......... .......... .......... .......... 88% 86.4M 0s
2022-12-20T16:51:30.475042387Z  14400K .......... .......... .......... .......... .......... 88% 55.8M 0s
2022-12-20T16:51:30.475448887Z  14450K .......... .......... .......... .......... .......... 89%  116M 0s
2022-12-20T16:51:30.475946481Z  14500K .......... .......... .......... .......... .......... 89% 95.2M 0s
2022-12-20T16:51:30.476683292Z  14550K .......... .......... .......... .......... .......... 89% 80.8M 0s
2022-12-20T16:51:30.477155450Z  14600K .......... .......... .......... .......... .......... 90% 79.6M 0s
2022-12-20T16:51:30.477687840Z  14650K .......... .......... .......... .......... .......... 90% 91.8M 0s
2022-12-20T16:51:30.478468560Z  14700K .......... .......... .......... .......... .......... 90% 63.8M 0s
2022-12-20T16:51:30.479057576Z  14750K .......... .......... .......... .......... .......... 90% 82.5M 0s
2022-12-20T16:51:30.553916040Z  14800K .......... .......... .......... .......... .......... 91%  673K 0s
2022-12-20T16:51:30.555315553Z  14850K .......... .......... .......... .......... .......... 91% 28.3M 0s
2022-12-20T16:51:30.557183182Z  14900K .......... .......... .......... .......... .......... 91% 26.6M 0s
2022-12-20T16:51:30.557654566Z  14950K .......... .......... .......... .......... .......... 92% 81.7M 0s
2022-12-20T16:51:30.558002025Z  15000K .......... .......... .......... .......... .......... 92%  118M 0s
2022-12-20T16:51:30.558356994Z  15050K .......... .......... .......... .......... .......... 92%  158M 0s
2022-12-20T16:51:30.558670948Z  15100K .......... .......... .......... .......... .......... 93%  133M 0s
2022-12-20T16:51:30.559021983Z  15150K .......... .......... .......... .......... .......... 93%  156M 0s
2022-12-20T16:51:30.559366983Z  15200K .......... .......... .......... .......... .......... 93%  123M 0s
2022-12-20T16:51:30.559675977Z  15250K .......... .......... .......... .......... .......... 94%  157M 0s
2022-12-20T16:51:30.559996930Z  15300K .......... .......... .......... .......... .......... 94%  161M 0s
2022-12-20T16:51:30.560425677Z  15350K .......... .......... .......... .......... .......... 94%  135M 0s
2022-12-20T16:51:30.560709807Z  15400K .......... .......... .......... .......... .......... 94%  137M 0s
2022-12-20T16:51:30.627277127Z  15450K .......... .......... .......... .......... .......... 95%  774K 0s
2022-12-20T16:51:30.628764193Z  15500K .......... .......... .......... .......... .......... 95% 18.3M 0s
2022-12-20T16:51:30.629663098Z  15550K .......... .......... .......... .......... .......... 95% 33.8M 0s
2022-12-20T16:51:30.630418866Z  15600K .......... .......... .......... .......... .......... 96% 53.7M 0s
2022-12-20T16:51:30.631087882Z  15650K .......... .......... .......... .......... .......... 96% 70.2M 0s
2022-12-20T16:51:30.631855669Z  15700K .......... .......... .......... .......... .......... 96% 69.8M 0s
2022-12-20T16:51:30.632343150Z  15750K .......... .......... .......... .......... .......... 97% 99.3M 0s
2022-12-20T16:51:30.632651758Z  15800K .......... .......... .......... .......... .......... 97%  119M 0s
2022-12-20T16:51:30.633005079Z  15850K .......... .......... .......... .......... .......... 97%  131M 0s
2022-12-20T16:51:30.633398543Z  15900K .......... .......... .......... .......... .......... 98%  134M 0s
2022-12-20T16:51:30.633791665Z  15950K .......... .......... .......... .......... .......... 98%  151M 0s
2022-12-20T16:51:30.634117233Z  16000K .......... .......... .......... .......... .......... 98%  116M 0s
2022-12-20T16:51:30.694550251Z  16050K .......... .......... .......... .......... .......... 98%  829K 0s
2022-12-20T16:51:30.695537126Z  16100K .......... .......... .......... .......... .......... 99% 45.9M 0s
2022-12-20T16:51:30.696461980Z  16150K .......... .......... .......... .......... .......... 99% 56.5M 0s
2022-12-20T16:51:30.696719765Z  16200K .......... .......... .......... .......... .......... 99%  126M 0s
2022-12-20T16:51:30.696852314Z  16250K .......... ...                                        100%  266M=1.5s
2022-12-20T16:51:30.696883238Z 
2022-12-20T16:51:30.696958946Z 2022-12-20 16:51:30 (10.3 MB/s) - '/tmp/storagenode.zip' saved [16654326/16654326]
2022-12-20T16:51:30.696967468Z 
2022-12-20T16:51:31.138401400Z 2022-12-20 16:51:31,138 INFO RPC interface 'supervisor' initialized
2022-12-20T16:51:31.138698531Z 2022-12-20 16:51:31,138 INFO supervisord started with pid 1
2022-12-20T16:51:32.141733543Z 2022-12-20 16:51:32,141 INFO spawned: 'processes-exit-eventlistener' with pid 57
2022-12-20T16:51:32.143983949Z 2022-12-20 16:51:32,143 INFO spawned: 'storagenode' with pid 58
2022-12-20T16:51:32.152235453Z 2022-12-20 16:51:32,151 INFO spawned: 'storagenode-updater' with pid 59
2022-12-20T16:51:32.180689639Z 2022-12-20T16:51:32.180Z	INFO	Configuration loaded	{"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
2022-12-20T16:51:32.180730734Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "server.address"}
2022-12-20T16:51:32.180741660Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
2022-12-20T16:51:32.180774066Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "contact.external-address"}
2022-12-20T16:51:32.180783156Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "operator.email"}
2022-12-20T16:51:32.180805376Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
2022-12-20T16:51:32.180813224Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "console.address"}
2022-12-20T16:51:32.180827074Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "server.private-address"}
2022-12-20T16:51:32.180834735Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "healthcheck.enabled"}
2022-12-20T16:51:32.180840914Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "healthcheck.details"}
2022-12-20T16:51:32.180854199Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "operator.wallet-features"}
2022-12-20T16:51:32.180864057Z 2022-12-20T16:51:32.180Z	INFO	Invalid configuration file key	{"Process": "storagenode-updater", "Key": "operator.wallet"}
2022-12-20T16:51:32.180878004Z 2022-12-20T16:51:32.180Z	INFO	Anonymized tracing enabled	{"Process": "storagenode-updater"}
2022-12-20T16:51:32.181255851Z 2022-12-20T16:51:32.181Z	INFO	Running on version	{"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.68.2"}
2022-12-20T16:51:32.181821978Z 2022-12-20T16:51:32.181Z	INFO	Downloading versions.	{"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
2022-12-20T16:51:32.203544907Z 2022-12-20T16:51:32.203Z	INFO	Configuration loaded	{"Process": "storagenode", "Location": "/app/config/config.yaml"}
2022-12-20T16:51:32.203774705Z 2022-12-20T16:51:32.203Z	INFO	Anonymized tracing enabled	{"Process": "storagenode"}
2022-12-20T16:51:32.205378471Z 2022-12-20T16:51:32.205Z	INFO	Operator email	{"Process": "storagenode", "Address": "storj@sendarian.co.uk"}
2022-12-20T16:51:32.205517464Z 2022-12-20T16:51:32.205Z	INFO	Operator wallet	{"Process": "storagenode", "Address": "0xe325a2779863C9dd3FC0874A51885Bb4DAc150F0"}
2022-12-20T16:51:32.506847929Z 2022-12-20T16:51:32.506Z	INFO	Current binary version	{"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.68.2"}
2022-12-20T16:51:32.506905436Z 2022-12-20T16:51:32.506Z	INFO	New version is being rolled out but hasn't made it to this node yet	{"Process": "storagenode-updater", "Service": "storagenode"}
2022-12-20T16:51:32.513286006Z 2022-12-20T16:51:32.513Z	INFO	Current binary version	{"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.68.2"}
2022-12-20T16:51:32.513308519Z 2022-12-20T16:51:32.513Z	INFO	New version is being rolled out but hasn't made it to this node yet	{"Process": "storagenode-updater", "Service": "storagenode-updater"}
2022-12-20T16:51:32.661768608Z 2022-12-20T16:51:32.658Z	INFO	Telemetry enabled	{"Process": "storagenode", "instance ID": "12AvLRHpL6hNFX7PJEeXCFce37rorrafyHYTuhQKr6sUPfdTcsf"}
2022-12-20T16:51:32.661833562Z 2022-12-20T16:51:32.658Z	INFO	Event collection enabled	{"Process": "storagenode", "instance ID": "12AvLRHpL6hNFX7PJEeXCFce37rorrafyHYTuhQKr6sUPfdTcsf"}
2022-12-20T16:51:32.681409897Z 2022-12-20T16:51:32.681Z	INFO	db.migration	Database Version	{"Process": "storagenode", "version": 54}
2022-12-20T16:51:32.742988468Z 2022-12-20T16:51:32.741Z	INFO	preflight:localtime	start checking local system clock with trusted satellites' system clock.	{"Process": "storagenode"}
2022-12-20T16:51:33.436332420Z 2022-12-20T16:51:33.435Z	INFO	preflight:localtime	local system clock is in sync with trusted satellites' system clock.	{"Process": "storagenode"}
2022-12-20T16:51:33.436425284Z 2022-12-20T16:51:33.435Z	INFO	bandwidth	Performing bandwidth usage rollups	{"Process": "storagenode"}
2022-12-20T16:51:33.436488079Z 2022-12-20 16:51:33,436 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-12-20T16:51:33.436560149Z 2022-12-20 16:51:33,436 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-12-20T16:51:33.436617163Z 2022-12-20 16:51:33,436 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-12-20T16:51:33.440218593Z 2022-12-20T16:51:33.436Z	INFO	Node 12AvLRHpL6hNFX7PJEeXCFce37rorrafyHYTuhQKr6sUPfdTcsf started	{"Process": "storagenode"}
2022-12-20T16:51:33.440286434Z 2022-12-20T16:51:33.436Z	INFO	Public server started on [::]:28967	{"Process": "storagenode"}
2022-12-20T16:51:33.440298676Z 2022-12-20T16:51:33.436Z	INFO	Private server started on 127.0.0.1:7778	{"Process": "storagenode"}
2022-12-20T16:51:33.440322154Z 2022-12-20T16:51:33.437Z	INFO	failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 48 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.	{"Process": "storagenode"}
2022-12-20T16:51:33.440333389Z 2022-12-20T16:51:33.437Z	INFO	trust	Scheduling next refresh	{"Process": "storagenode", "after": "5h59m28.613966524s"}
2022-12-20T16:51:33.447847289Z 2022-12-20T16:51:33.444Z	INFO	collector	deleted expired piece	{"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Piece ID": "YXUDNNSVP4ZFJVD5US6ILF6KK3A5GY2EADU5GFOIAM4MVNXD5SHQ"}
2022-12-20T16:51:33.447896584Z 2022-12-20T16:51:33.446Z	INFO	collector	collect	{"Process": "storagenode", "count": 1}
2022-12-20T16:51:34.123504387Z 2022-12-20T16:51:34.123Z	INFO	piecestore	download started	{"Process": "storagenode", "Piece ID": "GDLS7FB67K32HTQK67VAM67JVFALXP7VLEJD6DCRUDWAENVG6UEQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:34.407244686Z 2022-12-20T16:51:34.406Z	INFO	piecestore	download started	{"Process": "storagenode", "Piece ID": "XVMIS4OTY43OO5QCOCS2JEPNZBLY7ADUJ6K2TSUBOUKS3IC7SMOA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:34.442741398Z 2022-12-20T16:51:34.442Z	INFO	piecestore	downloaded	{"Process": "storagenode", "Piece ID": "GDLS7FB67K32HTQK67VAM67JVFALXP7VLEJD6DCRUDWAENVG6UEQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:34.762731514Z 2022-12-20T16:51:34.762Z	INFO	piecestore	upload started	{"Process": "storagenode", "Piece ID": "6I2T7U5ARVJZWRSJFFDJP5SB6VJYCHB477E5U3E2DD7PCMK4I7CA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Available Space": 1649137841727}
2022-12-20T16:51:34.877116493Z 2022-12-20T16:51:34.876Z	INFO	piecestore	downloaded	{"Process": "storagenode", "Piece ID": "XVMIS4OTY43OO5QCOCS2JEPNZBLY7ADUJ6K2TSUBOUKS3IC7SMOA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:36.622560557Z 2022-12-20T16:51:36.622Z	INFO	piecestore	download started	{"Process": "storagenode", "Piece ID": "4H76EHIWV4ST7NPZ5AOKIHJECZFF4IZSHWH2CVYAZYDSTO3AS2IQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:36.780647764Z 2022-12-20T16:51:36.780Z	INFO	piecestore	downloaded	{"Process": "storagenode", "Piece ID": "4H76EHIWV4ST7NPZ5AOKIHJECZFF4IZSHWH2CVYAZYDSTO3AS2IQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET"}
2022-12-20T16:51:37.576448428Z 2022-12-20T16:51:37.576Z	INFO	piecestore	upload started	{"Process": "storagenode", "Piece ID": "PVT5SDFYXYLKDKCP6H5LEEW5VAEQDCE6D7UG46MFGO5WBQP3WOIQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "PUT_REPAIR", "Available Space": 1649137841727}
2022-12-20T16:51:37.578632795Z 2022-12-20T16:51:37.578Z	INFO	piecestore	uploaded	{"Process": "storagenode", "Piece ID": "PVT5SDFYXYLKDKCP6H5LEEW5VAEQDCE6D7UG46MFGO5WBQP3WOIQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "PUT_REPAIR", "Size": 1024}
2022-12-20T16:51:37.712380460Z 2022-12-20T16:51:37.712Z	INFO	piecestore	uploaded	{"Process": "storagenode", "Piece ID": "6I2T7U5ARVJZWRSJFFDJP5SB6VJYCHB477E5U3E2DD7PCMK4I7CA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "Size": 290304}

Stob · December 20, 2022, 5:04pm

Check your container run parameters. If you deleted the container then it could have lost the specific port configurations from before.

But from the log it looks as though more is going on than you think as the node is running on port 28967 and there are both PUT and GET commands succeeding which suggests port forwarding through the router is working as it should.

aardvarkl · December 20, 2022, 5:34pm

When I configured the new container, I matched it to the firewall forwarding ports which is definately 20988. 28967 is pointed to a different internal address on the Synology. I re-followed the You Tube video from IX about how to set this up.

I can also confirm that traffic is coming in on 20988.

Rebooting the NAS with net.core.rmem_max=2500000 set as per UDP Receive Buffer Size · lucas-clemente/quic-go Wiki · GitHub and the error message in the logs and the QUIC comes up OK

It just takes a while to reboot as I have to migrate VM’s away from the shared storage before the reboot.

So was it the reboot OR the sysctl entry or a combination of both? Dunno