Anything that can be done , to make Storj a little more robust?

Ok, so now i think were getting a real error, after changing to the ip address

Error: Error creating revocation database: revocation database: boltdb: open config/revocations.db: permission denied
        storj.io/storj/storage/boltdb.New:44
        storj.io/storj/private/revocation.openDBBolt:52
        storj.io/storj/private/revocation.OpenDB:35
        storj.io/storj/private/revocation.OpenDBFromCfg:23
        main.cmdRun:202
        storj.io/private/process.cleanup.func1.4:377
        storj.io/private/process.cleanup.func1:395
        github.com/spf13/cobra.(*Command).execute:852
        github.com/spf13/cobra.(*Command).ExecuteC:960
        github.com/spf13/cobra.(*Command).Execute:897
        storj.io/private/process.ExecWithCustomConfigAndLogger:92
        main.main:478
        runtime.main:250
2023-03-14 08:04:12,837 INFO exited: storagenode (exit status 1; not expected)
2023-03-14 08:04:13,839 INFO gave up: storagenode entered FATAL state, too many start retries too quickly
2023-03-14 08:04:14,842 WARN received SIGQUIT indicating exit request
2023-03-14 08:04:14,845 INFO waiting for processes-exit-eventlistener, storagenode-updater to die
2023-03-14T08:04:14.842Z        INFO    Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
2023-03-14T08:04:14.843Z        ERROR   Error retrieving version info.  {"Process": "storagenode-updater", "error": "version checker client: Get \"https://version.storj.io\": context canceled", "errorVerbose": "version checker client: Get \"https://version.storj.io\": context canceled\n\tstorj.io/storj/private/version/checker.(*Client).All:68\n\tmain.loopFunc:21\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tmain.cmdRun:136\n\tstorj.io/private/process.cleanup.func1.4:377\n\tstorj.io/private/process.cleanup.func1:395\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfigAndLogger:92\n\tmain.main:20\n\truntime.main:250"}
2023-03-14 08:04:14,846 INFO stopped: storagenode-updater (exit status 0)
2023-03-14 08:04:14,847 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)

Okay, so i knoiw you guys want to keep focusing on my network being the problem.
So i just did an apt-get update && apt-get upgrade, on the vm that storjnode is running on, and it updated, no problem.

acking python3-apt (2.0.1ubuntu0.20.04.1) over (2.0.1) ...
Preparing to unpack .../ubuntu-release-upgrader-core_1%3a20.04.41_all.deb ...
Unpacking ubuntu-release-upgrader-core (1:20.04.41) over (1:20.04.40) ...
Preparing to unpack .../python3-distupgrade_1%3a20.04.41_all.deb ...
Unpacking python3-distupgrade (1:20.04.41) over (1:20.04.40) ...
Setting up motd-news-config (11ubuntu5.7) ...
Setting up python-apt-common (2.0.1ubuntu0.20.04.1) ...
Setting up python3-apt (2.0.1ubuntu0.20.04.1) ...
Setting up python3-distupgrade (1:20.04.41) ...
Setting up ubuntu-release-upgrader-core (1:20.04.41) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for plymouth-theme-ubuntu-text (0.9.4git20200323-0ubuntu6.2) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for install-info (6.7.0.dfsg.2-5) ...
Processing triggers for initramfs-tools (0.136ubuntu6.7) ...
update-initramfs: Generating /boot/initrd.img-5.4.0-144-generic
[storjnode]:</data/>

So iā€™m pretty sure thereā€™s something wrong inside stroragenode.

So is this a real error now ?

2023-03-14T20:38:41.458Z        ERROR   preflight:localtime
unable to get satellite system time
{"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", 
"error": "rpc: tcp connector failed: rpc: dial tcp: lookup eu1.storj.io: operation was canceled", 
"errorVerbose": "rpc: tcp connector failed: rpc: dial tcp: lookup eu1.storj.io: 
operation was canceled
\tstorj.io/common/rpc.HybridConnector.DialContext.func1:189"}

You need to change permissions in your data location to your user and group,

sudo chown $(id -u):$(id -g) -R /data

still suggests that it have problems with DNS resolution and may be there some firewall or iptables issues related to the docker installation - the outgoing requests are blocked for some reason.

Yes Alexey i have changed permiossions, somehow i had a mix of my ownership and root, smh

As i said before i have no firewall currently running running and also did not setup any iptables or anything, unless it got installed by default, which would have been months and months go, when all was still working.

So whatā€™s the best easiest way to reset this system ? start from fresh ?

It just seems like were chasing each other around the tree.

You may try to reinstall docker, may be it could fix issues with iptables.
But more likely some OS update changed some defaults.
For example CentOS is often unexpected blocking something, see

I have no idea what any of this means, iā€™ve never done anything with it.

torjnode]:</home/mike/> iptables -L
Fatal: can't open lock file /run/xtables.lock: Permission denied
[storjnode]:</home/mike/> sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain DOCKER (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             ftwlrtrj01.pn.at.cox.cci  tcp dpt:28967
ACCEPT     udp  --  anywhere             ftwlrtrj01.pn.at.cox.cci  udp dpt:28967
ACCEPT     tcp  --  anywhere             ftwlrtrj01.pn.at.cox.cci  tcp dpt:14002

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
[storjnode]:</home/mike/>
[storjnode]:</home/mike/> cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-pol
cy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
[storjnode]:</home/mike/>

Check that host here is correct and can be resolved.

Nope, doesnā€™t look like it, other than with nslookup

storjnode (172.17.0.1)                                 2023-03-16T00:29:55-0500
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. (no route to host)

woops, not with nslookup either

Iā€™ve also, never seeb that name before, anywhere

Then perhaps itā€™s better to remove them.
After that you likely will need to stop and remove the container and run it back using all your parameters.
Regarding the firewall, well it could be enabled by default, you may check:

sudo ufw status

are you running baremetal or in a hypervisorā€¦
i had one case where my hypervisor dns was wrong which cut the internet access for most of my vmā€™s.

happened because i was changing between ispā€™s and thus had two ispā€™s for a couple of months while everything was being reconfigured, so it ofc worked while both was active but then as the old one was disconnected, i started seeing issues, took me a bit to realize it was the hypervisor dns that make every container not work on that server.

come to think about it, it might only have been the containers that was affected, but still figured i would mention it, just in case.

[storjnode]:</home/mike/> sudo ufw status
Status: inactive

Vmware 5.5

any / all my other vms, are fine.

Have you removed wrong rules?

I donā€™t know what rules you are talking about.

the ufw status is inactive.

These rules, they filter the traffic to not existing host:

Ok, well now iā€™ve done it, lol

i donā€™t know anything about iptables , so i did a quick search and found the commands to remove all rules chains, whatever


[storjnode]:</home/mike/> sns
515b7b7b468571efa374c3ef0b3e018993ae6e175675025ca8f8ca0517de5108
docker: Error response from daemon: driver failed programming external connectivity on endpoint storagenode (ed36115e4170adbfc871c7425c6609fabeb5f6b9d24255f9d0700cc642582129): (iptables failed: iptables --wait -t filter -A DOCKER ! -i docker0 -o docker0 -p tcp -d 172.17.0.3 --dport 28967 -j ACCEPT: iptables: No chain/target/match by that name.
(exit status 1)).
[storjnode]:</home/mike/>

| Alexey Leader
March 18 |

  • | - |

These rules, they filter the traffic to not existing host: