Multinode blank page

Node Operators troubleshooting
,
1

Hi all,
Main question: is multinode dashboard maintained, does it have any sense to keep try to launch it?
Earlier, i had bug when some node was offline, dashboard page was blank, now even when all nodes are online it is blank(Tried to pull new image / rm old container / created new one, resulted in blank page always).

Here is command im using to launch it:
docker run -d --restart unless-stopped
-p 15002:15002/tcp
–mount type=bind,source=“/opt/multinode/”,destination=/app/config
–name multinode storjlabs/multinode:latest
Docker has access to opt folder, cause after container launch i see that it adds 2 files related to DB connection.
Here is logs after container launch:

image
image763×151 77.1 KB

And here is when i try to open dashboard in browser:
image
image760×265 60.7 KB

i believe this one is somehow related to cert (im using nginx proxy manager/lets encrypt cert to get access to dashboard), strange that browser tells that everything is ok with it.

with http i got more interesting error:

image
image764×264 58.8 KB

2
3

You need to check logs of all your nodes to figure out, which one could publish the same port (-e ADDRESS option for docker or contact.external-address: in the config.yaml for nodes installed as a service).

4

i have multiple nodes in 4 locations with same ports (unique ports on specific location).
ports should be unique on all of them, is this a problem?

5

No, if the IP is different, I have 2 nodes on port 14002 and two on port 14003, different machines, and the MND works
The ports to the outside must be different, 28967, 28968…
but if you are on a single IP you necessarily have to change them, but you have 4 IP’s….

6

Ports per location are unique, like:
ip1:28967, ip1:28968
ip2:28967, ip2:28968, ip3:28969
and so on.

will try to re setup everything from scratch…
wanted to avoid procedure of adding 18 nodes back inside it.

7

No need to setup everything from scratch. It could be a mentioned problem with MND itself, like when you registered the node with IP1:28967, then changed it to IP1:28968, and started a new one on a previously used IP1:28967.
But MND would think that IP1:28967 is still belongs to the node1. In that case you need to re-add this node, i.e. remove from the MND and add it back but with correct IP and port.

MND uses the node’s port, not the Single Node Dashboard’s port.

8

oh dear… rlly, i did a swap for some nodes between locations A<>B.
i will need to setup it from scratch cause i cant make any changes in nodes list, MND page is blank, cant do anything with it…

9

You actually can use the SQLite binary to modify its database, or take a binary, built by @ptdatta .

With a database change:

  1. Stop multinode
  2. execute:
docker exec -it --rm -v /opt/multinode/:/data sstc/sqlite3 sqlite3 master.db

it would open an SQLite prompt sqlite> and you may see what’s values you provided:

select name, public_address from nodes;

then update them, e.g.

update nodes set public_address="your_address:28968" from nodes where public_address="your_address:28967";

and exit

.exit
10

It you want any features or find bugs in the multinode UI. Please add it here I will gradually work on them.
Currently I am little occupied so I am inactive to the community.

3 Likes
11

Blank Page PR is now live.

@clement can you please review these PRs, they are waiting for so long.

1 Like
12

Interesting, tried to add node on completely new multinode setup:
In multinode container log i found next:

2024-10-31T20:05:06Z    ERROR   console:endpoint        controllers/nodes.go:76 could not add node      {"error": "nodes: rpc: tcp connector failed: rpc: tls peer certificate verification: tlsopts: peer ID did not match requested ID", "errorVerbose": "nodes: rpc: tcp connector failed: rpc: tls peer certificate verification: tlsopts: peer ID did not match requested ID\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
storj.io/storj/multinode/console/controllers.(*Nodes).Add
        /go/src/storj.io/storj/multinode/console/controllers/nodes.go:76
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2171
github.com/gorilla/mux.(*Router).ServeHTTP
        /go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210
net/http.serverHandler.ServeHTTP
        /usr/local/go/src/net/http/server.go:3142
net/http.(*conn).serve
        /usr/local/go/src/net/http/server.go:2044

Node online, dashboard accessible, and im pretty sure that everything is ok with it.

In same time second node from this machine was successfully added.
node 1 port = 28967(default)
node 2 port = 28968
Checked configs files for both nodes and this param same:

# the public address of the node, useful for nodes behind NAT
contact.external-address: ""

Nodes in docker looks like:
image

13

This may mean, that you still have an old node in the database, which was with a different identity, but with the same external address and port as a new one.

14

100% new setup.
so i deleted old container, and created new one and pointed to new folder:

docker run -d --restart unless-stopped \
    -p 15002:15002/tcp \
    --mount type=bind,source="/opt/multinode/",destination=/app/config \
    --name multinode storjlabs/multinode:latest

and new one:

docker run -d --restart unless-stopped \
    -p 15002:15002/tcp \
    --mount type=bind,source="/opt/multinode1/",destination=/app/config \
    --name multinode storjlabs/multinode:latest
15

In that case the provided NodeID and the external address and port doesn’t match. I.e. you copied NodeID from the one node, but provided either API key or external address and port from another one.
By the way, if your multinode dashboard resides in the same LAN, you may use the local IP instead of external one. Just make sure that you specified a correct port, NodeID and API key.
I would recommend to use

docker exec -it storagenode /app/bin/storagenode info --config-dir config --identity-dir identity

and

docker ps storagenode

to see the external port (it would be on the left side of the port mapping).