[Tech Preview] Storage Node Multinode Dashboard

This is a very early test version of the multinode dashboard. Yes, it is difficult to install and not all of you will have the tools required to compile this. Yes, the multinode dashboard has a lot of bugs and is essentially useless at the moment. If you still want to try it here is what you have to do.

First, compile it including the web interface. For the moment we have to use the main branch because there are a few commits that are not included in the latest release. Remember that this comes with a risk. Long term you might want to use the latest release tag instead. At some point, we will publish binaries on github so that you can download und setup the multinode dashboard without having to compile it.

git clone https://github.com/storj/storj -b main $HOME/multinode
cd $HOME/multinode
go install ./cmd/multinode
go install ./cmd/identity
cd web/multinode
npm install
npm run build

After that let’s set up the multinode dashboard. Yes, we are aware that this step should be 2 lines and not 4. We are working on that already.

identity create multinode --difficulty 1
multinode setup --console.static-dir "$HOME/multinode/web/multinode"
multinode create-schema
multinode run

Now you should be able to see the multinode dashboard running on port 15002 by default. Open it in your webbrowser. In order to add a storage node to the multinode dashboard you need a storage node api key.

Please only add storage nodes that are running at least v1.22.2. If you add an older storage node the multinode dashboard will break and stop displaying even the nodes you had added successfully earlier. You would be forced to manually delete the entry from the SQLite database.

storagenode issue-apikey --log.output stdout --config-dir ... --identity-dir ...

After adding your first storage node you will notice that the dashboard is showing only a very limited amount of information and in many fields even display wrong values. For the moment I would just ignore these details. This early test is more about getting you prepared for the next round.

Good luck everyone.

8 Likes

Is there a possibility of dashboard actually breaking something on nodes or it’s just a display tool?

It should be display only. If you manage to crash your storage node we would have to fix that asap.

Is it possible to post some pictures for those who don’t want to go through the installation process? (or can’t)

5 Likes

Alright, will test it on nodes I don’t value :slight_smile:

2 Likes

It worked well up until the point of me adding the node. Docker is still on 1.21.2. Too bad.

Can it use internal dashboard instead of going over internet?

Yes it should. I would expect to be able to add localhost:13010 (storj-sim port mapping)

You can open the sqlite3 db. .schema will help you.

Yes, the unfortunate issue is all my nodes run in docker so I don’t have a node to connect to yet. Maybe one day I’ll move them to native linux.

Hello.
Ubuntu 18.04

    root@ubuntu:~/multinode# go install ./cmd/multinode
cmd/multinode/main.go:11:2: cannot find package "github.com/spf13/cobra" in any of:
        /usr/src/github.com/spf13/cobra (from $GOROOT)
        /root/go/src/github.com/spf13/cobra (from $GOPATH)
cmd/multinode/main.go:12:2: cannot find package "github.com/zeebo/errs" in any of:
        /usr/src/github.com/zeebo/errs (from $GOROOT)
        /root/go/src/github.com/zeebo/errs (from $GOPATH)
cmd/multinode/main.go:13:2: cannot find package "go.uber.org/zap" in any of:
        /usr/src/go.uber.org/zap (from $GOROOT)
        /root/go/src/go.uber.org/zap (from $GOPATH)
cmd/multinode/main.go:15:2: cannot find package "storj.io/common/fpath" in any of:
        /usr/src/storj.io/common/fpath (from $GOROOT)
        /root/go/src/storj.io/common/fpath (from $GOPATH)
cmd/multinode/main.go:16:2: cannot find package "storj.io/private/cfgstruct" in any of:
        /usr/src/storj.io/private/cfgstruct (from $GOROOT)
        /root/go/src/storj.io/private/cfgstruct (from $GOPATH)
cmd/multinode/main.go:17:2: cannot find package "storj.io/private/process" in any of:
        /usr/src/storj.io/private/process (from $GOROOT)
        /root/go/src/storj.io/private/process (from $GOPATH)
cmd/multinode/main.go:18:2: cannot find package "storj.io/storj/multinode" in any of:
        /usr/src/storj.io/storj/multinode (from $GOROOT)
        /root/go/src/storj.io/storj/multinode (from $GOPATH)
cmd/multinode/main.go:19:2: cannot find package "storj.io/storj/multinode/multinodedb" in any of:
        /usr/src/storj.io/storj/multinode/multinodedb (from $GOROOT)
        /root/go/src/storj.io/storj/multinode/multinodedb (from $GOPATH)

So far it look promising :slight_smile: Had to build own docker test image on 1.22.2 as docker only at 1.21.2. The adding a new node is picky - had to add port to IP address, else it create incorrect entry in DB and had to rest database as I lazy.

Looking forward to seeing this progress, still in very alpha stages so not pointing at my Live node.

Screen shot for those who interested - CP

6 Likes

Your golang installation is incorrect.

Thanks, it worked.
We are waiting for the docker update on 1.22.2
screenshot

I apologize for the stupid question, but where can I find the api key for the node?

How about the search function? Inlcuding my post now there are 4 hits for “api” in this threads. I am sure you can find the other 2 hits right at the beginning :smiley:

1 Like

I not sure, but I had to build own docker image on 1.22.2 as it not on hub for testing, or you could compile from source already on machine go install ./cmd/storagenode (although not sure that good idea)

I setup node like normal, generated new identity, and run initial setup - then made changes to config.yml to limit node size to 1GB, as don’t want to be storing data.

Then when node running normal, ran command below to generate API key.

docker exec -it storagenode /app/storagenode issue-apikey --config-dir /app/config --identity-dir /app/identity

You then can find, NodeID and then under you get API key you need both of these plus IP address to add node to dashboard.

yes thanks for the hint. set everything up, looked, closed :slight_smile:

I’m testing this out with a Docker node and I’m unable to add a node. I get the following error:

2021-02-18T12:15:44.326-0700    ERROR   console:endpoint        controllers/nodes.go:190        list node infos internal error  {"error": "nodes service error: rpc: tls: first record does not look like a TLS handshake", "errorVerbose": "nodes service error: rpc: tls: first record does not look like a TLS handshake\n\tstorj.io/common/rpc.TCPConnector.DialContext:91\n\tstorj.io/common/rpc.Dialer.dialEncryptedConn:180\n\tstorj.io/common/rpc.Dialer.DialNodeURL.func1:101\n\tstorj.io/common/rpc/rpcpool.(*Pool).Get:87\n\tstorj.io/common/rpc.Dialer.dialPool:146\n\tstorj.io/common/rpc.Dialer.DialNodeURL:100\n\tstorj.io/storj/multinode/nodes.(*Service).ListInfos.func1:90\n\tstorj.io/storj/multinode/nodes.(*Service).ListInfos:149\n\tstorj.io/storj/multinode/console/controllers.(*Nodes).ListInfos:188\n\tnet/http.HandlerFunc.ServeHTTP:2042\n\tgithub.com/gorilla/mux.(*Router).ServeHTTP:210\n\tnet/http.serverHandler.ServeHTTP:2843\n\tnet/http.(*conn).serve:1925"}
storj.io/storj/multinode/console/controllers.(*Nodes).ListInfos
        /root/multinode/multinode/console/controllers/nodes.go:190
net/http.HandlerFunc.ServeHTTP
        /usr/lib/go-1.15/src/net/http/server.go:2042
github.com/gorilla/mux.(*Router).ServeHTTP
        /root/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210
net/http.serverHandler.ServeHTTP
        /usr/lib/go-1.15/src/net/http/server.go:2843
net/http.(*conn).serve
        /usr/lib/go-1.15/src/net/http/server.go:1925

The console endpoint for the storagenode is http-only and is available via http://<IP>:<PORT> on a different machine on the LAN.

Any insight on how to resolve this? Thanks.

1 Like

You need to add the private port. So the port that you are sending to the satellite is also the port you have to enter in the multinode dashboard.

I have updated my multinode dashboard to the latest version and I also added all of my storage nodes. I can now see the free space and the traffic of all of my nodes. I was watching especially my free space. That is working quite good. I hope soon the payout column will also get corrected.

I am now running my multinode dashboard as a service similar to the storage node service.

# This is a SystemD unit file for the Storage Node
# To configure:
# - Update the user and group that the service will run as (User & Group below)
# - Ensure that the Storage Node binary is in /usr/local/bin and is named storagenode (or edit the ExecStart line
#   below to reflect the name and location of your binary
# - Ensure that you've run setup and have edited the configuration appropriately prior to starting the
#   service with this script
# To use:
# - Place this file in /etc/systemd/system/ or wherever your SystemD unit files are stored
# - Run systemctl daemon-reload
# - To start run systemctl start storagenode

[Unit]
Description  = Storage Node service
After        = network-online.target
Wants        = network-online.target

[Service]
User         = storagenode
Group        = storagenode
ExecStart    = /home/storagenode/go/bin/multinode run
Restart      = always
Type         = simple
NotifyAccess = main

[Install]
WantedBy     = multi-user.target

And I am writing everything into a logfile. On my multinode config file:

# can be stdout, stderr, or a filename
log.output: /home/storagenode/multinode1.log

<facepalm>. I was trying to use the API port instead. I’ll replace with the node private port and try again. Thanks!