Multinode Setup Bug

ptdatta · September 6, 2024, 8:20am

I am able to setup the multinode. However when adding a node(from here) I am getting this error

Which I know comes from the beforeEach guard of the router. On clicking the Add Node the vue store should dispatch a node/add event which is working fine(no error in that).
await this.$router.push(RouterConfig.MyNodes.path);
This is the line responsible for the error. Which always call the beforeEach guard of the router. It check if node added or not. Which fails in my case. The value for store.state.nodes.nodes.length always comes 0. Which causes the error. However, the nodes/add dispatch event works fine.
Is it a bug or I am missing something? I tried with both storj-sim and multinode run.

Alexey · September 7, 2024, 2:47am

I believe it’s a bug, but seems not affecting the business logic.

I don’t think so, and it’s easy to check - just call a GET method for the hostname with port.
You can always use nodes from storj-up or storj-sim for tests.

ptdatta · September 7, 2024, 2:48am

I am just logging to the server with one of the nodes and got these

rpc: tcp connector failed: rpc: dial tcp: lookup awn7k09ts6mxbgau.myfritz.net on 127.0.0.53:53: no such host

Alexey · September 7, 2024, 2:51am

yep.
But you may also just check it like this: http://awn7k09ts6mxbgau.myfritz.net:13010
If the node is alive, you should receive something like that:

{
  "Statuses": null,
  "Help": "To access Storagenode services, please use DRPC protocol!",
  "AllHealthy": true
}

So, it’s better to use your own nodes instead. You may run them with storj-up or storj-sim. And even publish them through your router’s NAT.

To generate an API key for docker nodes:

docker exec -it storagenode223 ./storagenode info --config-dir config --identity-dir identity

The public IP would be empty, but you may take it as your localhost and the related port, the port you can see with

docker ps

for binary nodes (storj-sim)

./storagenode info --config-dir /path/to/config --identity-dir /path/to/identity

ptdatta · September 7, 2024, 3:28am

I initially thought that I was doing something wrong so I started logging which actually helped me to understand the whole multinode codebase from the client to cmd to server. But after coming to the server realised the node wasn’t valid so I tried with the storj-up nodes and it worked.
Thank you for the help.

Alexey · September 7, 2024, 5:50am

You are welcome!
Do not hesitate to ask questions, we would try to help.
Even if we do not know something, we can always invite our developers.

ptdatta · September 7, 2024, 7:43am

@Alexey I used satellite info inside my storagenode and I got this

ID: 1PastuS5ce9TT2A5AsCf5Qkwnrt2cCwVTkssAjJY9vN1GuHvYF
API Secret: QMBO3ars4TjdELzzfcx203yxpdf2DNq2q5KTAYTMV3A=
Public Address: storagenode1:30001

So I ran the multinode add API(http://localhost:15002/api/v0/nodes) with it but I got rpc: tcp connector failed: rpc: dial tcp: lookup storagenode1 on 127.0.0.53:53: server misbehaving this error
So I thought there might be a DNS resolution problem so I tried the Public Address with 172.18.0.7:30001 but the API goes loading and then timed out
For localhost:30001 and 0.0.0.0:30001 I got rpc: tcp connector failed: rpc: dial tcp 127.0.0.1:30001: connect: connection refused
For localhost:30000 and ``0.0.0.0:30000 I got rpc: tcp connector failed: rpc: tls: first record does not look like a TLS handshake
So I tried to look inside the DRPC protocol but didn’t find any solution.

Alexey · September 7, 2024, 7:50am

It should be storagenode info, not a satellite info

However, the output looks valid.

Did you change the IP for storj-up? then you need to restart the whole stack. I do not know, why is that.
So,

docker compose restart

or

docker compose stop

and

docker compose start

should fix this issue. However, you need to wait until the satellite would be fully available I guess for any operations, because nodes would likely restarting every time until it would be available.

For that you need to be in the same network. I do not know details of your setup, but perhaps your docker-compose.yaml does not have options to expose these ports to your host or it’s limited to a some IP (like a localhost).

ptdatta · September 7, 2024, 7:52am

Yes yess sorry it’s storagenode info

ptdatta · September 7, 2024, 8:10am

I did not change the IP for storj-up. Tried restarting the system but it still had the same issue.

Alexey · September 7, 2024, 8:57am

I see, could you please show the part of the docker-compose.yaml file, where you forwarded nodes ports to the host?

ptdatta · September 7, 2024, 12:44pm

Here it is

storagenode1:
    command:
    - storagenode
    - run
    - api
    - --defaults=dev
    - --identity-dir=/var/lib/storj/.local/share/storj/identity/storagenode/
    deploy:
      replicas: 1
    environment:
      STORJUP_AUTHSERVICE: http://authservice:8888
      STORJUP_ROLE: storagenode
      STORJUP_SATELLITE: satellite-api
      STORJ_CONSOLE_ADDRESS: 0.0.0.0:30000
      STORJ_CONTACT_EXTERNAL_ADDRESS: storagenode1:30001
      STORJ_DEBUG_ADDR: 0.0.0.0:11111
      STORJ_DEFAULTS: dev
      STORJ_IDENTITY_DIR: /var/lib/storj/.local/share/storj/identity/storagenode/
      STORJ_LOG_LEVEL: debug
      STORJ_METRICS_APP_SUFFIX: sim
      STORJ_OPERATOR_EMAIL: storage9@mail.test
      STORJ_OPERATOR_WALLET: "0x0123456789012345678901234567890123456789"
      STORJ_ROLE: storagenode
      STORJ_SERVER_ADDRESS: 0.0.0.0:30001
      STORJ_SERVER_EXTENSIONS_REVOCATION: "false"
      STORJ_SERVER_PRIVATE_ADDRESS: 0.0.0.0:30002
      STORJ_SERVER_USE_PEER_CA_WHITELIST: "false"
      STORJ_STORAGE2_MONITOR_MINIMUM_DISK_SPACE: "0"
      STORJ_STORAGE2_TRUST_SOURCES: 12whfK1EDvHJtajBiAUeajQLYcWqxcQmdYQU5zX5cCf6bAxfgu4@satellite-api:7777
      STORJ_STORAGE_ALLOCATED_DISK_SPACE: 1G
      STORJ_VERSION_SERVER_ADDRESS: http://localhost:8080
      STORJ_WAIT_FOR_SATELLITE: "true"
    image: img.dev.storj.io/storjup/storj:1.108.2
    networks:
      default: null
    ports:
    - mode: ingress
      target: 30000
      published: 30000
      protocol: tcp
    volumes:
    - type: bind
      source: /home/parthib/go/bin/storagenode
      target: /var/lib/storj/go/bin/storagenode
      bind: {}
    - type: bind
      source: ./web/storagenode
      target: /var/lib/storj/web/storagenode
      bind: {}
  storagenode10:
    command:
    - storagenode
    - run
    - api
    - --defaults=dev
    - --identity-dir=/var/lib/storj/.local/share/storj/identity/storagenode/
    deploy:
      replicas: 1
    environment:
      STORJUP_AUTHSERVICE: http://authservice:8888
      STORJUP_ROLE: storagenode
      STORJUP_SATELLITE: satellite-api
      STORJ_CONSOLE_ADDRESS: 0.0.0.0:30090
      STORJ_CONTACT_EXTERNAL_ADDRESS: storagenode10:30091
      STORJ_DEBUG_ADDR: 0.0.0.0:11111
      STORJ_DEFAULTS: dev
      STORJ_IDENTITY_DIR: /var/lib/storj/.local/share/storj/identity/storagenode/
      STORJ_LOG_LEVEL: debug
      STORJ_METRICS_APP_SUFFIX: sim
      STORJ_OPERATOR_EMAIL: storage9@mail.test
      STORJ_OPERATOR_WALLET: "0x0123456789012345678901234567890123456789"
      STORJ_ROLE: storagenode
      STORJ_SERVER_ADDRESS: 0.0.0.0:30091
      STORJ_SERVER_EXTENSIONS_REVOCATION: "false"
      STORJ_SERVER_PRIVATE_ADDRESS: 0.0.0.0:30092
      STORJ_SERVER_USE_PEER_CA_WHITELIST: "false"
      STORJ_STORAGE2_MONITOR_MINIMUM_DISK_SPACE: "0"
      STORJ_STORAGE2_TRUST_SOURCES: 12whfK1EDvHJtajBiAUeajQLYcWqxcQmdYQU5zX5cCf6bAxfgu4@satellite-api:7777
      STORJ_STORAGE_ALLOCATED_DISK_SPACE: 1G
      STORJ_VERSION_SERVER_ADDRESS: http://localhost:8080
      STORJ_WAIT_FOR_SATELLITE: "true"
    image: img.dev.storj.io/storjup/storj:1.108.2
    networks:
      default: null
    ports:
    - mode: ingress
      target: 30090
      published: 30090
      protocol: tcp
    volumes:
    - type: bind
      source: ./web/storagenode
      target: /var/lib/storj/web/storagenode
      bind: {}

ptdatta · September 7, 2024, 7:40pm

Hey @Alexey rpc: tcp connector failed: rpc: tls: first record does not look like a TLS handshake Can you please tell me why it’s happening in localhost:30000?

Alexey · September 8, 2024, 2:22am

You need to publish this port, because the external address is used to contact the node. I.e.:

    ports:
    - mode: ingress
      target: 30000
      published: 30000
      protocol: tcp
    - mode: ingress
      target: 30001
      published: 30001
      protocol: tcp

this is a single node dashboard port, it can be used to see the single node dashboard in your browser, e.g. http://localhost:30000, and some API calls, like a reputation and usage, but it’s an insecure http protocol.
The Multinode dashboard uses dRPC and secure channel to connect the storagenode port, i.e. 30001.

ptdatta · September 8, 2024, 4:20am

Oh, yes. How can I miss that? Thank you so much for your help.

Alexey · September 11, 2024, 6:55am

3 posts were split to a new topic: Multinode payout information bug

Alexey · September 11, 2024, 7:44am

A post was split to a new topic: Should I implement the notification system for satellite to the multinode dashboard?