Looking for help on a project/experiment [x10 nodes]

Hello!

Looking for some help on a project/experiment.

Project
This will involve 10x storj nodes.
5x will be hosted locally, sharing the same public IP.
5x will be hosted remotely, each with unique locations and public IPs.
Not within the same public /24 subnet.

These will all be turned on within a reasonable short time of each other.

Equipment
Odriod-HC2 with 6tb to 12tb drives to each node. Where I need help is not well versed in Linux or Docker. Metrics and analytics are a requirement: ping, cpu load, memory usage, bandwidth usage, I/O performance, disc volume tracking. To compare 5x nodes centralized vs 5x decentralized.

Have a server with EXSi and static IPs locally for a centralized motoring system that everything can be connected back for reporting.

Diagram

The monitoring system and equipment is etched in pencil. Seen grafana + influxDB + prometheus monitoring on the forums. I’m all ears on other options to accomplish the same goal. Even if there is a cheaper or more solid solutions than the Odriod-HC2. Granted these have an output of 25w.
Which is around $24.28/per node/per year depending on the kWh charge in your area.
Calculation above is based off: $00.115 per kWh charge.

Requirements

  • 10x nodes [5x local, 5x remote]
  • Remote access to remote nodes
  • Monitoring on local and remote nodes: ping, cpu load, memory usage, bandwidth usage, I/O performance, disc volume tracking

Will publicize the findings from this project/experiment as an on going blog.

Anyone up for helping me turn this idea into a reality?

2 Likes

Upping the ante. Giving away a free node, type used for the project above, to the one that helps out.

What do you need help with? I mean there are some general things you wrote, but what specific parts need help? Configuring the VPN? Installing a node? Something else?

Just out of curiosity, what kind of trends are you looking to find?

Setting up the nodes. Linux/docker, Storj nodes, and VPN connections.

I can get the VPN concentrator up. Not sure which vpn will be best. Sstp, pptp, L2tp, openvpn, wireguard.

I would prefer a sstp if that can be configured on linux(arm)

Want to see how much more the nodes fill up as a decentralized solutions vs a stacked local, IP filtered solution.

So… mmm… pretty much everything :)?

By the way, I prefer OpenVPN and use it pretty much everywhere I can. Where I cannot do that (because the outer on one side is a Mikrotik that only partially supports OpenVPN or something that does not support OpenVPN at all), I just use IPSec.

I guess you can ask questions and I (or somebody else) can answer them. I am not good with providing long lectures on a broad subject, I am much better at answering specific questions.

My core router is a Mikrotik CCR. Have used a Mikrotik RB4011 as my VPN concentrator for 300+ deployment around the United States for my company. This is my experience. You tend to want to use what you have in the past. Not sure it’s the best solution. The VPN connection can not be the default route. Want to use the public IP for the node. But need the VPN access for collecting the SNMP.

Granted I was connecting all my devices via sstp to the VPN concentrator. Main reason I liked this protocol it it’s similar to openvpn but easier to setup and runs of port 443. No issue on the ISP provider blocking it.

Shoot me a PM @Pentium100

While I don’t personally have experience with the HC2 in particular, @will.topping made an excellent guide for getting them set up. Definitely a good starting point.

Watched the video and have chatted with will.topping.

This video changed the way I wanted to deploy my project.

How so? Keep me posted…I’ve been looking to try something similar. Currently sitting on five nodes that need to be deployed similarly.

Looking at using softether for the vpn with [sstp, openvpn or SSL vpn] all will work through firewalls

Going to start with zabbix for snmp monitoring, running a server and agents on the HC2s. WHich I need the vpn remote access for snmp collection.

I would help you, but dont know how.
There are any problems left?

What I need help with is getting NetXMS Agent on my Odroid-HC2. The documentation is dated for the newer versions released and not well versed with the Linux/Ubuntu.

NetXMS
NetXMS Agent Binaries
Available Documentation

Can do an AnyDesk RDP session for remote access.

Dropped NetXMS for Zabbix. Have grafana connected and working through zabbix.

Final part is getting a management vpn solutions for the odriod for remote access through http(80)/https(443) protocols. Then firewalls won;t be an issue.

Looking at Softether, wireguard, OpenVPN, SSTP (linux) solutions.
have softether connected, but its routing all my traffic through it. I only need management. Anyone with experience with Softether or vpns?

i use PFsence very good router, VPN. have own linux os

Softether is a multi protocol application, which actual VPN protocol or you running?

If you use IPSec, you can try Shrewsoft VPN client, this has an option to only route traffic that is targetted to the remote network through the VPN

Need anything that will pass through http/https so I dont have to deal with carrier/isp firewalls. As well as be a management access only. Meaning the default gateway is still the local public IP.

From the softether not even sure which protocl it was using when it connected. THink its was the native softether one.

It made me create a virtual interface and I believe that was my problem, cause then it routed all my traffic out through the vpn server.

1 Like

New diagram with greater detail, everything else is working as it should besides the vpn tunnel.

Currently trying a solutions @Pentium100 using openvpn.

1 Like

Hope it will run soon :slight_smile:

That is not necessarily the cause of your problem. You can usually setup the routing in a way that sends everything to your default gateway EXCEPT for traffic that is targetted to your remote VPN network