FreeBSD (FreeNas, TrueNas) missing guide

Hello everyone,

Hopefully someone can help me out or point me in the right direction.

I have decided to run storagenode.

I do have Truenas that I Have approx 16tb of memory unused.
Because Network storage is not recommended can not install it on my windows server.
And I would prefer to run the node from Truenas jail.

I found binaries here for freebsd: Releases · storj/storj · GitHub but have zero ideas how to run everything.
So was hoping that someone could point me in the right direction to some guides or step by step setup how to use correctly binaries.

Thanks in advance

1 Like

It works fine with Iscsi, just not very well with NFS.

You can create an ISCSI “share” using TrueNas term, and mount this on Windows using ISCSI initiator… It’s great, as the drive appears in Windows as if it was a hard disk on the machine.

You will need 1GB minimum network between your windows box, and TrueNas

https://www.truenas.com/docs/core/sharing/iscsi/iscsishare/

Docker was supported on FreeNas, I never bothered with TrueNas as memory footprint was huge :frowning: but I’m sure someone else in community can give some pointers if you want to go that way.

Thank you for your advice. I just tried ISCSI approach and still got the same error when running the installation. ‘There is a problem with this Windows installer package Contact your support personnel or package vendor.’

So that was a bust :smiley:

I have two storagenodes running as jails on my TrueNAS server, maybe I should write a guide sometime.

Each are configured using their own VNET and IP address. On the first I have separate vdevs/mountpoints for storage, database and logs and on the second just one vdev/mointpoint containing all three. You can make it as simple/complicated as you like.

As this isn’t a docker environment you need to download and run the storagenode freebsd binary from here: Releases · storj/storj · GitHub
I’ve installed and configured supervisord to handle restarts and similarly logrotate to rotate the node log.

For ease I used my windows laptop to generate the identities as it has better CPU resources than my NAS.

BSD doesn’t like you replacing running binaries so I wrote a script run by cron that periodically checks for updates and downloads them automatically and then notifies me that there’s a new version downloaded and ready.
I have another script I run manually when I’m ready, that stops the supervisord storagenode service, replaces the current binary and restarts the service.

I can share more details if it’s something you’re interested in.

So far I generated the identities on my pc and I am quite familiar with Jail network setup.

I have downloaded the binaries.
So at the moment I would approach this with baby steps.

I am very interested in the actual script that runs the storagenode as I could not get that even up and running.

if you could share your scripts here it would be absolutely amazing.
And I could try to generate step by step guide for others

1st thing i do is create a storj user that i run everything as: on both my nodes it’s: uid=1002(storj) gid=1002(storj) groups=1002(storj)

after installing supervisord i create /usr/local/etc/supervisor/conf.d/storagenode.conf
mine contains:

[program:storagenode]
command=/home/storj/bin/storagenode run
directory=/home/storj
autostart=true
autorestart=true
startretries=3
stderr_logfile=NONE
stdout_logfile=NONE
user=storj
environment=HOME="/home/storj",USER="storj"

cat /usr/local/etc/logrotate.d/storagenode 
    /var/log/storj/node.log {
        daily
        su storj storj
        rotate 366
        copytruncate
        notifempty
        missingok
        dateext
        extension .log
        dateyesterday
        compress
        compresscmd xz
        delaycompress
    }

That should get you going, I may want to publish my scripts via github.

Don’t I need to run the installer or smthing to configure storagenode first ?

Indeed you should run storagenode with the setup command with the appropriate args for your install. Without arguments the default is to create the config at $HOME/.local/share/storj/storagenode/config.yaml,
look for the identity information in $HOME/.local/share/storj/identity/storagenode/ and store files at $HOME/.local/share/storj/storagenode/storage/

I personally keep the default locations for the config and identity and customize the storage.path in the config file once it has been created (as well as other settings). I also use a dedicated storj user and group to run the storagenode.

These are the config options I use:

# ensures console is accessible from your network on port 14002
console.address: 0.0.0.0:14002
# address and port to find your server
contact.external-address: "node.wibble.com:28967"
# ensure the log directory is owned by the user running the storagenode
log.output: "/var/log/storj/node.log"
# the email address used by storj to contact you with updates etc.
operator.email: "user@wibble.com"
#only if you opt in to zksync
operator.wallet-features: ["zksync"]
operator.wallet: "<wallet string starting 0x>"
# the port used for storage access, you should forward traffic on both UDP and TCP from contact.external-address to this port
server.address: :28967
storage.path: /mnt/storj/storagenode/storage
storage.allocated-disk-space: 750 GB

To avoid a warning about not being able to increase UDP buffer size I also the the following tunable in TrueNAS itself:
kern.ipc.maxsockbuf 30146560 SYSCTL

./storagenode help
StorageNode


Usage:
  storagenode [command]

Available Commands:
  config         Edit config files
  dashboard      Display a dashboard
  diag           Diagnostic Tool support
  exit-satellite Initiate graceful exit
  exit-status    Display graceful exit status
  help           Help about any command
  issue-apikey   Issue apikey for mnd
  run            Run the storagenode
  setup          Create config files
  version        output the version's build information, if any

Flags:
      --color                            use color in user interface
      --config-dir string                main directory for storagenode configuration (default "/home/pi/.local/share/storj/storagenode")
      --db.conn_max_lifetime duration    Maximum Database Connection Lifetime, -1ns means the stdlib default (default 30m0s)
      --db.max_idle_conns int            Maximum Amount of Idle Database connections, -1 means the stdlib default (default 1)
      --db.max_open_conns int            Maximum Amount of Open Database connections, -1 means the stdlib default (default 5)
      --debug.addr string                address to listen on for debug endpoints (default "127.0.0.1:0")
      --debug.trace-out string           If set, a path to write a process trace SVG to
      --defaults string                  determines which set of configuration defaults to use. can either be 'dev' or 'release' (default "release")
  -h, --help                             help for storagenode
      --identity-dir string              main directory for storagenode identity credentials (default "/home/pi/.local/share/storj/identity/storagenode")
      --log.caller                       if true, log function filename and line number
      --log.development                  if true, set logging to development mode
      --log.encoding string              configures log encoding. can either be 'console', 'json', or 'pretty'.
      --log.level Level                  the minimum log level to log (default info)
      --log.output string                can be stdout, stderr, or a filename (default "stderr")
      --log.stack                        if true, log stack traces
      --metrics.addr string              address(es) to send telemetry to (comma-separated) (default "collectora.storj.io:9000")
      --metrics.app string               application name for telemetry identification (default "storagenode")
      --metrics.app-suffix string        application suffix (default "-release")
      --metrics.instance-prefix string   instance id prefix
      --metrics.interval duration        how frequently to send up telemetry (default 1m0s)
      --tracing.agent-addr string        address for jaeger agent (default "agent.tracing.datasci.storj.io:5775")
      --tracing.app string               application name for tracing identification (default "storagenode")
      --tracing.app-suffix string        application suffix (default "-release")
      --tracing.buffer-size int          buffer size for collector batch packet size
      --tracing.enabled                  whether tracing collector is enabled
      --tracing.interval duration        how frequently to flush traces to tracing agent (default 0s)
      --tracing.queue-size int           buffer size for collector queue size
      --tracing.sample float             how frequent to sample traces

Hi @hobitsbat -

Something that you may find useful is some work that was done around FreeNAS integration: GitHub - storj-thirdparty/freenas-storagenode-app: FreeNAS application providing packaging to Storj storagenode tools

Best of luck!

-K

Thank you for all the replies, I did see the github work that was done, but it does not work with V3 any more :frowning:

Just got back from travels. Will try to use binaries directly.

I will post the outcome of this later Today.

Okey Getting somewhere.

My node is up and running, happy times.

Looking at logs now And I was getting a lot of this:

2021-06-05T11:54:37.207+0100 ERROR contact:service ping satellite failed {“Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “attempts”: 1

Should I be worried ?
P.S. The Node seems to be up and running.

Yes, you should. Your node is not reachable from outside, i.e. it’s offline.
Please, use this check-list to troubleshoot:

An interesting thing is that I just install pm2 to auto launch the app and after full reboot the errors are gone. I won’t cycle the logs, not yet at least just to make sure everything is up and running, but this far everything

1 Like

I used my Freenas system as well, didn’t know there were plugins/packages for it though. I just set up an ubuntu server VM and did the normal storj install within that, nothing special needed. Would be great if there was an official plugin to do it all though!

Still making my update scripts production ready. Will have them ready over the next week - you won’t get blocked before then…
Just about completed my update script that downloads updates when they are ready after a re-thought on how it should work (it has all the intelligence now). Now need to update my restart script that can now be stupid - it will only need to check if a file is different.
If you’re using PM2 then you’ll need to re-implement the more stupid restart script I’m working on…

We have the updater binary for FreeBSD as well. Would you like to give it a try? You would be the first guinea pig. There is a good chance that the updater will not work and if so I don’t think we are going to fix that any time soon. However, if we are lucky it might work just fine. So we should at least try it.

2 posts were split to a new topic: Custom Updater for FreeBSD storagenode version

Then we can move it to a separate thread and make it a manual.

2 Likes