I intend to build a miniPC only for runing 4-6 nodes on 4-6 HDDs.
I’m not a linux user, but I can use the cli and follow guides, as I do with my current Docker/synology nodes. I intend to use only ext4 fs and Docker nodes.
What do you recommend to go with? Ubuntu or TrusNAS scale?
Does TrueNAS scale supports running multiple Docker nodes? Does it supports Wireguard client? Does it have any benefit over Ubuntu, if I only run Storj nodes on it?
What else I should be aware of? Is TCP fastopen working on TrueNAS? I don’t want to repeat the Synology expirience.
If you are not going to be using ZFS — then there is no point in true nas scale (leaving it’s readiness and stability compared to core aside for now).
I would go with fedora/rhel/Oracle Linux, definitely not Ubuntu.
I would further recommend podman in place of docker.
And even further — run nodes directly. On a resource constrained system there is no reason to waste them for OS virtualization.
But on the other hand, I would consider going with TrueNAS Core, single ZFS pool, with special devices for metadata using fast SSDs, and running storagenodes in jails.
That is if you want ultimate performance, scalability, and what’s more important, stability. But you need to have at least 16GB of ram, and the more - the better. (fastopen is not yet enabled for FreeBSD nodes, but I don’t think it’s such a big issue; it’s a small trade off for stability and resilience of FreeBSD and ZFS; ultimately I hope storj will enable it for this os as well Please enable TCP fastopen on your storage nodes - #8 by jtolio)
Overkill.
use plain debian + docker.
5 Likes
yes
web ui, but more resource consuming because of this shell.
likely yes, but hard to achieve because of kubernetes under the hood (see Using sysctls in a Kubernetes Cluster | Kubernetes).
you can mount it, but not format to ext4. The default is zfs without a choice, as far as I know.
What’s the problem with it?
Actually, there is. FreeBSD has far less hardware support compared to Linux based systems. You get access to a far wider range of hardware by using TrueNAS Scale over TrueNAS Core.
That said the TrueNAS team regard their system as an appliance which is why they deny access to apt by default. There is an extensive discussion n their forum about that.
I notice CentOS is not in your list?
Personally I would not touch RHEL or Oracle Linux. I would have used Centos before the changes to Stream. For physical machines these days I prefer Linux Mint.
My heritage is *BSD based (I ran all three of the major BSD’s for years) but tend to run Linux more now. Of the BSD’s I’d still prefer NetBSD.
1 Like
Not necessary a problem, but for a server os I would pick something stable and less bloated. Your don’t need all that addon stuff canonical crams to it.
You only need it to support your hardware. I find stuff that it does not support I would not want to use anyway — such as realatek network adapters and questionable sata controllers. Essentially, this serves as a filter forcing you to stable hardware choices.
Missed that. Of course CentOS too.
Edit. Actually, centos is discontinued. I did not know, I don’t keep track of that. So, it’s accidentally correctly not on the list 
Understandable. My perspective: both Linux and free/netBSD are horrific as desktop OSes from usability perspective. My use of them therefore is limited to server headless systems, where stability is important. In those environments you want few features but a lot of polish and testing. FreeBSD and enterprise Linux trains fit the bill. Desktop oriented linuxes dont. Of of these I prefer BSD systems, partly because of my macOS background, partly due to excellent documentation, debugability (dtrace is my everything), and mostly os design choices and features. I find FreeBSD superior to anything in the Linux world that bafflingly goes in entirely different direction. It’s a personal opinion.
Ultimately, node can be made to run on anything. Even Alpine Linux. Why not? Being familiar with the os is another major factor in choosing the platform — you want to minimize time spent maintaining.
But if you start from scracth I don’t see the reason not to pick something stable, server oriented, and ideally, with a commercial company behind.
This is inherently opinionated topic, we shall be very careful not to derail to a flame war
So many linux flavors… no wonder why so many go with windows 
. There the choices are very simple; go with the newest version.
Many go with windows due to excellent Microsoft efforts to cram it to everyone’s throat early on, and providing excellent long term enterprise support, carrying over backwards compatibility for obscenely long time.
I’ve been a user and the kernel and user space developer on windows systems for 15+ years in the past, including on a little known platforms: Windows Phone 7 and 8 (love these things, and of course MS killed it, I guess it was too good) but today I won’t touch windows with 10 ft stolen stick. Stability is garbage, usability is nonexistent, everything is wrong there, (ads in start menu of my paid professional editition? Candy crush?! Wtf?!) I have no idea how people tolerate it.
I guess for the same reason — inertia and “known issues are better than unknown ones”.
For most desktop users of course, windows is fine. But so is ChromeOS. But if you want to do anything nontrivial — oh holy crap what a heap of haphazardly put garbage it is… I gave up long ago. I still have a PC for a few games I play occasionally that don’t work on a Mac but that is rapidly changing and I cant wait to get rid of it for good. Essentially it’s a game console now. I don’t interact with explorer.
The thought of using it as a server though — no thank you :). That thing corrupts its own filesystem on every reboot, turns on features I turn off, registry is a mess (who thought it was a good idea?), and resource usage is like there is no tomorrow. Still does not support multiple connections with different credentials to the same SMB server. And those folks invented SMB. I don’t get it.
But credit where credit is due — iSCSI initiator on windows is top notch. But that’s the only compliment.
… and it’s free. If you want to change your wallpaper you can find license keys around 5$. So pretty much free.
I agree that at least in the last year they realy try hard to destroy the wide adoption, and annoy every win 10 and 11 user. How the hack can you restart my computer with open programs and work, in the midle of the night, just to install the last update that brings more bugs and problems that solves? How? Why?!?
Since 2018 when I started to be interested in crypto and mining, I discovered Linux and I started to love it more and more, but since my daily work involves only windows, the linux ecosystem remains just a hobby and a way to learn some stuff and make some pennies.
1 Like
That would have been ok if it supported application state restore and ecosystem adopted the practices. But seeing how Microsoft violates their own application developers guide — we get what we get.
But I agree, the audacity of thinking that some update is more important than users data is just from another world. On the other hand — how else will you ensure users actually update and not post phone forever? Application state restore, that’s how. It’s as if various Microsoft department don’t talk to each other.
On my previous job I got so fed up by windows and Lenovo Thinkpad trash in particular (bastards shittified a perfectly great IBM business) that I had the department buy me a MacBook. My productivity soared. I stopped wasting time fighting my own computer and freed more time for actual productive work. Some low level windows-only debuggers and tools — like lauterbach Trace32 and WinDBG — I ran in a VM.
Well, if it was free then sure, show me ads. But mine isn’t free. It’s a win8 pro license, later upgraded to 10. I paid for it. With money. And yet, it’s now a cesspool. People write apps to suppress bloatware on paid editions of windows. Welcome to the future where nothing makes sense.
Centos has evolved into Centos Stream now. It hits a different purpose.
This I disagree with. There is a lot of Enterprise hardware out there FreeBSD just doesn’t work with. I’m not talking Realtek junk. !0Gbit Nic’s are a good example. I had a hard time getting 10Bit working under TrueNAS Core - working fine under Scale. There was also a HP branded Broadcomm NIC I had that refused to work with FreeBSD but has worked fine under Linux systems. Given it was from Proliant hardware one can assume it worked with the OS’es that HP support as well.
Well, I am still using pfsense so it’s not like FreeBSD has gone from the LAN.
I come from the days of Unix workstations and CDE so this just doesn’t impact me as much. I find a Linux workstation to be fine unless I have to do stuff in M365 and then I start to need a Windows Desktop.
and in the latest batch of craziness they decided to rename Azure AD for some unknown reason.
I am fortunate at my current job I get to do a mix if things including Linux, Proxmox and others. (Finally our obsolete Esxi is gone - yay!) But the MAC users drive me nuts the most. Many are simply incapable of following instructions.
2 Likes
Mac users are spoiled by the ecosystem where everything is supposed to work intuitively and not require instructions in the first place. And it’s a good thing.
I’m one of such users myself.
Use these users’s complaints to drive improvements to the system. Eliminate need for following instructions. Users should not need to bend to conform to the software or processes, it’s the other way around. I’m sure these users are great in their roles, and they don’t need to have to become an IT pro to get the job done.
In other word if they are masters in adobe premier but can’t follow 13 step instruction on how to connect to the storage server is this really their fault?
Provision their system for automatic connection so they don’t need to worry or know about these things.
So I’m on this one I’m with those users.
Fair enough. However for home users/hobbyists, that mostly source hardware on secondary market, the lack of support in FreeBSD still correlated pretty well with crappiness.