TrueNas Scale - upgrade to 24.10 (Kubernetes to Docker) Anyone done it yet?

Hi

Just wondering if anyone running as an app in Truenas Scale has migrated to 24.10 yet? It migrates away from kubernettes to docker - and seems a pretty big deal.

Not sure how easy a rollback would be if anything goes wrong, given its mitrating apps, so just wondering what precautions to take other than the usual truenas config backup, and maybe identity folder?

Thanks
CC

Huh. IXSystems continues making questionable profit driven decisions.

While I can’t answer your question directly, I can share some guidance

  • don’t update to new version of anything from iXSystems of untying, unless there is a bug fix affecting you directly. There will be some unexpected fallout. Their testing is nonexistent
  • It does not matter which process virtualization engine they use; storagenode is already monolithic executable and it does not care what architecture it runs on, let alone though what mechanism is it launched.
  • all data must be outside of the container of course, including config, identity, databases, blobs, etc. if you can delete the container and start a new one in the current setup — you will be able to run the same container with another orchestration solution.

But if the current setup works — keep running it. I don’t remember a single iXSystems release that didn’t bring a bunch of new bugs. Just not worth the turmoil.

I am already running 24.10. Upgrade went smooth, all my Truenas Apps migrated automatically and I am now running multiple storagenodes directly on Truenas via docker. So far everything works fine :slight_smile:

4 Likes

profit driven decisions

Correct, they are a privately held for-profit company.

I don’t remember a single iXSystems release that didn’t bring a bunch of new bugs. Just not worth the turmoil.

Seems pretty good so far, what issues have you seen?

You can follow their 24.10 Known Issues here: 24.10 (Electric Eel) Version Notes | TrueNAS Documentation Hub

And their 24.10 specific Jira cards here: Issue navigator - iXsystems TrueNAS Jira

3 Likes

I’m staying away from scale as far as possible, and never updating to it. My experience with them is exclusively with Core.

And I’m money bringing customer who votes with the wallet. Regardless of financial arrangement company must do what’s right. And if what’s right does not bring money — close the shop. I’m really baffled by this comment. Once company starts making exclusively profit driven decisions — it’s dead. The biggest of it was migrating to Linux in the first place. But that’s my opinion and offtopic anyway.

relevant post: What is the future of TrueNAS CORE? | Page 2 | TrueNAS Community

I just did the upgrade. No issues, all 8 nodes still working after reboot.

1 Like

I’m done that. Working perfectly and uses 2x less RAM. You also would get a new Cloud Backup feature with restic under the hood, which allows you to do backup better than a Cloud Sync (which is not actually a backup, unless you also enable versioning and TTL in your access grant/s3 credentials to cleanup older versions).
The migration was smooth and storagenode there is working fine as before. I didn’t test all applications though, so may say only for Cloud Sync and Storj application (storagenode).
It also allows to create a new bucket even if your Storj account has been created not via the promotional link. Of course you wouldn’t get their Starter Package (1 year) with the usual Storj account created not via their link, but you may use it for backups to a TrueNAS newly created bucket.

Your opinion of “what’s right” and someone else’s may, of course, be different…

1 Like

I’m not well versed in *nix systems enough to fully grasp why the move to Linux is such a big deal. Why is FreeBSD a superiors OS? Does it really matter? Is the software implementation of the NAS software on top of the OS not the right place to look?

Update complete and all running fine, no drama, just the way we like things.

Thanks for the fast helpful responses once again.

CC

2 Likes

I feel this is important information to share. Were you all running the TrueCharts or the integrated catalog version of the storj container?

No, I am running docker containers with my own scripts. Not the official storj app.

1 Like

It makes zero difference for the end user of the storage appliance what’s inside as long as it stable and performs.

Everything we are digging deeper into here, however, is behind that “as long”.

The company decided back in the days that the specific OS is a good basis to build appliance on top of. It happens to be FreeBSD. Great. All the reasons they picked FreeBSD then are still here today. Some features only recently, decades later, migrated to Linux — like eBPF dynamic tracing (dtrace) and boot environments (not sure about the latter though).

But no, “users want apps and containers” (nobody asked for them on a storage server in the enterprise, they run apps on application servers, not storage) so ixsystem neglected core and pour all efforts into new shiny Linux because “containers” and “better hardware support” whatever the heck that supposed to mean. It took them ages to bring performance to parity with core. Ages. If they spent a fraction of that on updating core — everyone would have been much happier.

They say everyone abandons FreeBSD, and it lags in releases, etc etc; we’ve been hearing last three decades, and it’s irrelevant. This is such a bullshit spineless reason. Think for yourself, what’s best for your company. You poured decades into stabilizing the appliance. Take it and run with it. Why start over again on an entirely new OS? “Containers!!!” Nobody wants them. Nobody. Want to make application server — make one. Leave storage alone. It’s stable. It requires minimum work to keep up with upstream changes. It needs to be boring and ultra stable. What are you thinking bringing hot mess of Linux into this?! Hardware support — no shit trash hardware is not supported. Don’t use trash hardware and don’t let your customer use one. Grow a pair. Customer want apps on storage box with Realtek Ethernet — well, educate them, and reject their fantasies. We sell reliable storage. It won’t run apps and won’t support Realtek. But we want to! Good bye, next!.

Trying to please everyone will end up pleasing cheap prosumers and turn away enterprises, where the money is. Such a dumb move.

Their enterprise customers still stick to Core because it’s stable and works. They still sell appliances with core because it’s stable and works. All technologies that existed in FreeBSD for ages are only recently getting ported to Linux. Being early adopter of anything in the storage space is reckless.

Disclaimer. My hatred towards Linux as a concept does not negate validity of these statements.

Of course. And I as a customer, can decide where to spend my money per my opinion.

I’m not questioning the status quo of Linux or windows, both of which took over the world for the wrong reasons, I’m questioning storage company jumping ship from working and stable solution to appease imaginary needs of their ignorant customers in the worst possible way — uprooting everything and starting over.

As @Ottetal noted above — users don’t and shouldn’t give two shits about what’s inside. They decided using linix now will pay off in the future for the massive expense building and stabilizing new system. Good luck with that. Seeing how their software is managed and how bugs are handled I don’t keep my hopes up. They have brilliant engineers and yet trivial bugs take three revisions to get resolved. Where is QA? And with that they decided uprooting everything is the right choice.

They blame complexity and hope a lot of it will be handled for them upstream, and will reduce their workload — which is not how anything works. Linux rapidly evolves. It prioritizes features, yes, even Debian is too fast for storage.

Anyway, this is wrong forum for that, and the linked thread above has more information. They are responsive and transparent on the forum, and I appreciate that. I disagree with their decision to move to Linux, and will not be using them in the future, because I don’t trust them to be able to stabilize Linux based solution sufficiently in the observable future. Tomorrow they will jump to something else (yes, they said that, it’s from the horses mouth) and I don’t have time nor desire to go retest everything every time they are exited about every new fad, nor redo existing scripting and harnesses to monitor the appliances.

See what I am talking about? Fresh new release and they only now managed to fix memory overuse. Come on. It was, is, and will be for foreseeable future an unstable hot garbage.

Have they deprecated Core, then?
I thought they were supporting and developing both products?

Core is effectively dead. They are leaving is to “die off on a vine” as someone said on their forums. Current stable release is based on FreeBSD 13.1. It’s already EOL. And yet, to their enterprise customers they recommend the one based on 13.0, that has been EOL since 2022.

They made last release based on 13.3 for the community, with a few bug fixes, but they don’t offer it as an update automatically, you have to manually install it.
And yet, 13.3 FreeBSD is already old news, it’s already legacy release and is will be EOL soon. Current production release is 14.1. 15 is in the works. 13.4 is also in the works, but iXSystems does not have any plans to update to that ether. It’s obviously, and clearly dead to them. They starting slowly suffocating it long time ago – the whole inconsistent marketing on broken plugins story is glaring example.

Once freeBSD release is EOL – the ports stop being updated, so all your jailed software slowly dies. Base OS also stops receiving security updates. Not a big deal for the storage appliance, but upgrading also bring in the improvements in the OS.

2 Likes

I used the TrueChart, then migrated to this wonderful version (sorry, but my Hypervisor said me “thank you!”, because now it uses only half of the RAM than before), everything working smoothly.
I do not know, why they decided to use a Kubernetes (even a lightweight version like k3s) to run applications? If they likes the concept of pods, they may just use Podman instead.

For me Kubernetes and k3s simplification is needed only when you have more than a one node. Otherwise - it’s a development toolset otherwise not needed for anyone consumer.

Even if I agree with you, but containers makes the live of developers a little bit easier - they do not need to implement a different binary for each existing platform. It’s also good for distribution (just take a look, how many packaging systems we have now - it’s insane). It’s lighter than any similar approaches like Snap and CO.

Yeap. Unfortunately - our reality.

I started out with the integrated catalogue app supported by IX systems - and deliberatly missed the Truecharts stuff.

CC

To clarify, I’m not against containers, they are useful for all the reasons you listed. I’m against containers on the storage appliance. It’s a “nice to have for home users” feature, to run some lightweight automation on already running NAS hardware, but hardly a reason to uproot entire ecosystem. Enterprise customers don’t run apps on their storage devices because every bit of ram can be better used for caching. Taking it away to run apps is dumb counterproductive.

but it would suggest only Charts if your TrueNAS version is below 24.10, the normal docker apps would be available only in 24.10 as far as I understand.