I initially setup my node on an ubuntu system running docker and portainer. The storage is a raid pool i setup in ubuntu and its formatted xfs. I am in the processes of trying to transition everything over to a TrueNAS Scale system. I have the rsync coping without any issues. my problem is transitioning the docker instance over to kubernetes. I have not found much information on doing this type of conversion so i’m thinking it might not be possible. The reason I want to transfer instead of start a new instance is that the node is currently full up. switching to TrueNAS will make expanding the data pool easier.
Do I actually need to copy the docker over or can i just create a new instance entirely that will take over the old identity of the docker?
I am in a similar situation and want to migrate my existing nodes to a new Truenas Scale System. I think it’s possible to run the storagenode-docker containers also directly on Truenas, but unfortunately you won’t be able to use the docker cli or portainer for that. You have to create the k3s container-templates via the TrueNas GUI as far as I know. I am right now testing this.
That is what I have seen from my research. I just have no clue. I’ve never messed with Kubernetes. Please let me know how things go for you. I’m still transitionin data. Been running for a week now. 8TB is a lot!
Your node = your identity + config/data. All the others (binary, containers, …) can be replaced, just be extra careful to use your right direction.
A common risk (as an example):
you copy the identity to the right directory
you copy the data to the wrong directory
Storagenode will start with the old identity, but pieces will be missing. You will see it as working, but you will be DQed very soon…
I am not familiar with TrueNAS, but based on the documentation, you should create two “datasets”.
You should do the same thing, but fill the datasets (identity + config/data) with your old data, instead of generating/signing identity.
You can also create a temporary Storagenode strictly following the instruction and investigate the directory structure of the datasets, just to learn how should it look like.
Thank you! I did see those instructions but the way that TrueNAS created their pod you are required to enter an authentication token so I stopped following those instructions. I will take a deeper look. Maybe I can manually create the pod.
@elek Thank you so much for your guidance! I did test and was able to see how things work now. I ended up having to manually create the Kubernetes pod. The one TrueNAS lists links to a very old version and also attempts to create the the node identity automatically. Making sure the path directorys are correct is very important as you mentioned. I just need to wait for everything coping now!
Note: Doesn’t really matter, I checked that old version and it has the auto-update inside the container, so it supposed to download the latest version during startup.
And also they updated the helm charts. But you need to upgrade to a minimum version first, then - to 2.x.x
However, it doesn’t affect your node, since it has an integrated auto updater, the node will be updated to the latest version automatically (this is unrelated to a chart version).
So how did it go? I am also thinking about moving to TrueNAS Scale from Ubuntu, but I didn’t know about such complexity and I have an ext4 node with 12Tb data, so copying will take a while…
I actually finally did the transition 3 days ago. So for it has been an amazing upgrade for me. I was starting to have a lot of metadata errors on the old server so I finally transitioned over. Apparently, there was a massive backlog of data expirations that didn’t get processed on the old system. within 24hour it has moved 2TB of data to trash. The new system with TrueNAS is also an older system. I went from a Ryzen 3 3100 to a FX6300. The new system is also using an LSI 9207-8i storage controller. If you get one I highly recommend getting a Noctua 40x20 fan for the heatsink. The data copy did take me several days at 7.5TiB. For me, TrueNAS Scale has worked out great!
I switched one of my QNAP boxes (TVS-h1688X) from the original OS (QuTS Hero) to TrueNAS Scale. Running 7 nodes so far with no issues. I guess it could easily handle all 16 drive bays used for storj nodes.