Migrating from Windows Node to TrueNAS node, how to?

Node Operators
, , , ,
1

Hi All,
I currently run a Windows storj Node with app. 14 TB data.
In my quest to streamline my setup, I would like to migrate this node to one of my TrueNAS Scale servers.
I have read the guide on how to setup a new storj node in TrueNAS, but I do not know how to migrate the data from my Windows node to a TrueNAS storj Node.

But, how do I do that?

My take on it is to do a filesync from the Windows Node to the TrueNAS dataset, but how exactly should the dataset/folder structure looks in TrueNAS for the data to be recognized when I start the node on TrueNAS?

I would really appreciate your help on this.

2

There are many was to do it.

I did it like this: How to move StorJ from Windows to Linux - YouTube

3

Thanks a lot.
The video is for migrating to a docker not to TrueNAS, so I am still unsure about how exactly to structure the ZFS dataset/folders in TrueNAS?

4

Depending on which version of TrueNAS scale is just Debian with bits on top. You could run it in docker here. Just present a folder for it to live it, and you should be golden.

Are you planning on running it on single disk, or as part of your entire infrastructure?

5

TrueNAS has a storj app now, which I intend to use, instead of making either a docker for it :slight_smile:

It will run on its own single disk, the same disk that is in my Windows node at the moment, so I will:

  1. Copy all the node data from the Windows NTFS disk to a temporary dataset.
  2. Move the disk to the TrueNA server and create a new Pool on the disk.
  3. Move the data from the temporary dataset to the newly created pool.
  4. And, then hopefully get the node up and running with the data intact.
6

“Apps” on scale are containers. And there is no good reason to introduce yet another layer of abstraction.

Why not keep node in that temporary dataset on the main pool?

1 Like
7

Because I do not want to use my main pool for storj, as it serves a very different purpose. Also my main pool will soon not have enough space for the node. Thirdly the storj node will live on my second TrueNAS server, which does not contain my main pool

8

Hello @HyggeJJ,
Welcome back!

As many said already, the TrueNAS Storj app is a container app, but with some UI sugar, so it uses docker under the hood (if your version is newer than 24.04, before that they used a k3s kubernetes single node cluster, which worked with docker then containerd, so still containers but control plan was different).

Basically everything described in migration guides like Migrating from Windows GUI installation to Docker CLI - Storj Docs is still applied, but with some nuances, if you want to use only UI.

First of all I would suggest to setup a new node on TrueNAS, it would create all needed structures and the configuration. After that you can stop this node and replace all data with your Windows node’s data. You just need to place the identity (6 files) to the proper dataset/folder and its data to the proper dataset/folder.

For example, my setup (25.04) has this structure:

image
image1101Ă—633 27.7 KB

The config of this node:

image
image738Ă—883 34.2 KB

so, in my case the identity files are placed to /mnt/data/storagenode3/identity, the node’s data is placed to /mnt/data/storagenode3/config and the file tree looks like this:

image
image1319Ă—623 25.2 KB

image
image1306Ă—690 25.3 KB

In that case all data from the Windows node should be in subfolder /mnt/data/storagenode3/config/storage:

image
image1278Ă—829 32.3 KB

Of course you may also do not create a new Storj node on TrueNAS before migration, because it’s a time consuming (it will generate a new identity), and just prepare the structure as I showed above, then setup a Storj node and configure correct paths in the storage configuration, of course you need to specify the wallet, the external address, email, etc.
Just make sure that the original node is already stopped and data is migrated before you start it on TrueNAS.

Perhaps you would need to update permissions on your datasets:

image
image1575Ă—727 50.8 KB

image
image1608Ă—713 51.1 KB

2 Likes
9

If you are familiar with yaml files you can also run storj nodes in TrueNAS Scale as custom app via yaml. I like this much more than the dedicated app.

1 Like
10

@Alexey , thanks a lot for the very elaborate answer, it is much appreciated.
I will follow your advice of setting up the new node on TrueNAS, and then migrate all the data into the new structure before starting the node.

1 Like
11

For how long can my node be offline, without getting disqualified?

Migrating my storj node data takes MUCH longer than anticipated, app. 35 million files totalling 8TB data. Using robocopy to mirror them from one NTFS disk to another NTFS disk has so far taken 3 days, and is only 50% done. After this I will need to Rsync them from the NTFS disk to a ZFS disk, and only then can I get the node up and running.
So my node will be down for 2 weeks, or something like that :frowning:

12

So many questions….

  1. Why are you offlining your node during transfer? You only offline it for the last final sync. Keep it running otherwise.
  2. I thought you are migrating to Linux? Why do you copy to another NTFS disk? And if you were to copy to another NTFS disk — copy entire partition, not files one by one.

Follow the guide How do I migrate my node to a new device? - Storj Docs and migrate data over network, your downtime will be a couple of minutes and how long transfer takes would be irrelevant.

1 Like
13

tl;dr: A node will be disqualified if it has been offline for more than 30 days or when the online score becomes 0, whichever comes first.

But I do not understand too, why you don’t sync over the network directly to your NAS while the node is running in the source?

14

True, my process for this migration has been flawed.

I originally tried to use robocopy from my storj node (SFF Lenovo) to a TrueNAS ZFS dataset via a SMB share. It was very slow so I gave up on that method, and pursued other solutions.

Copying the entire partition takes days, since the source disk is 16TB and the destination is a 12TB disk. Just resizing the 16TB partition to 12TB is estimated to take 25 hours (plus another 24 hours or so to copy the resized partition to the other disk), so I also gave up on that idea.

My Windows node is back online, and I am now using robocopy from the 16TB source to the 12TB destination, while the node is running on the 12TB disk. When the copy is done, I will:

  1. Switch the Windows node to the 12TB disk.
  2. Mount the 16TB disk in my TrueNAS backup server.
  3. Robocopy via SMB from the Windows node to the TrueNAS ZFS 16TB dataset.
  4. Start the storj node in TrueNAS.

It will take a loooong time but eventually I will get there, and downtime should be minimal.

15

I tried that, but robocopy to a SMB share was very slow, so I tried to find a different and faster way of doing it.
But, my original node is back online and as described to arrogantrabbit I will sync via the network, and just let it run for however long it needs :slight_smile:

1 Like
16

I assume you meant 16TB disk here.

I’m still not sure if the 4-step process is necessary here.

Why not:

  1. Robocopy/rsync from your running windows node to your NAS . (Few days/weeks — does not matter. There is no hurry.)
  2. Shut down node on windows server and run last robocopy/raync wirh /mir or -delete (few minutes)
  3. Start node on NAS.

Use rsync. It will be faster. However, SMB shall be not much slower, if not faster — what is the bottleneck? Does your zfs server have enough ram and special device? It shall be copying just as fast as to local disk.
The throughput is not an issue, seek latency is, and your NAS is better equipped to hide it than naked single drive. You can also try NFS if you don’t want to setup rsync daemon.

By introducing the intermediary disk you are complicating setup, doubling transfer time, doubling sync passes, and risking data loss — reading NTFS disk on Linux is not advisable.

1 Like
17

Be faster if you mount 12tb (as ZFS) in Truenas Server
Copy 16Tb node to 12Tb via rsync/nfs/SMB
If U want to copy 12tb back to 16tb use “ZFS send / recv” to copy dataset.

Using ZFS on a single drive is far from ideal.

18

I want the node data to be on the 16TB disk in my backup-TrueNAS server, I do not see how your 3-step process can accomplish that?

The particular TrueNAS server is my backup server, not my main server. It only has 16GB of RAM, and no special dev, but it also does not run a lot of services (mainly a nightly snapshot of my main TrueNAS server). Throughput is not a problem, when I make snapshots it easily saturates the 1GbE NIC (which will be updated to 2.5GbE NIC’s), so I do not know why the storj data robocopy via SMB was slow. So I also wonder what the bottleneck was, when transferring many small files via SMB.

19

Seek latency is responsible for both.

Where is the node now? On the same 16TB disk that you want to end up in your TrueNAS server?

20

Yes, the Windows node data are on the same 16TB disk that I want to use for the node in the TrueNAS server, precisely :slight_smile:
That is why I am copying all the node data to the 12TB disk now, such that I can shift the Windows node to the 12TB disk and put the 16TB disk into the TrueNAS server (and copy all the data again).