Storj data directory on a virtual drive (VHDX)?

Hello all,

been running a Windows node for a few months. A small one, like a hobby thing. So far so good. However, neither the computer or the physical drive are dedicated to Storj. I have tweaked my install some, and now I am in the process of moving the data directory.

Not reinstalling the node, just tweaking config.yaml in order to move the actual payload data to a more convenient disk - namely a virtual disk.

Even my small 650GB node seems to have almost 4 million (!) files, and (robo)copying it is quite a chore for my setup, just because there are sooo many files.

So I was planning to put the data directory on a VHDX in case I need to relocate the data again. Seems much simpler - and a lot faster - to just shut down Storj service, detach the VHDX, move one large .vhdx file to a different physical drive, reattach the .vhdx and restart Storj service.
Planning to try ReFS for Storj on the virtual drive.

Thoughts?

Hello @JuhaML,
Welcome to the forum!

Too complicated in my opinion, starting from the fact, that VHDX doesn’t attach automatically after restart and ending with another virtualization layer. It could work slower than a direct access, too complicate extending and shrinking, etc., however it could work, but I wouldn’t recommend to do so.
Regarding copying - you may use a clone partition programs or even dd or GParted from a Linux bootable stick.
It also possible to move data from one disk to another if you use Windows Storage Spaces, but it requires that disk has been prepared for that long before, there is no way to preserve data during adding to Storage Spaces.
Regarding ReFS, see Topics tagged refs

So, in my opinion, the simpler is better, i.e. data and identity are stored directly on the disk formatted to NTFS.

1 Like

A follow-up to this topic.
I am indeed (still) using Windows DevDrive/ReFS virtual disk for storing node data. Agreed, it isn’t ideal, but I wasn’t able to shrink my existing partition sufficiently to give Storj data a dedicated partition, so I am stuck with the virtual drive for a while.

ReFS was something of a misstep, I believe vanilla VHDX/NTFS would have been a better choice after all, for a small 1TB node.

As for the added manual steps, I don’t really mind. My setup is weird indeed to begin with, I tweaked the service(s) to run on a regular account and I update&restart my node manually, whenever I need to update&restart my Windows desktop.

Come to think of it, I don’t quite understand why the node is installed as a service, or why it has to be installed at all. I would appreciate the option to just tweak config.yaml and run the executables directly, as a non-admin user. Oh well, I guess the following accomplishes that, more or less:

I still keep identity at the default location, but I’ve cleaned up the Storj installation directory with config.yaml tweaks.

AFAICT, the node works just fine this way, too. :scream:

For this size I think it’s okay but when the node grows bigger then my recommendation is to switch to Linux and put node data on a separate real HDD, use EXT4 or XFS. It’ll save you from alot of headaches later.

1 Like

I was thinking of retiring an older rig and replacing a couple of drives with bigger ones some time after new year. Actually those bits of hardware still pack some oomph, so I think I could put together a more decent Storj node by repurposing said bits. Forearmed with some Storj experience, it would not be quite as kludge-y. Linux-based, and about 1.5TB.

Been running this node since mid-July, so according to current Graceful Exit Guide I even might be able to shut it down sooner than I expected and start fresh - or maybe try to migrate neatly. I suppose I will be asking further questions once I am ready to up my Storj game a bit. Thx for the suggestions thus far.

For the generic user it’s a simplest option, the node will start automatically on boot, it will be automatically updated in time.

You actually can.

"c:\Program Files\Storj\Storage Node\storagenode.exe" run --config-dir "c:\Program Files\Storj\Storage Node\\" --identity-dir "c:\Users\Storj\AppData\roaming\Storj\Identity\storagenode\\"

or, in your case

"C:\Users\StorJ\StorjApp\storagenode.exe" run --config-dir "C:\Users\StorJ\StorjApp\\" --identity-dir "c:\Users\Storj\AppData\roaming\Storj\Identity\storagenode\\"

But in this case the designed automatic updater cannot work properly and you need to update the node manually, and this removes all safety checks (our updater updates nodes in waves to do not shutdown the whole network in the same time).

1 Like

Nice!

I’ve noticed that the updater service does not work if I run it as a standard Windows user. I guess the update script fails because a standard user does not have the privilege(s) required to restart services?

Fortunately I have remained somewhat compliant with safety checks in the sense that I start/stop the updater service periodically, then check the updater log. It gives me a heads-up when an update is about to arrive, and I update manually once the log shows that an update was actually attempted.

BTW, would it actually be necessary to specify --identity-dir in the command prompt version? The service version only seems to specify --config-dir, and there are identity.* entries in config.yaml.

# path to the certificate chain for this identity
identity.cert-path: C:\Users\StorJ\AppData\Roaming\Storj\Identity\storagenode\identity.cert

# path to the private key for this identity
identity.key-path: C:\Users\StorJ\AppData\Roaming\Storj\Identity\storagenode\identity.key

Anyway, been a nice little learning experience, me getting a bit more CLI-oriented with Storj. Hopefully all this makes it a bit easier to switch to a Linux-based node later on. :sweat_smile:

Speaking of which, can I keep accumulating $ to my current identity if I do a Graceful Exit first, spend some time offline tinkering with new hardware, then get back to Storj once I am confident enough that the new stuff is good enough for a longer haul? After just 6 months, I certainly haven’t accumulated enough to just grab the cash and start completely fresh.

I think so. It’s not a script, it’s a fullfil binary. But you are right - it uses system privileges to restart itself and storagenode service.

As far as I know - yes. Even if you provided a correct path to the config.yaml. I do not know - why.

I do not think, that’s a great idea, because the grow of the new node will take time. A lot.
I would just migrate, if I was in your shoes.