Hi,
I’m thinking about running Storj Node, but I have a few questions for you, because I’m not sure if I can use this configuration.
I have a Proxmox server and a Synology NAS.
I want to install Storj Node on one VM.
Then on Synology I want to create an 8TB iSCSI share and connect it to the VM with Storj Node.
I have a direct connection (10Gbps) from Proxmox server to Synology.
Is this configuration allowed?
While using network attached storage is not be the best option, given that you may experience performance issues, there is nothing that prohibits it.
There are documented cases of some network protocols causing issues with the DB (e.g. NFS), but iSCSI should be fine.
I would recommend to try it out yourself and see how well it performs.
I run a ISCSI node since 5 years now in a similar setup without problems.
Make sure to mount the LUN directly to the VM instead to Proxmox itself to avoid overhead.
Hello @osalj,
Welcome to the forum!
Why do not run a node directly on Synology?
Hello @osalj
When I began hosting, I did ISCSI from my Synology to a VMware VMhost where I mounted a 20TB ISCSI drive, and then created vDEVs for my windows VMs.
It works flawlessly, but it has overhead and is not very efficient.
If I were you, I would run a node directly on the Synology as @Alexey suggests.
I’d follow this tutorial, if I were you 
Or this:
https://forum.storj.io/t/my-docker-run-commands-for-multinodes-on-synology-nas/22034
Also take a look at this:
https://forum.storj.io/t/synology-memory-upgrade-guide/20743
More RAM is better for Storj nodes.
My Synology has only 8GB RAM and I try to use it only as storage, I keep all dockers and virtual machines on servers with Proxmox.
I assigned an iSCSI disk to one virtual machine (Storj) and the performance is identical when the disk is directly in the computer.
Synology has 4x 1Gbps. One port is configured for management, one for Plex. Also has 2x SFP+, one port is configured for Storj only, and the other for backups and data.
For now I am happy with this configuration.
Okay, I have one node with a 2TB iSCSi disk.
This node is currently a test node and will definitely be online all the time.
I found in the garage a 1U case with a motherboard with a Celeron N3060 CPU and 8GB RAM.
The motherboard has two SATA3 connectors and a few USB3 ports.
I wonder if the N3060 will be suitable for a Storj node?
I would connect two 4TB drives to the N3060 for storage and a 128GB NVMe system drive (Debian 12) in an adapter connected to USB3.
I could run two nodes in docker on this, each with a 4TB drive.
What do you think about this?
The CPU is not a problem. Storj needs RAM, SATA, wired internet connection with fix public IP and CMR drives.
And from all of these, the only crucial thing is the wired internet and a fix IP, or a DDNS service.
I ran 2 nodes on 1GB machine, so… 8 is fine.
Try to avoid USB connections. They work, many SNO use them, but usualy gets you in trouble, because the addapters are unreliable.
SMR drives are verry slow and give you timeout errors. So stay with CMR drives, formatted to the native format (4k usualy), ext4 -noatime, linux.
There no need for fixed IP (in fact, you better off without it, to support failover), nor public IP: most CGNAT providers will forward a port to you on-request.
The only requirements is system configuration that can handle random access to hundreds of millions of small files at reasonable rate.
Thanks for your answer!
Internet to this server will be via cable, I do not have a fixed IP, but I will use DDNS.
Two disks are CMR 4TB.
Unfortunately, this motherboard has only two SATA connectors and I am forced to use an NVMe to USB adapter for the system.
Thanks
There are cheap PCIE SATA controllers on eBay. Just avoid port multipliers. I would not use USB and storage together in any sentence.
Better yet look for used HBA adapters (like LXI/broadcom) from decommissioned servers, like this one: Supermicro AOC-S3008L-L8E SAS3 12Gbps 8-Port Internal PCI-e 3.0 HBA Controller | eBay
My motherboard only has one PCIE x1 port
I have some different controllers, but they require a PCIE x4 or x8 port
You can then get cheap used motherboard with more ports for not much more, sometimes together with processor and heatsink.
I would not trust any data to anything that has usb in the data path, and assuming you are not doing this just to run nodes for storj, but you actually need a home server or for entertainment, getting a used server motherboard would be the best course of action. You shall be able to get the whole thing for under $40. Look at supermicro, dell, and asrock rack. With server motherboards you get much better reliability and indispensable features like IPMI — once you have an option of not needing to haul display, keyboard, and boot disks to the server in the closet every time you want to mess with the server, there is no going back.
Basically, any money you will “save” using consumer usb disk adapters you will spend many times over later with your time troubleshooting weird issues.
I know USB drives are not the best idea, but that’s all I have at the moment.
The 1U case I have is only for mini-itx boards.
I found a few motherboards on eBay UK, but they cost £60 or more.
The ones I found have 4x SATA ports, some even have a slot for an NVMe drive.
A motherboard with four SATA ports and a slot for an NVMe drive would be ideal, but I’m currently trying to use what I have.
I’ve now installed the system on the NVMe drive that’s in the adapter.
I’ll test this solution a bit and see how it works.
I know about IPMI, all my servers that I have have this feature.
It makes managing servers much easier.
Now I’m wondering about one thing.
Maybe instead of installing the system on the NVMe drive that’s in the adapter, it’s better to allocate 32GB to one 4TB drive and install the system on it?
Does this configuration make sense? How will this work if the same disk is a system disk and I storage for STORJ files on it?
I know it’s a platter disk, but it would probably work for a lightly loaded system?
I have another idea.
I have 3x servers with Proxmox. Each of them has 4 or 8 bays for 3.5" drives. Currently all the bays are occupied, but I could check each server and after cleaning up the disks from VMs I could definitely remove two disks from one of the servers to put 4TB 3.5" disks.
If you only have one IP address then you don’t have to go crazy. Set up one Storj node… let it fill up… then decide what to do next (or start with 2x4TB in your N3060). More nodes sharing the same IP won’t fill any faster, so no need to start reconfiguring your proxmox systems.
At the current rates it may take you a year to fill 4TB: so you have time.
Currently I have few static public IP addresses from two different subnets, plus my home internet has a dynamic public IP.
Currently, one node is running as a virtual machine with a 2TB CMR drive.
After the first 40 hours, I have used about 75GB, so until the 2TB disk fills up I have time to decide.
It shall be fine. You can also install the system on a cheap usb thumb drive too — depending on the OS it may be easy to configure writes (logs, traces, etc) to go to main array, and then even the cheapest of the thumb drives will live forever. And if it dies anyway — no problem: once it fails to boot image the system onto the new one and continue.
Il trying to say that boot media is used read only and only once per boot, allocating “good” ports for it seems wasteful.