My databases for node are on storage folder on hdd. Is it important for node performance moving the databases on ssd? Or it matters only for my stats…
Only if your node’s performance is affected by slow I/O (example symptom: your node uses more than 1GB of RAM). If not, don’t bother.
2 Likes
I would add that you should probably pre-emptively move them if you use an SMR HDD. Those will slow down eventually. Even if they seem fast now.
5 Likes
i would like to add, over time, esp. with ntfs and no defrag, the dbs will be fragmented and slow down the loading of the dashboard.
1 Like
Ok, I really don’t understand all this. I moved my dbs to SSDs a while ago based on what everyone has said about performance, however in doing so I realized there nothing. I notice absolutely no difference. I mean, it only took half a second to copy the dbs. My average is 12MB/node. It’s literally nothing. If you have slow drives or high I/O / ram usage, it isn’t because of the dbs.
Edit: I also played around with larger record sizes which actually crippled I/O (TrueNAS Core, ZFS, brand new 18TB CMRs). Currently running 256K. Don’t have all the exact numbers but 1M record size brought average rsync rates down to between 6-15 MB/s with a single node running on it vs ~50-60 MB/s doing the same with 128K. Just throwing that out there.
not all nodes profit equaly from moving dbs.
i noticed the drive “runs smoother/more quiet” while under high load(defrag/filewalk)
dashboard much more responsive.
little filled,fast node: no diff.
slow drive, nearly filed profits more.
i did the experiment and moved the dbs to an Samsung usb stick. 5y warranty 16€. reformatet to NTFS/4k good and not the “cheapest” solution.
Samsung USB-Stick Typ-A BAR Plus (MUF-128BE4/APC), 128 GB, 400 MB/s 60 MB/s
waiting how long it will last, just to trigger @arrogantrabbit .
its an minipc with no other free connection ports than usb3.
1 Like
I’ve seen at a friend a small “silent” home server with 3×"decent flash drive" in RAID1. He had to replace a drive every two-three months. I’m curious about your experiment!
its about the dbs, not the node data.
node data is on wd elements 12tb external drive. but it turned out: its slow.
so its only the dbs who will be recreated in case of “5y warranty” not enough.
It’s likely (to avoid saying most definitely) write amplification because your friend did not override sector size when assembling the array. (Also, raid1 of three drives for a home server ?! Something does not add up here. It’s definitely not a full story).
It was few years ago, I don’t know all the details. But he’s a hardened sysadmin, I’d find it likely that he did set up things correctly.
And yes, it was RAID1 exactly because of risk of drive failure.
But three drives? However, these are cheap ones…
But I don’t often come across such a configuration. I guess they uses some checksum FS?
I just thought of such config myself to be honest, because my “server” was in the sleeping room (it was also a living room too… we were lived in a one-big-space apartment that time…)
Likely ext4 on mdraid, knowing him, he’s the type of an old bearded unix admin. Yeah, this was kind of an experiment. He did have access to a supply of flash drives from the company he worked for, he had an old terminal-style minipc, so it was all free. He hosted some small services like gitolite.
1 Like
In my case it was a media/game server, and I started my first node (v2) there, so I’ll prefer to run it 24/7. Also some side projects… so, it should be online. And when I found Storj (the far away 2017…), I was just very enthusiastic to run it, especially when I searched exactly for something like this. This was a such excitement! I already used a bittorent sync on that time, but it required to use an own servers, and I wanted something like that, but not on my servers.
Storj become is what I needed!
Now it’s a serious trustworthy project, not some startup with unknown future, so I still excited to be part of that!
5 Likes
I just want to speed FW run, because I have 14 TB nodes almost full and the 20TB ones are filling.
I bought the same model as @daki82 :
Samsung USB Flash Drive BAR Plus 3.1 128GB (MUF-256BE4/APC).
I will test it in Synology DS220+. The specs that I managed to scrap from Internet are:
-
Official read speed 400MB/s - 128 and 256GB models.
-
First tester - 256GB model:
- Seq read 283.48MB/s, write 61.84MB/s
- 2MB random read 269.31MB/s, write 56.01MB/s
- 4K random read 3009 IOPS, write 1884 IOPS
- Second tester - 2GB data, 256GB model:
- Seq Q32T1 - read 273.6MB/s, write 59.47MB/s
- 4KiB Q8T8 - read 13.57MB/s, write 12.56MB/s
- 4KiB Q32T1 - read 13.30MB/s, write 13.37MB/s
- 4KiB Q1T1 - read 12.83MB/s, write 12.65MB/s
From those articles I get that they tested first generation of 300MB/s, but now these are 400MB/s models. I don’t expect too much difference between 128 and 256GB models. The speeds are tire 1 for 32-64GB, and tier 2 for 128-256GB.
I don’t exactely know what speeds are applicable to Storj use case for databases; the 4K IOPS maybe?
I settled for Samsung Bar Plus 128GB because I checked the other offers from the major brands and for this form factor, this model came on top. There are ofcourse more perfomant models from many vendors, but they are too big for my needs - front USB on NAS.
I found something realy cool:
Corsair Flash Voyager GTX USB 3.1 128GB (CMFVYGTX3C-128GB). It is built with 3D NAND SSD cels.
- 440MB/s read and 440MB/s write speeds;
- Max Random 4K QD32 Read up to 33K IOPS;
- Max Random 4K QD32 Write up to 40K IOPS.
What could we put on it? Databases? Logs? Orders? Matadata cache?
If it’s a Synology NAS, then perhaps use their technology to add a SSD as a cache device.
If you would have a free space after that, you likely also may move databases there. Logs are insignificant load for honestly, even for BTRFS, since it’s append-only mode.
It’s only 2 bay NAS, both filed with HDDs. No extra SATA or M.2 slot available. Regarding that specific flash drive, connected to USB, even though it’s built on SSD tech, I don’t believe Synology will see it like a SSD through USB. I believe it will see it as USB external storage. As I understand, Syno can use only internal drives as cache, and even those with some limitations. Maybe I’m wrong, I don’t have expirience with cache or external drives on Syno.
Databases and logs only, for me its to risky to put orders or metadata. (Maybe as an read cache if its possible to do so)
2 Likes
I just want to add that speed tests can be very missleading if they don’t reseamble your use case.
Many many flash brands put 1 GB of high speed memory of some sorts to look good in speed tests, because the default tests use 1GB of data, but after that, the rest of the memory is a few times lower in performance. I don’t know how they achieve that and how it works, but I’ve seen in many flash drives the same manifestation.
Here i read a lot about it.
also reformating for the usecase is important. i used ntfs NOT exfat
2 Likes
I didn’t know about this site. But they also use 1GB test data. They should speak openly about this gimmick. “We know that the 1GB test sample is not representative for the entire capacity of the drive, but we show you only these tests”. But than you won’t click the damn Amazon affiliate link. 
NTFS for Win nodes or ext4 for Linux nodes.
BTW… how should I format these flash drives for linux use? I have zero expirience with manual formating of drives, in linux.
I want to keep using ext4, but what else should I set, and what is the proper command?