14TB Ultrastar HDD - best setup - Ubuntu Server

Node Operators
, ,
1

Hi all,

I’m about to recieve a 14TB Ultrastar… This is my biggest node by a long long shot… I want to make sure i get the HDD setup correctly to best handle the data / data sizes…

Looking for help / guidance on configuration.

There is also an option if it is better suited to have this on Windows ( but i thought Linux is best suited )

Cheers,

2

Don’t use it on windows IF YOU have experience with linux and like linux.

On windows You need to make sure the disk will be formatted on default and on the exact same windows version (win10/win11) You will use it for storage. But the eventual problems with perfromance still can occure later after the disk will be somehow full. There was some topics about windows node optimization here, also NTFS “adventures”. Linux seems to be problemless where in later use, if You are familiar with linux dont look back to windows in terms of Storj Nodes i warn You or You will be spending time later to figure out why the filewalker is so slow and how to fix that.

3

Understood !

I have 4 Linux Storj Nodes currently… so I will just add another :slight_smile:

i normally format to ext4, leaving everything default… - do you know if i should set anything special when making it ext4?

4

I don’t like linux, but it is the way to go for running a big number of nodes.

@ItsHass
I recommend to use zfs for the storj disks. For the OS ext4 is ok.

5

I use the same drives with EXT4, no ZFS.
Turn off write caching, disable power save (so disk never spins down)
One HDD per node with 10% headroom.

1 Like
6

Any specific settings for ext4 or just out of the box configuration ?

7

Pretty much out of the box, other than disabling write-caching and turning off power save.
The biggest issue for people is that the SATA connector has a ‘power disable’ pin which may need a special cable if the drive won’t power up.

8

I would probably leave the write caching enabled for performance? storj has a boatload of small writes you’d think it would help.

I recently got a couple of 14TB drives. Definitely set up a new node for each drive.

Other than that there’s still some debate over optimal setup. Especially with the bulk of test data coming in, filewalker problems, etc.

I have an ext4 drive that got overly fragmented at 7TB and performance is the pits.

my new drives i’m setting up with zfs. each drive is an independent zpool.

  • redundant_metadata=some
  • secondarycache=metadata
  • a small SSD partition being used as l2arc, 5GB per TB.

It seems… okay… so far. Big test will be as drives get fuller and fragmentier and filewalker starts taking a couple of weeks to run.

9

Sounds like you have a good setup strategy.

Are you familiar with any defrag for Linux?

10

Please recommend me an unframer for Linux.

I use ext4

11

What is an “unframer”?

1 Like
12

@Toyoo has a really good post regarding fine tuning EXT4 for Storj.
I recommend you reading that thread for some nuggets of information :slight_smile:

2 Likes
13

sorry, defrag

thx…

14 
e4defrag

20 characters

2 Likes
15

Honestly I didn’t know, that it’s needed for ext4

16

That’s a different question :wink:

I have a virtual full 16 TB node

Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/storagenode4-node   16T   16T   83G 100% /mnt/storagenode4

I checked 1 of the folders in the saltlake satellite’s blob folder and it returned


Total/best extents                             82048/77808
 Average size per extent                        135 KB
 Fragmentation score                            1
 [0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
 This directory (/mnt/storagenode4/node/storage/blobs/pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa/ds/) does not need defragmentation.
 Done.
2 Likes