Post pictures of your storagenode rig(s)

Node Operators
1227

Something is very wrong with the fans. None is aligned with the heatsink fins direction, let alone its absolutely inadequate if you want airflow and not just fans looking expensive.I would use actual fans, not noctua, and ensure airflow is laminar and everting is aligned. I assume there are fans behind drives as well?

If you want low noise ask AI to create software fan controller for you. (I’ve done just that and saved another 5 watts on airflow). Depending on what mlb you are using there might be already fan controller — for example, the popular nuvoton BMC chips have Smart Fan IV profile that you can feed HDD temperatures to in software. Aspeed chips don’t expose such interface but in this case you can drive fans in software. (BTW - pi(d) controller won’t work well for this, use segmented curves with hysteresis instead)

Depends on what else are you going to store there and access patterns. You can add (and remove, if your array is just mirrors) special device later. You can also add/remove slog and/or l2 arc anytime

Start without any caching or special devices, then measure performance over few days of steady typical use. This will inform what you need if anything. Maybe the right thing to do would be removing ram, not adding more hardware.

1 Like
1228

Thank you for the kind words all

@Roxor correct; it’s that exact sliger model. I’ve not slid the motherboard tray all the way into the chassis yet, giving it an extra ~10cm of working space, which really, really is a lot in that case. I don’t like the performance loss of running RAIDz2, and I really, really don’t like the performance loss of not having two striped vdevs; I’ll stay on RAIDz1 on 5 physical devices. Makes a future expand much easier too, albeit it’s a bit of a moot point with zfs volumes now being able to be expanded upon.

@snorkel the motherboard in here does not matter much; it’s just a placeholder. Some gaming Gigabyte board I had in an older sff machine; AMD 3600x, 16GB RAM and Noctua NH-L12 cooler.

Will be replaced with a Gigabyte MJ11-EC1, which is in the mail. That will have 128GB ECCRAM, CPU is an embedded EPYC3151. I’ll be utilizing the 8x onboard SATA ports together with an M.2 addincard for additional 6x ports. 14 is more than I need right now.

@andrew2.hart yes, it is. The side mounted fan is unconventional, but the entire chassis is only 15" deep, so it’s not possible to get a normal fanwall behind the drives. The drives are suspended with a ~10mm gap inbetween them, so cooling with be a nonissue.

I just have a rack at home; no cold isles; no fancy AC equipment. Thermal load of the box is less than 150 watts, cooling won’t be an issue.

@arrogantrabbit looked forward to suggestions and scrutiny from you! :slight_smile: CPU cooler cannot be turned due to boardlayout. Won’t matter; swapping to another board - Gigabyte MJ11-EC1.
Regarding cooling, the drive spacing gives so much air, that ambient cooling is almost enough, (mid 40s with fans OFF), and with fans on at ~1000 RPM drive temperatures is mid 30s.

Thank you for linking your fan control. Currently the noctuas are good enough, but if I do decide to upgrade to 120.38 fans, I’ll most definitely take a deeper look.

Disks will be 2x RAIDz1 volumes striped together, so removing the special device is not quite possible, but you’re right, it’s much better to look at performance without any caching, and go from there.

The box will house and host all my StorJ and store but not host files. Immich data will live on the box, but the app will be hosted by another box, most likely a M2 Mac Mini. That data will be backed up at another location. Movies, TV-shows and most other data will ditto, but not backed up externally.

3 Likes
1229

lol I just assumed it was in the data centre since that’s all I’m doing at the moment. I hate these AI servers - 8 GPUs and 6 3000W power supplies - what a stupid design. Our racks can’t provide that much power

1230

I have a huge problem with this mlb conceptually. It is great on first look, but if you start looking carefully, the design decisions they made are a bit questionable.

  1. 2x 1Gbps ethernet. Why? Why two? Why 1 Gbps? Why not 2.5Gbps? Why tie up two PCIE lines with two slow ethernet controllers?
  2. SATA – 4 onboard ports, fine, even though one would suffice; but if you have four, it’s tempting to connect data drives there, but 4 is then too few, and you would need to buy a slimsas-to-4x sata cable, which is surprisingly expensive, and STILL be limited to only sata disks; often SAS disks are cheaper, so you are overpaying there too.
  3. M.2 on the board is 100% gimmick. It’s completely useless. What are you going to plug into there? Just give me a proper PCIE slot and I’ll decide what to stick in this. Oh yes, you can connect m2 to PCIE daughterboard, but it’s an extra expense and fragility. You mentioned m2 sata controllers. They don’t exist in the enterprise. It will have to be some rinky-dink Shenzhen combabulation – so overall reliability will plummet through the floor.
  4. Why do I need two PCIe slots?
    • One is self-inflicted – 2.5Gbps network. Makes a huge difference. 1Gbps is dead. People internet nowadays is faster.
    • Second one – for a proper HBA. Like Broadcom 9500-16i or 8i. Consumes 4 watts, and provides you 16 or 8 devices, sas and sata, and even nvme if needed. There are even 9400-16i on ebay for about $100. That may be what you need.
  5. Nice to see IPMI, but Aspeed 2500 is not exactly a a paragon in power efficiency - rather important for a tiny power efficient board.

It’s mostly a me problem, devices that try to be jack of all trades master of none annoy me beyond reason. This could have been so much better product – instead, it’s a collection of shortcuts and compromises. no surprise gigabyte thew in a towel on a whole product line.

2 Likes
1231

I hide my last node behind the TV. Installed ARMbian linux on the TV BOX and attached 4GB of external HDD.

photo_6314172859877625736_y
photo_6314172859877625736_y960×1280 146 KB

1 Like
1232

This could have used ten seconds of dusting and a photo without flash.

Also, probably storj customers after seeing where is their data actually stored:
:scream:

1 Like
1233

I suggest you to open the case, that hdd can get some better cooling, or it will die soon.

1234

these HDD’s aren’t very longlived anyways. i guess they are all SMR too

1235

(post deleted by author)