How much "slower" is ZFS?

[gingerbread233]

@arrogantrabbit

Slog will help offload IO from main army for the intent log. 100-200MB is plenty for most users. Buy smallest optane you can find - which is 16GB for $10 last I checked.

That’s why, also I want to use my zfs for own data too. That’s why. I built a small test setup to play with, before migrate my data and storj node to ZFS.

[alpharabbit]

@Roxor

Also… wasn’t the node switched to request async writes by default?

Announcement: major storage node release (potential config changes needed!)

Hello everyone! If you’ve been following the forum closely, you may have noticed three separate things: This thread, where we talked about storage node performance benchmarking. Release v1.104, where we have many changes from the benchmarking thread This running operational update, where we documented the rollout of the v1.104. But if you haven’t been following the forum as closely, this thread is for you! Big performance changes for nodes in v1.104.x v1.104.1 has started rolling out, and i…

So sync=default should be ok.

[arrogantrabbit]

Yes, but obviously, not fake :). There are legitimate used ones on eBay at about $10. How would legitimate M10 be sold on aliexpress? They are discontinued, if you find “new” m10 at low price — well, it’s other not new (good) or fake (bad) or old factory reject (you also don’t want it), or old stock (but how would you know?)

What type of other data do you store? You can turn off sync there as well or it is not needed in the first place?. You have a UPS, right?

[gingerbread233]

I want to store private data too, photos videos etc., some game “servers”, teamspeak etc. I would asnyc the storj dataset, since I have an UPS, and might got with 128GB too. So sync or “default” will only be applied to my “private” datasets.

[arrogantrabbit]

The point being, build the pool first. Start using it. See if there are any sync writes in the first place. Then check if those sync writes have any effect on performance. Only then consider slog. You will likely discover you don’t need it.

@gingerbread233

would asnyc the storj dataset, since I have an UPS

I would disable sync on storj even if I did not have a UPS. Storj data is not important and can tolerate occasional loss. But I guess they no longer send sync writes anyway.

[alpharabbit]

BTW: I have changed primarycache from metadata back to default for all my storj nodes. This results in significantly less drive activity.

I don’t know why this is. The reason could be egress being less random than expected. Also I am not sure how read-ahead works? If it uses arc this might be another reason.

[EasyRhino]

@alpharabbit

BTW: I have changed primarycache from metadata back to default for all my storj nodes. This results in significantly less drive activity.

oh the PRIMARY ARC was metadata only? yeah that seems not good. Partly becuause my server hosts things other than storj data, and partly because, you know, maybe some very recently accessed files need to be held in ram, I just leave it on default.

You could look at arc_summary and try to see the percent of types of data stored and hit rates.

for L2ARC I do metadata only. mainly to ensure that I size the partitions okay to hold all the metadata and not get it clogged with other blocks.