Power Usage: Is There a Difference Between Hashstore and Piecestore?

I’m curious about the power usage implications of using Hashstore versus Piecestore in Storj. Does anyone know if there’s a significant difference in energy consumption between these two models? Hashstore seems to focus on metadata and hashes, which could mean less strain on disks. What are your thoughts?

It might be the same - but when having many nodes - alot of power is being used. So here even 10% reduction would be awesome

I expect hashstore to be more energy efficient due to cheaper deletions.

4 Likes

That would be cool - im looking forward to seeing it. Hopefully we can get some more concrete idea of the difference

I personally doubt it.

The hard drive is still basically active 24/7, no chance to spin down and always some random seeks keeping the actuators moving around.

I don’t know if CPU usage is really different, or different enough to be measurable.

Shouldn’t hashstore need more power for the hdd part because all data is rewritten on compaction again and again? :thinking:

looks like i have compaction cycle, my cpu 12700 has 100% with 17 nodes

Fully agree, it’s a pure theoretical discussion. Like dropping a needle in the ocean raises water levels, but not really.

There are three major energy consumers in HDD

  • motor, overcoming friction, all the time
  • Actuators moving heads
  • Electronics running firmware.

You can’t do anything about the first, but you sure can, and drives do, everything about the second and last. Search for “Extended Power Conditions”. On FreeBSD you can use camcontrol to check what power modes your HDD suppoprts

camcontrol epc da0 -c status

In the frame of this discussion – fewer actuator jerks, fewer energy wasted, and in parallel, more opportunity for firmware to sleep.

Writing logs requires fewer IO than writing individual files, so here where the differnce will come from. Granted, ZFS already does it with piecestore as well, so no difference there.

No, because it happens infrequently and all at once, and then back to idle, allowing CPUs and HDDs firmware to sleep. Short bursts of compactions is therefore irrelevant, what matters is what happens all the time.

Which brings us to the deletion. There are no piece deletions with hashstore, at all, and with piece store deleting takes CPU time and memory pressure, all of which are quite power hungry. This is another source of pin in the ocean power savings.

1 Like

What is Hashstore and Piecestore? Which one is the default option? That’s what I am using.

If you dont know - you are currently on piecestore
but in some time we will all go to the new hashstore - as far as i know - you dont need to do anything.

1 Like

Thanks, that’s perfect, I will leave it at that. Will let storj do it’s thing. I have 6 nodes and they all are working ok, I am not tinkering with it.

2 Likes

That’s the spirit!

pieces store is the current method where there are millions of small files.

hashstore will compact the files into “logs” of 1GB each to better handle massive file accesses and bulk deletes, etc.

If you feel sassy you can beta start migrating a node or two:
[Tech Preview] Hashstore backend for storage nodes - Product Discussions / STORJLINGS - Storj Community Forum (official)

Hashstore Migration Guide? - Node Operators / getting started - Storj Community Forum (official)

3 Likes

Thank you. Do you know with which version is the automatic migration coming?
Some of my nodes are on v1.135.5, and some on v1.136.4.
After migration, do I need to delete any temporary directories? Guide on one of the links you provided mentions deleting files from blobs directory.

The automatic migration hasn’t been officially started or been scheduled yet. So just.. “the future”.

2 Likes