Sharing my STBB Docker image + hashstore analysis script

Node Operators
, ,
1

I’ve been doing quite a bit of digging into hashstore behavior lately, especially around reclaimable space and compaction not keeping up.

To make analysis easier (and more reproducible), I put together:

Suggested settings:

–hashstore.compaction.alive-fraction float
–hashstore.compaction.probability-power float
–hashstore.compaction.rewrite-multiple float

What this does

The script will:

  • Detect Storj containers (docker-compose based)
  • Optionally stop them safely
  • Run STBB against all hashstores (per satellite)
  • Generate reports (stat / logs / etc.)
  • Restart containers afterwards

This made it a lot easier for me to get a clearer picture of:

  • How much data is actually reclaimable
  • How “alive” the log files are
  • Whether compaction settings are realistic

This is a technical tool, not a polished UI…

Why I built this??

The dashboard doesn’t always reflect what’s actually happening inside hashstore.

Using STBB directly gives much better insight, but:

Running it manually across multiple nodes/satellites is tedious
reading all logs manually is… not great

This setup just automates that workflow.

If anyone has suggestions or improvements, feel free.

2

Looks like a cool tool!

When you say:

…do you mean like a bug: where uncompacted space will keep climbing? Or do you mean it technically is keeping up (given the chosen params) but some SNOs may be unhappy with the average amount of reclaimable space their node stays at?

3

i mean.. the default settings are a bit conservative in my opinion.
I think my nodes could do with a bit more aggressive settings.
And i found it hard to find a decisive answer on the forums.
So at a night a couple of weeks back… when i had nothing else to do i went into the rabbithole of hashstore.

And this tool is the result. (but it took longer than 1 night ;-))

Edit: i am still playing around with different settings to see what i like

4

I wonder if the results (best settings based on these tools analysis) are the same for any setup/node out there, or every SNO can find different results?
If var.1 is the case, could someone test a big node and put here the results for anyone to apply them?

5

Made with the help of Claude

IMG_4612
IMG_46121170×2278 281 KB

IMG_4611
IMG_46111170×2210 180 KB

More aggressive hashstore settings

6

what constitutes a big node? i have got a 14T one. is that enough?

7

It’s perfect! Way bigger than mine.

8

image
image474×112 7.83 KB

Place your web site behind cloudflare. You will get authentication, if needed, and cloudflare will take care of certificates and fight off bots.

9

afbeelding
afbeelding785×809 39.2 KB

This is the 14T node

i am still looking at the 0.65 suggestion… i brought some nodes to 0.35 and that already took a lot of trash away compared to 0.25 default.