Manually start filewalker

Ideas & Suggestions
1

It woud be usefull to start filewalker manually if you have more nodes on the same server.

2 Likes
2

By default the used-space filewalker runs every restart of the node, so just restart your node to trigger it.

Garbage collection filewalker only runs when you receive a bloom filter, but in newer storagenode releases there will be ability to store the bloom filter to rerun if garbage collection was interrupted.

3

Startup filewaker set to FALSE in config for obviously reason. I want to decide exactly when to start. If I reboot the server could be a mess with automatic startup filewalker

4

Then it seems you would have to set filewalker to TRUE, restart, set to FALSE and restart again. :confounded:

1 Like
5

So… if you have many nodes it would be usefull to start filewalker manually

6

This is where the power of scripting comes in.

For example (need to first uncomment the line in the config):

#!/bin/bash
CONTAINER_NAME=storagenode
CONFIG_PATH=/mnt/storj/config.yaml

docker container stop $CONTAINER_NAME
sed -i "s/^storage2.piece-scan-on-startup: .*/storage2.piece-scan-on-startup: true/g" $CONFIG_PATH
docker container start $CONTAINER_NAME
sed -i "s/^storage2.piece-scan-on-startup: .*/storage2.piece-scan-on-startup: false/g" $CONFIG_PATH

However I do agree with having more feature rich APIs. It would be nice to start/stop filewalkers and query their status from the web API.

7

You didn’t get it. He/we need a way to delay the filewalker on some nodes, until it finishes on one node or at most 2. Like: you have 4 nodes and you know roughfly it takes 3 days for the FW to finish. You set the first 2 nodes to run Fw at startup, and for the other 2, you set a delay of 4 days after startup. They will run the FW unattended, 2 and 2.

8

I’m not sure what the purpose is of staggering filewalkers for different nodes. If you are abiding by one node per disk then reads are independent.

However you could also script that functionality.

1 Like
9

They consume RAM, aka more RAM you have, much faster FW runs. Maybe dosen’t matter too much, or maybe it does, running them one by one. I didn’t made any tests.

10

I think in setup like th3van(100 nodes) is not a good idea to let FileMaker start on boot. All together.
In setup with one hba card (6g) for many nodes too