The cursor moves from 10% to 25% in one step.
We do not wait on the individual nodes, since that would be a huge bottleneck and attack vector (ie on purpose slowing down)
I see. I hoped for more granular moves, not necessarily waiting for nodes.
I will therefore need to figure out how to opt out of this mechanism to avoid murdering my HDDs then.
move the storagenode database to an ssd is suppose to be a pretty good option.
else adding this is suppose to help with disk fragmentation
docker run -e RUN_PARAMS="--filestore.write-buffer-size 4096kiB --pieces.write-prealloc-size 4096kiB" -d --stop-timeout 300
the concept behind it is pretty sound, been trying to test it, but ended up disabling it again, due to other unrelated issues, which i how got solved.
so i will most likely be adding it back when i restart my nodes next time.
but been running it for a good while and haven’t seen any ill effects…
What does this even have to do with the file walker process? It’s not writing anything.
2 Likes
no way to avoid the filewalker, only thing you can do is free up more io, by methods like those… or metadata caching, like a ssd read cache, but metadata gets pretty large for big disks.
you would need like 1 or 2GB pr 1 TB storj data just to cache the metadata for all the small files.
i’ve seem people say it took nearly a week for their SMR drive to do run the filewalker…
HDD’s are pretty reliable… they really shouldn’t wear out just because of a bit of heavy IO.
SO LONG AS YOU KEEP THEM COOLED.
30-50 C is optimal, basically they shouldn’t be painful to touch… thats an easy way to guestimate 50 C
you can also see their temps in smartdata or using tools like hddtemp
Alright. I lived dangerous and chose to upgrade. I use Synology DS920+ and basicly. I stop the node. put “5f2777af9-v1.50.4-go1.17.5” instead of “latest”. It pulled the image and I hade to remove the old node and run the command once again.
1 Like
There is a config option for the check interval. By default it is 15 minutes. You could set that to 12 hours. Now if you start 2 nodes at the same time they would still update at the same time. Just restart one of them 12 hours later and that node should keep its offset trough multiple updates. So if your system never stops it should update both nodes with 12 hours difference even halve a year later.
Yes me. Issue is atime. Just disable it and the SMR drive will finish the filewalker as fast as any other drive.
2 Likes
So I gave the new docker image a go on my less valuable node. Everything seems to have worked. I ran it with the 5f2777af9-v1.50.4-go1.17.5 tag and it started fine. Following the logs it appears to have started, then almost immediately stopped and restarted. I am assuming it did the update at this time. Node is now running v1.50.4.
uname -a
Linux rock64 4.4.202-1237-rockchip-ayufan-gfd4492386213 #1 SMP Sat Nov 23 13:55:47 UTC 2019 aarch64 GNU/Linux
(please don’t shame me for my ancient kernel)
Should I no longer have watchtower monitor this node, or is leaving it under watchtowers watch okay?
Quite a bit confusing and strange, that there is no clear statement of the storjlings… Asking for testing support, but limiting testing users to those, who are hardcore users. ^^
I would expect something like this:
docker pull storjlabs/storagenode:5f2777af9-v1.50.4-go1.17.5
docker stop -t 300 sn1
docker rm sn1
docker run -d --restart unless-stopped --stop-timeout 300 \
-p 28967:28967/tcp \
-p 28967:28967/udp \
-p 127.0.0.1:14002:14002 \
-e WALLET="xxx" \
-e EMAIL="xxx@gmail.com" \
-e ADDRESS="xxx: 28967" \
--mount type=bind,source="<identity-dir>",destination=/app/identity \
--mount type=bind,source="<storage-dir>",destination=/app/config \
--mount type=bind,source="<db-dir>",destination=/app/dbs \
--name sn1 storjlabs/storagenode:5f2777af9-v1.50.4-go1.17.5
Is that something with what a noob like me can work with?
Is that what is expected to be tested and consequently watch the logs after restart?
Sounds like a solution. Is this configuration option documented?
Yes, that is all you need to do. You can actually skip the docker pull step if you want, as docker will pull the image itself when it doesn’t find it on the system. Pulling it first might save a few seconds of downtime though.
Yes you can find it here: Please join our public test network
In your case you would need the oposit direction. You want to increase it. In the linked thread the idea was to decrease the interval for faster updates.
for those interested.
i did the docker pull command which is.
docker pull storjlabs/storagenode:5f2777af9-v1.50.4-go1.17.5
and then i did my docker run command which looks something like this.
docker run -d -e RUN_PARAMS="--filestore.write-buffer-size 4096kiB --pieces.write-prealloc-size 4096kiB" -d --restart unless-stopped --stop-timeout 300 -p 192.168.1.100:28967:28967/tcp -p 192.168.1.100:28967:28967/udp \
#--log-opt max-size=1m \
-p 192.168.1.100:14002:14002 -e WALLET="0x111111111111111111111" \
-e EMAIL="your@email.com" -e ADDRESS="global.ip.inet:28967" \
-e STORAGE="4TB" --mount type=bind,source="/sn3/id-sn3",destination=/app/identity \
--mount type=bind,source="/sn3/storj",destination=/app/config --name sn3 storjlabs/storagenode:5f2777af9-v1.50.4-go1.17.5
note the storjlabs/storagenode:5f2777af9-v1.50.4-go1.17.5 at the end of the command.
incase anyone was confused… looks at bivvo
the node seemed to start up just fine, but after running a brief moment it stopped.
with this… which is most likely cause because i don’t run the automatic storagenode updater stuff nor watchtower. @littleskunk
2022-03-22 20:41:39,398 INFO spawned: 'storagenode' with pid 93
2022-03-22 20:41:39,399 WARN received SIGQUIT indicating exit request
2022-03-22 20:41:39,405 INFO waiting for processes, storagenode, storagenode-updater to die
2022-03-22T20:41:39.405Z INFO Got a signal from the OS: "terminated"
2022-03-22T20:41:39.438Z INFO Restarting service. {"Service": "storagenode-updater"}
2022-03-22 20:41:39,443 INFO stopped: storagenode-updater (exit status 1)
2022-03-22 20:41:39,451 INFO stopped: storagenode (terminated by SIGTERM)
2022-03-22 20:41:39,452 INFO stopped: processes (terminated by SIGTERM)
going to launch the node back up on the latest image
let me know if there is anything else i should do.
1 Like
let the script run as mentioned below 5mins ago - and it works for me.
so far (Debian bullseye on RPi4B).
Me, too.
Your NodeIDs are so close?
I have three docker nodes, only one updated to the latest version so far with this image. Even Windows node still not updated.
tried to relaunch the node, it just stopped again…
near instantly…
i’m on proxmox debian 5.13 running in containers.
Could you please describe more details, what’s that mean?
Do you use docker or LXC?
If LXC, the Debian 5.13 does not supported anymore, accordingly their documentation: Linux Container - Proxmox VE
Do you may be mean the kernel version?
I’ve got 16 nodes. This means the jump from 10% to 25% will likely cover 3 of them, or maybe more.
> table(replicate(10000, {x <- runif(16); sum(x > 0.1 & x < 0.25)}))
0 1 2 3 4 5 6 7 8
727 2129 2830 2258 1246 574 187 40 9So it was published, but not documented. Well, whatever, will try.
And all of them uses the same disk?