Alpha testing is done before beta as far as I know.
In layman’s terms.
Alpha version: Testing internally by the devs and internal testers (QAs).
Beta version: Tested by non staff or larger audience from end users who opt to test beta versions. You can experience this in an Android app’s description where you can opt for beta releases.
Release Candidate: Devs did their best to address all issues but this needs extensive testing before being deployed.
Final Version: Ready for end user.
Does this mean that with every piece write/deletion, the db regarding used space is not updated anymore? Meaning less IO on drive?
I think yes. This is the whole point of using a system reported space instead of calculating it.
I checked my nodes and they are still running with no out of memory restart. And my system doesn’t have a lot of memory to begin with. So I can’t confirm a memory leak. It runs nice and stable on my nodes.
i would like to use that feature, and turn off filewalker permanently, as my 16TB disk is dedicated to Storj, BUT rarely its 100% true, i mean i keep GB’s of logs on it, and sometimes small files backups, that i don’t have space anywhere else - could be some 100-200GB sometimes more, wonder if the feature can substract all the other than storj folders from the storjs free disk space to know how much left, or is it just listening for what the OS system says about total free space on that disk/volume?
1.114.1 introduced the high memory usage with this commit: 9e5a1ea storagenode/store: Fix data race
It was reverted in 1.114.2, fixing the high memory usage: bb2c09b Revert “storagenode/store: Fix data race”
1.114.3 has a revised fix for the data race: c0018dd storagenode/orders: fix data race on unsent map
It used the reported used/free space by the OS and honor the reserved free space for this feature (300GB by default).
So… don’t install pre-release versions and complain about it.
Reporting an issue is not complaining.
I tried to update one of my running 1.113.2 windows node to the 1.114.3, but received the below error during the first start. I did not changed anything else but overwrite the old exe file with the new one.
The same PC has 4 other perfectly running 1.113.2 nodes.
Due to database update, I can not go back to version 1.113.2.
2024-10-01T13:34:03+02:00 INFO db.migration migrate/versions.go:221 Database Version {"version": 62}
2024-10-01T13:34:03+02:00 INFO preflight:localtime preflight/localtime.go:54 start checking local system clock with trusted satellites' system clock.
2024-10-01T13:34:03+02:00 ERROR storagenode/peer.go:1010 Failed preflight check. {"error": "system clock is out of sync: system clock is out of sync with all trusted satellites", "errorVerbose": "system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:1009\n\tmain.cmdRun:125\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
storj.io/storj/storagenode.(*Peer).Run
/go/src/storj.io/storj/storagenode/peer.go:1010
main.cmdRun
/go/src/storj.io/storj/cmd/storagenode/cmd_run.go:125
main.newRunCmd.func1
/go/src/storj.io/storj/cmd/storagenode/cmd_run.go:33
storj.io/common/process.cleanup.func1.4
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:392
storj.io/common/process.cleanup.func1
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:410
github.com/spf13/cobra.(*Command).execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983
github.com/spf13/cobra.(*Command).ExecuteC
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
storj.io/common/process.ExecWithCustomOptions
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:112
storj.io/common/process.ExecWithCustomConfigAndLogger
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:77
storj.io/common/process.ExecWithCustomConfig
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:72
storj.io/common/process.Exec
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:62
main.(*service).Execute.func1
/go/src/storj.io/storj/cmd/storagenode/service_windows.go:107
golang.org/x/sync/errgroup.(*Group).Go.func1
/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
2024-10-01T13:34:03+02:00 ERROR storagenode/cmd_run.go:50 failure during run {"error": "system clock is out of sync: system clock is out of sync with all trusted satellites", "errorVerbose": "system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:1009\n\tmain.cmdRun:125\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
main.cmdRun.func1
/go/src/storj.io/storj/cmd/storagenode/cmd_run.go:50
main.cmdRun
/go/src/storj.io/storj/cmd/storagenode/cmd_run.go:128
main.newRunCmd.func1
/go/src/storj.io/storj/cmd/storagenode/cmd_run.go:33
storj.io/common/process.cleanup.func1.4
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:392
storj.io/common/process.cleanup.func1
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:410
github.com/spf13/cobra.(*Command).execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983
github.com/spf13/cobra.(*Command).ExecuteC
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
storj.io/common/process.ExecWithCustomOptions
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:112
storj.io/common/process.ExecWithCustomConfigAndLogger
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:77
storj.io/common/process.ExecWithCustomConfig
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:72
storj.io/common/process.Exec
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:62
main.(*service).Execute.func1
/go/src/storj.io/storj/cmd/storagenode/service_windows.go:107
golang.org/x/sync/errgroup.(*Group).Go.func1
/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
2024-10-01T13:34:03+02:00 FATAL process/exec_conf.go:428 Unrecoverable error {"error": "system clock is out of sync: system clock is out of sync with all trusted satellites", "errorVerbose": "system clock is out of sync: system clock is out of sync with all trusted satellites\n\tstorj.io/storj/storagenode/preflight.(*LocalTime).Check:96\n\tstorj.io/storj/storagenode.(*Peer).Run:1009\n\tmain.cmdRun:125\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:392\n\tstorj.io/common/process.cleanup.func1:410\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
storj.io/common/process.cleanup.func1
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:428
github.com/spf13/cobra.(*Command).execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983
github.com/spf13/cobra.(*Command).ExecuteC
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
storj.io/common/process.ExecWithCustomOptions
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:112
storj.io/common/process.ExecWithCustomConfigAndLogger
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:77
storj.io/common/process.ExecWithCustomConfig
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:72
storj.io/common/process.Exec
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/process/exec_conf.go:62
main.(*service).Execute.func1
/go/src/storj.io/storj/cmd/storagenode/service_windows.go:107
golang.org/x/sync/errgroup.(*Group).Go.func1
/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
Ok, so in config.yaml, there is a line for:
# list of trust sources
# storage2.trust.sources: https://www.storj.io/dcs-satellites
If I take off the # from the second line, the node starts with 1.114.3, but the log is full with such error messages. It is online, but misconfigured.
2024-10-01T14:06:39+02:00 ERROR contact:service contact/service.go:161 ping satellite failed {"Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:202\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:156\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
storj.io/storj/storagenode/contact.(*Service).pingSatellite
/go/src/storj.io/storj/storagenode/contact/service.go:161
storj.io/storj/storagenode/contact.(*Chore).updateCycles.func1
/go/src/storj.io/storj/storagenode/contact/chore.go:87
storj.io/common/sync2.(*Cycle).Run
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/sync2/cycle.go:99
storj.io/common/sync2.(*Cycle).Start.func1
/go/pkg/mod/storj.io/common@v0.0.0-20240913104706-8fd6fa0286a3/sync2/cycle.go:77
golang.org/x/sync/errgroup.(*Group).Go.func1
/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78
No, you may install, but I would suggest to do not install it on production nodes. There are could be dragons.
This is due to inability to contact your node by the satellite.
Please search for the reason (PowerShell):
sls "ping satellite failed" "C:\Program Files\Storj\Storage Node\storagenode.log" | sls -NotMatch "rate" | select -last 10
@Alexey
Sorry, it was my mistake as I installed it for a production node.
If you mind, please delete/hide my posts, so it will not mislead anyone else.
Thank you!
not needed. Or you may delete it yourself, if you want. But I believe it may be helpful for another one SNO, who do not follow the official update plan from https://version.storj.io
Ok, what am I missing here? Been busy so haven’t been following things here to closely. I updated nodes to 1.114.3. Nodes on ubuntu system work fine, but get the clock out of sync error on all bsd nodes. Added the ‘storage2.trust.sources:’ line causes the nodes to start but now they get the rate limit warning and no ingress… egress is unaffected. I changed nothing else except updating the executable and restarting. I’m getting really close to letting my nodes go. It’s not that their not profitable, but even with several hundred TBs, there not nearly profitable enough to account for my time to troubleshoot basic updates like this.
Действительно, на Windows OS после обновления на 1.114.3 нет входящего\исходящего трафика.
I would insist on using the recommended version from https://version.storj.io
@Vasabi you too, please.
Please, do not install not ready versions for production nodes, use them only for the QA satellite.
I believe, that there is a check, that versions which didn’t match the minimum or a recommended are excluded from the node selection and maybe even rejected to check-in.
Я и использую рекомендуемую версию с https://version.storj.io
Просто иногда на малых узлах пробую запустить новые релизы.
13 posts were split to a new topic: The whole disk/partition allocation feature