Disk usage discrepancy?

There is an emergency check - if the free space in the allocation or on the disk is lower than 5GB, the node will immediately report to the satellites, that’s full.

1 Like

Can you please send me a link to do this? I looked at many forums but there were so many that it was difficult to determine which one corresponded to my symptoms.

You should have some logs entries on the Synology, if the disk is disappearing. Is it an external HDD? If so, it need to have an additional power supply.

It is not an external hard drive. :grinning:

it’s also possible to specify it as a command line argument in your docker run command after the image name as --pieces.enable-lazy-filewalker=false
However, after that you would not see filewalker in your logs, you need to track it differently

1 Like

See here:
https://forum.storj.io/t/my-docker-run-commands-for-multinodes-on-synology-nas/22034?u=snorkel

I don’t use that line about user and group. Try to remove it, because you run it like root; you start it with sudo.

1 Like
sudo docker run -d --restart unless-stopped --stop-timeout 300 \
    -p 28967:28967/tcp \
    -p 28967:28967/udp \
    -p 14002:14002 \
    -p 5999:5999 \
    -e WALLET="0xCb97E443b7458eE" \
    -e EMAIL="***@gmail.com" \
    -e ADDRESS="***:28967" \
    -e STORAGE="14.3TB" \
    --log-driver json-file \
    --log-opt max-size=10m \
    --log-opt max-file=5 \
    --mount type=bind,source="/volume2/storj/identity",destination=/app/identity \
    --mount type=bind,source="/volume2/storj/data",destination=/app/config \
    --mount type=bind,source="/volume2/storj/data/node.log",destination=/app/logs/node.log \
    --name storagenode storjlabs/storagenode:latest \
    --log.output=/app/logs/node.log \
    --storage2.piece-scan-on-startup=true \
    --server.address=":28967" \
    --console.address=":14002" \
    --debug.addr=":5999" \
    --log.level=error \
    --filestore.write-buffer-size 4MiB \
    --pieces.write-prealloc-size 4MiB \
    --operator.wallet-features=zksync \
    --pieces.enable-lazy-filewalker=false \

I configured lazy-filewalkers and restarted. Can I just leave it like this?

There doesn’t seem to be any change in my eyes right away, so it seems like it’s going to take some time, right?

I’m sorry to bother you.

As you suggested in the link, I removed --user $(id -u):$(id -g)! :grinning:

F:\storagenode.log:90113951:2024-06-29T17:31:02+01:00   ERROR   lazyfilewalker.used-space-filewalker    failed to start
subprocess      {"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
F:\storagenode.log:90113954:2024-06-29T17:31:02+01:00   ERROR   lazyfilewalker.used-space-filewalker    failed to start
subprocess      {"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "context canceled"}
F:\storagenode.log:90113957:2024-06-29T17:31:02+01:00   ERROR   lazyfilewalker.used-space-filewalker    failed to start
subprocess      {"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
F:\storagenode.log:90114044:2024-06-29T17:31:02+01:00   ERROR   filewalker      failed to get progress from database   {"error":
"gc_filewalker_progress_db: context canceled", "errorVerbose": "gc_filewalker_progress_db: context canceled\n\tstorj.io
/storj/storagenode/storagenodedb.(*gcFilewalkerProgressDB).Get:47\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).Wa
lkSatellitePiecesToTrash:154\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:565\n\tstorj.io/s
torj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgo
lang.org/x/sync/errgroup.(*Group).Go.func1:78"}
F:\storagenode.log:90114045:2024-06-29T17:31:02+01:00   ERROR   filewalker      failed to reset progress in database   {"error":
"gc_filewalker_progress_db: context canceled", "errorVerbose": "gc_filewalker_progress_db: context canceled\n\tstorj.io
/storj/storagenode/storagenodedb.(*gcFilewalkerProgressDB).Reset:58\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).
WalkSatellitePiecesToTrash.func1:171\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePiecesToTrash:244\
n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:565\n\tstorj.io/storj/storagenode/retain.(*Ser
vice).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Gr
oup).Go.func1:78"}
F:\storagenode.log:90114055:2024-06-29T17:32:31+01:00   ERROR   failure during run      {"error": "Error migrating tables for
database on storagenode: migrate: database: info opening file \"F:\\\\info.db\" failed: context canceled\n\tstorj.io/st
orj/storagenode/storagenodedb.(*DB).openDatabase:364\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).Migration.func1:
684\n\tstorj.io/storj/private/migrate.(*Migration).Run:167\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToL
atest:387\n\tmain.cmdRun:98\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io/commo
n/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).Execu
teC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstor
j.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/com
mon/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78",
"errorVerbose": "Error migrating tables for database on storagenode: migrate: database: info opening file
\"F:\\\\info.db\" failed: context canceled\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:364\n\tstorj.
io/storj/storagenode/storagenodedb.(*DB).Migration.func1:684\n\tstorj.io/storj/private/migrate.(*Migration).Run:167\n\t
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:387\n\tmain.cmdRun:98\n\tmain.newRunCmd.func1:33\n\tstor
j.io/common/process.cleanup.func1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Comman
d).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\t
storj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.
io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgo
lang.org/x/sync/errgroup.(*Group).Go.func1:78\n\tmain.cmdRun:100\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.
cleanup.func1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgi
thub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/proc
ess.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.Ex
ecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errg
roup.(*Group).Go.func1:78"}
F:\storagenode.log:90114056:2024-06-29T17:32:31+01:00   FATAL   Unrecoverable error     {"error": "Error migrating tables for
database on storagenode: migrate: database: info opening file \"F:\\\\info.db\" failed: context canceled\n\tstorj.io/st
orj/storagenode/storagenodedb.(*DB).openDatabase:364\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).Migration.func1:
684\n\tstorj.io/storj/private/migrate.(*Migration).Run:167\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).MigrateToL
atest:387\n\tmain.cmdRun:98\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io/commo
n/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).Execu
teC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstor
j.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/com
mon/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78",
"errorVerbose": "Error migrating tables for database on storagenode: migrate: database: info opening file
\"F:\\\\info.db\" failed: context canceled\n\tstorj.io/storj/storagenode/storagenodedb.(*DB).openDatabase:364\n\tstorj.
io/storj/storagenode/storagenodedb.(*DB).Migration.func1:684\n\tstorj.io/storj/private/migrate.(*Migration).Run:167\n\t
storj.io/storj/storagenode/storagenodedb.(*DB).MigrateToLatest:387\n\tmain.cmdRun:98\n\tmain.newRunCmd.func1:33\n\tstor
j.io/common/process.cleanup.func1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Comman
d).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\t
storj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.
io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgo
lang.org/x/sync/errgroup.(*Group).Go.func1:78\n\tmain.cmdRun:100\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.
cleanup.func1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgi
thub.com/spf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/proc
ess.ExecWithCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.Ex
ecWithCustomConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errg
roup.(*Group).Go.func1:78"}
F:\storagenode.log:90114373:2024-06-29T17:37:00+01:00   ERROR   failure during run      {"error": "Invalid configuration:
invalid contact.external-address: lookup \"mystorjtoshiba.ddns.net\" failed: lookup mystorjtoshiba.ddns.net:
getaddrinfow: A non-recoverable error occurred during a database lookup.\n\tstorj.io/storj/storagenode.(*Config).Verify
:171\n\tmain.cmdRun:61\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io/common/pro
cess.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1
115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/
common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/p
rocess.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78",
"errorVerbose": "Invalid configuration: invalid contact.external-address: lookup \"mystorjtoshiba.ddns.net\" failed:
lookup mystorjtoshiba.ddns.net: getaddrinfow: A non-recoverable error occurred during a database lookup.\n\tstorj.io/st
orj/storagenode.(*Config).Verify:171\n\tmain.cmdRun:61\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.fu
nc1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/s
pf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWi
thCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCust
omConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Gro
up).Go.func1:78\n\tmain.cmdRun:62\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io
/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command)
.ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n
\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.
io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
F:\storagenode.log:90114374:2024-06-29T17:37:00+01:00   FATAL   Unrecoverable error     {"error": "Invalid configuration:
invalid contact.external-address: lookup \"mystorjtoshiba.ddns.net\" failed: lookup mystorjtoshiba.ddns.net:
getaddrinfow: A non-recoverable error occurred during a database lookup.\n\tstorj.io/storj/storagenode.(*Config).Verify
:171\n\tmain.cmdRun:61\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io/common/pro
cess.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:1
115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n\tstorj.io/
common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.io/common/p
rocess.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78",
"errorVerbose": "Invalid configuration: invalid contact.external-address: lookup \"mystorjtoshiba.ddns.net\" failed:
lookup mystorjtoshiba.ddns.net: getaddrinfow: A non-recoverable error occurred during a database lookup.\n\tstorj.io/st
orj/storagenode.(*Config).Verify:171\n\tmain.cmdRun:61\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.fu
nc1.4:393\n\tstorj.io/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/s
pf13/cobra.(*Command).ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWi
thCustomOptions:112\n\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCust
omConfig:72\n\tstorj.io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Gro
up).Go.func1:78\n\tmain.cmdRun:62\n\tmain.newRunCmd.func1:33\n\tstorj.io/common/process.cleanup.func1.4:393\n\tstorj.io
/common/process.cleanup.func1:411\n\tgithub.com/spf13/cobra.(*Command).execute:983\n\tgithub.com/spf13/cobra.(*Command)
.ExecuteC:1115\n\tgithub.com/spf13/cobra.(*Command).Execute:1039\n\tstorj.io/common/process.ExecWithCustomOptions:112\n
\tstorj.io/common/process.ExecWithCustomConfigAndLogger:77\n\tstorj.io/common/process.ExecWithCustomConfig:72\n\tstorj.
io/common/process.Exec:62\n\tmain.(*service).Execute.func1:107\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
F:\storagenode.log:90115620:2024-06-29T17:47:17+01:00   INFO    lazyfilewalker.used-space-filewalker    subprocess exited with
status  {"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "status": 1, "error": "exit status 1"}

I have done defrag the disk using Iobit Smart defrag
Doing the system repaire now
didnt understand what i need to do for point 2,3 5

Hello,
as you can see on screens below, I have a problem with disk space and what node sees on disk. The difference is huge.


2024-07-01 17_53_35-192.168.1.12 - Connexion Bureau à distance

I looked the logs, there is some error with the lazyfilewalker, but dont know what I can do.

I am running a CHKDSK. Not even sure what parameters should I use : x/ c/ r/ ?

Ligne   16: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	starting subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne   19: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.used-space-filewalker	starting subprocess	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   23: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.gc-filewalker	starting subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne   25: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.gc-filewalker	starting subprocess	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   26: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.used-space-filewalker	subprocess started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   27: 2024-06-16T21:57:03+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess started	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne   30: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker	subprocess started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne   32: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker	subprocess started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   33: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker started	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "dateBefore": "2024-06-09T21:57:03+02:00", "Process": "storagenode"}
	Ligne   36: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	Database started	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode"}
	Ligne   37: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker completed	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
	Ligne   38: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess finished successfully	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne   41: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	starting subprocess	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   43: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   44: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.used-space-filewalker.subprocess	Database started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
	Ligne   45: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.used-space-filewalker.subprocess	used-space-filewalker started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
	Ligne   46: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker.subprocess	Database started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode"}
	Ligne   47: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker.subprocess	gc-filewalker started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "bloomFilterSize": 10215237, "Process": "storagenode", "createdBefore": "2024-05-30T19:59:59+02:00"}
	Ligne   48: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker.subprocess	Database started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
	Ligne   49: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.gc-filewalker.subprocess	gc-filewalker started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode", "createdBefore": "2024-06-09T19:59:59+02:00", "bloomFilterSize": 11917820}
	Ligne   50: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode", "dateBefore": "2024-06-09T21:57:04+02:00"}
	Ligne   52: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	Database started	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
	Ligne   53: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker completed	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
	Ligne   54: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess finished successfully	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne   57: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	starting subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne   58: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne   61: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "dateBefore": "2024-06-09T21:57:04+02:00"}
	Ligne   62: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	Database started	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode"}
	Ligne   63: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker completed	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
	Ligne   64: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess finished successfully	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne   67: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	starting subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne   68: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess started	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne   70: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker started	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "dateBefore": "2024-06-09T21:57:04+02:00", "Process": "storagenode"}
	Ligne   71: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	Database started	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Process": "storagenode"}
	Ligne   72: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker.subprocess	trash-filewalker completed	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "numKeysDeleted": 0, "Process": "storagenode", "bytesDeleted": 0}
	Ligne   73: 2024-06-16T21:57:04+02:00	INFO	lazyfilewalker.trash-cleanup-filewalker	subprocess finished successfully	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne  714: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.gc-filewalker	subprocess exited with status	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "status": 1, "error": "exit status 1"}
	Ligne  715: 2024-06-16T21:58:59+02:00	ERROR	pieces	lazyfilewalker failed	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkSatellitePiecesToTrash:160\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:558\n\tstorj.io/storj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
	Ligne  717: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.used-space-filewalker	subprocess exited with status	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "status": 1, "error": "exit status 1"}
	Ligne  718: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.gc-filewalker	subprocess exited with status	{"satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "status": 1, "error": "exit status 1"}
	Ligne  719: 2024-06-16T21:58:59+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne  720: 2024-06-16T21:58:59+02:00	ERROR	pieces	lazyfilewalker failed	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkSatellitePiecesToTrash:160\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:558\n\tstorj.io/storj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
	Ligne  726: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.used-space-filewalker	starting subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne  727: 2024-06-16T21:58:59+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
	Ligne  728: 2024-06-16T21:58:59+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne  729: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.used-space-filewalker	starting subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne  730: 2024-06-16T21:58:59+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "context canceled"}
	Ligne  731: 2024-06-16T21:58:59+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne  732: 2024-06-16T21:58:59+02:00	INFO	lazyfilewalker.used-space-filewalker	starting subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne  733: 2024-06-16T21:58:59+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
	Ligne  734: 2024-06-16T21:58:59+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne  742: 2024-06-16T22:00:18+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne 3231: 2024-06-21T03:05:53+02:00	ERROR	pieces:trash	emptying trash failed	{"error": "pieces error: lazyfilewalker: exit status 1", "errorVerbose": "pieces error: lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkCleanupTrash:187\n\tstorj.io/storj/storagenode/pieces.(*Store).EmptyTrash:419\n\tstorj.io/storj/storagenode/pieces.(*TrashChore).Run.func1.1:84\n\tstorj.io/common/sync2.(*Workplace).Start.func1:89"}
	Ligne 3235: 2024-06-21T03:05:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
	Ligne 3236: 2024-06-21T03:05:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne 3237: 2024-06-21T03:05:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "context canceled"}
	Ligne 3238: 2024-06-21T03:05:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne 3239: 2024-06-21T03:05:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
	Ligne 3240: 2024-06-21T03:05:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne 3243: 2024-06-21T03:05:56+02:00	ERROR	pieces	lazyfilewalker failed	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkSatellitePiecesToTrash:160\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:558\n\tstorj.io/storj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
	Ligne 3247: 2024-06-21T03:05:57+02:00	ERROR	pieces	lazyfilewalker failed	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkSatellitePiecesToTrash:160\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:558\n\tstorj.io/storj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
	Ligne 9153: 2024-06-29T22:47:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne 9154: 2024-06-29T22:47:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
	Ligne 9155: 2024-06-29T22:47:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne 9156: 2024-06-29T22:47:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "context canceled"}
	Ligne 9157: 2024-06-29T22:47:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne 9158: 2024-06-29T22:47:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
	Ligne 9159: 2024-06-29T22:47:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:704\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
	Ligne 9161: 2024-06-29T22:47:56+02:00	ERROR	pieces	lazyfilewalker failed	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkSatellitePiecesToTrash:160\n\tstorj.io/storj/storagenode/pieces.(*Store).WalkSatellitePiecesToTrash:558\n\tstorj.io/storj/storagenode/retain.(*Service).retainPieces:369\n\tstorj.io/storj/storagenode/retain.(*Service).Run.func2:258\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78"}
	Ligne 9941: 2024-07-01T17:59:55+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: exit status 1", "errorVerbose": "lazyfilewalker: exit status 1\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:85\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:707\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
	Ligne 9942: 2024-07-01T17:59:55+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
	Ligne 9943: 2024-07-01T17:59:55+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:707\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
	Ligne 9944: 2024-07-01T17:59:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "context canceled"}
	Ligne 9945: 2024-07-01T17:59:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:707\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
	Ligne 9946: 2024-07-01T17:59:56+02:00	ERROR	lazyfilewalker.used-space-filewalker	failed to start subprocess	{"satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
	Ligne 9947: 2024-07-01T17:59:56+02:00	ERROR	pieces	failed to lazywalk space used by satellite	{"error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:73\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:130\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:707\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:58\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:78", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}

So if the node can complete it’s filewalkers the disk space should match better. not perfect but better.

I’ve heard the “context canceled” can be cause simply by slow hardware. you’re running windows? What does task manager show as the disk busyness when storj starts up? What about cpu or ram being maxed out? (unlikely)

is the drive a SMR drive? Also unlikely.

Is the drive and file system working okay? Can you manually create a new file on it, etc?

Also you could try editing your config.yaml file, and setting lazy filewalker to false. This will run the “non lazy” file walker right away which might finish better.

2 Likes

F:\storagenode.log:90115620:2024-06-29T17:47:17+01:00 INFO lazyfilewalker.used-space-filewalker subprocess exited with
status {“satelliteID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “status”: 1, “error”: “exit status 1”}
F:\storagenode.log:95001439:2024-07-01T13:15:02+01:00 INFO lazyfilewalker.used-space-filewalker subprocess exited with
status {“satelliteID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “status”: 1, “error”: “exit status 1”}
F:\storagenode.log:95001442:2024-07-01T13:15:02+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “error”: “context canceled”}
F:\storagenode.log:95001445:2024-07-01T13:15:02+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “error”: “context canceled”}
F:\storagenode.log:95001448:2024-07-01T13:15:02+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “error”: “context canceled”}
F:\storagenode.log:95011466:2024-07-01T15:34:20+01:00 INFO lazyfilewalker.used-space-filewalker subprocess exited with
status {“satelliteID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “status”: 1, “error”: “exit status 1”}
F:\storagenode.log:95011469:2024-07-01T15:34:20+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “error”: “context canceled”}
F:\storagenode.log:95011472:2024-07-01T15:34:20+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “error”: “context canceled”}
F:\storagenode.log:95011475:2024-07-01T15:34:20+01:00 ERROR lazyfilewalker.used-space-filewalker failed to start
subprocess {“satelliteID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “error”: “context canceled”}
F:\storagenode.log:95034295:2024-07-01T16:04:41+01:00 ERROR orders cleaning DB archive {“error”: “ordersdb: database
is locked”, “errorVerbose”: “ordersdb: database is locked\n\tstorj.io/storj/storagenode/storagenodedb.(*ordersDB).Clean
Archive:325\n\tstorj.io/storj/storagenode/orders.(*Service).CleanArchive:162\n\tstorj.io/storj/storagenode/orders.(*Ser
vice).Run.func2:146\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolan
g.org/x/sync/errgroup.(*Group).Go.func1:78”}

Thank you for the answer.

It is windows, but the computer doesnt seem slow. Node is the only thing running on it

2024-07-01 21_08_39-192.168.1.12 - Connexion Bureau à distance

HDD is CMR. No problem to create a file

I will try disabling lazyfilewalker

yeah system seems fine except for the hard drive being maxed out 100%.

Check the HDD utilization when not running storj at all, make sure something else isn’t using it.

Also maybe reduce your node size to less than the current size, so that it’s not trying to ingress new data (increasing load) while also doing filewalkers.

if you can make it through the filewalkers successfully then you can restore your node size to the normal desired.

is it an USB hard drive or an internal?

You have an extra backslash in the last line in the docker run, it’s placed to allow to add another option after that on the new line. So

should be

--pieces.enable-lazy-filewalker=false

otherwise terminal would wait for the input instead of run. Or you would need to hit Enter one more time :slight_smile:
Since now the lazy mode is disabled, the used-space-filewalker will not report to the logs about a progress. So you need to wait until it would finish. You may use these methods to track the progress mentioned above. When it would finish for all satellites and update database, the usage should become close to the actual usage on the disk.

1 Like

These are old errors from 2024-06-29
The last one is

So, seems also the network is not functioning properly on this setup?

You need to perform the suggested actions in the linked threads, it should optimize the filesystem performance.
The Write cache policy is configured in the Disk Management - Your disk - Properties - Policies.

@Alexey now I have a proof that filewalker not help to resolve virtual trash problem or it Ended only to 1 sattelite?
I found that it started to next sat, so it like week long procest, and if pc restart then start from beggining?

2024-07-02T03:33:03+03:00 INFO lazyfilewalker.used-space-filewalker.subprocess used-space-filewalker completed {“satelliteID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Process”: “storagenode”, “piecesTotal”: 68149138688, “piecesContentSize”: 68006838016}
2024-07-02T03:33:03+03:00 INFO lazyfilewalker.used-space-filewalker subprocess finished successfully {“satelliteID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”}

Screenshot 2024-07-02 073217

Screenshot 2024-07-02 073235

in trash there is only 50GB files

You need to have all 4 scans to be completed. I only know that the bug with not updating stats by the trash filewalker is fixed only in 1.105.x, so one scan should be enough to fix usage discrepancies, and the next trash filewalkers should update the stat correctly - it should decrease it. However, I’m not sure how it would be processed, while the used-space-filewalker is still doing the scan and since it’s running too long, it likely would have the usage wrong anyway while the usage always changing.

I understood that now all I had to do was wait. This seems to be the most advanced step that a beginner can do. Thanks to you, I am learning a lot, but it is difficult to take on the challenge without some computer knowledge. I hope the day will come when you can just press Next like in Windows! :yum:

If any new problems arise, I will report them. thank you!

1 Like