Like topic title I now seeing my node continue delete/moving data to trash more than a day and still continue doing it, not sure this is a normal behavior
Node is in version 1.101.3
Yes, deleting data is normal.
3 Likes
yeah but constantly delete over 2 days now …
I don’t remember seeing this happen before
To remove data from your disk, the node needs to check every piece to determine if it should be deleted or kept. If you have a slow hard drive, lots of capacity, using lazyfilewalker, not enough RAM to cache metadata, etc. it can be a long time to do this.
1 Like
Deletion is now performed by an expiration chore (collector) and trash chore, not by customers directly. The deleted data is gathered by a garbage collector (it’s invoked by a Bloom Filter sent from the each satellite) and sent to the trash first, then it will be permanently removed after 7 days. So, yes, removing can take a lot of time on slow disks.
2 Likes
Thanks for your information
2 Likes
Hi Alexey, I see a problem with this approach now
So after 7 days, expired data are now being remove from the node but also seeing another round of data moving to trash.
Combine this 2 things at the same time cause the disk in heavy stress
1 Like
Data deleting is completely normal? you don’t delete your data? However in this case this may come from the saltlake sat, which is test data.
I understand data delete is normal, but the way storj is implement I don’t think good for disks
When they will have to handle both actual data deletion from disk and continue moving data to trash
Never seeing this behavior before
It’s like a backup-plan for them. Sometimes they implement things in the wrong way which makes good data to be deleted, so a “recycle bin” is necessary to recover the data in case of wrong deletion.
1 Like
Yeah but actual delete data from disk and moving data to trash should not happen at same time ?
They are async processes not dependent on each other. The trash removal is happening every 24h since the node start, the garbage collection - since the last Bloom Filter received from the satellites.
So you may place them in a different time interval by the simple restart.
Is there a log/setting that indicate trash data remove from disk?
I just know this happen when I seeing my free disk storage increase
Yes, all log entries for each filewalker/chore you can see there:
However, some entries could be outdated (I didn’t update the article after the latest change yet), I would appreciate if your could fix them, I made it a wiki.
Not seeing any log relate to “used-space-filewalker” but see “trash-cleanup-filewalker” so update your note with that
check your log level in config?
used-space-filewalker is running only on start, if you didn’t disable it.
1 Like
make sense, I’m not disable it
also, Is there any public link to sample config.yaml that contain all the setting with explain what setting will do?
No and unlikely will be, it’s constantly changes, we cannot keep it updated so it will be always outdated.
But you may run this command to get all parameters and their defaults:
docker
docker exec -it storagenode ./storagenode setup --help
Please note, in the docker setup you will pass most of needed parameters directly in your docker run command, not in the config.yaml file, because command line arguments have a precedence above parameters in the config.yaml file.
PowerShell and Windows GUI:
& "$env:ProgramFiles\Storj\Storage Node\storagenode.exe" setup --help
You may also take a look on (but it’s likely outdated already):
1 Like
Yep, it’s a wiki, but I can’t update it on every version, only when I start a new node, which is rare, or start a new node just to get the config, and remove it, which I don’t do, or when others contribute.
1 Like