Size to do disk space accounting (updating trash size, so that you have a shiny number on your dashboard).
Modification time to avoid running bloom filter on pieces that were uploaded after the bloom filter was created. Modification time is the only timestamp that is consistently preserved across OSes and file systems, so it’s not possible to depend on, let say, ctime.
I have updated the config wiki (Config.yaml - Linux/Docker all parameters) with the new version. Can someone explain a couple of things for the new logging changes?
…And that cache for slow disks/low mem.
I hope those log changes won’t start filling my log like crazy and have to switch it to FATAL again. Now I run custom log info mode, with FATAL only on piecestore and collector.
For honestly, I do not see a reason to reduce the log level for the collector
$ grep -c "collector" /mnt/y/storagenode3/storagenode.log
93
A problem with 107 logging millions of “warn collector” entries.
Take a look at the code change. It isn’t that big of a deal. This is more like a quick idea to try out. From the point of view of a developer there is a list of possible solutions and it makes sense to start with everything that is cheap to implement. If that proves to be inefficient then go on with the next idea. Sure this doesn’t eliminate the downsides but it justifies spending the required resources on the more expensive solutions down the road that you would only consider if everything else didn’t work.
We are mixing the topics a bit. The used space file walker needs the size but not the mod time. Garbage collection needs the mod time but the size only for the pieces it moves in the trash folder. The mod time is needed because by the time your node is processing the bloom filter it will have stored a few more pieces that haven’t been around while the bloom filter was generated. These pieces will be a negative match against the bloom filter. So the following if statement checks the age of the piece. Is it a new piece then keep. If it is an old piece it is safe to move into the trash folder.
Is it possible to make that SLC satellite deletes will not go to trash but deleted directly, there is no concerns do loose test data, it is even with TTL.
I dont think it is good on other satellite only SLC.
By this we hold trash more clean, give clients more space to use.
I get the following errors with version v.1108 on ALL upgraded nodes:
{L:ERROR,T:2024-07-11T15:34:51Z,N:piecestore,M:upload internal error,Process:storagenode,error:pieceexpirationdb: context canceled,errorVerbose:pieceexpirationdb: context canceled
storj.io/storj/storagenode/storagenodedb.(pieceExpirationDB).SetExpiration:115
storj.io/storj/storagenode/pieces.(Store).SetExpiration:588
storj.io/storj/storagenode/piecestore.(Endpoint).Upload.func6:497
storj.io/storj/storagenode/piecestore.(Endpoint).Upload:535
storj.io/common/pb.DRPCPiecestoreDescription.Method.func1:294
storj.io/drpc/drpcmux.(Mux).HandleRPC:33
storj.io/common/rpc/rpctracing.(Handler).HandleRPC:61
storj.io/common/experiment.(Handler).HandleRPC:42
storj.io/drpc/drpcserver.(Server).handleRPC:167
storj.io/drpc/drpcserver.(Server).ServeOne:109
storj.io/drpc/drpcserver.(Server).Serve.func2:157
storj.io/drpc/drpcctx.(*Tracker).track:35}
These errors do not occur on old nodes (1.105.4).
Should i be worried?
I cannot confirm. I have several nodes that were already upgraded on 1.108.3 but all of them are working fine.
After updating all nodes to 1.108 it started to show more occupied space and less garbage than before. but it also can related to restart of all nodes
We have to run one used space scan after upgrading to correct the numbers on the dashboard.
I runed before this 1.107 for several days, is it make difference?
how the hell are you already running 1.108 I still got 1.105 and I hate it cause my disk is always at 100% cause the filewalker isnt doing his job
If you finished the used space filewalker with 1.107 that should be fine as well. If you did not run the used space filewalker it would have carried over the incorrect values from the previous release. So one run somewhere after upgrading to 107 or 108 should do the trick.
Because version.storj.io show it as last version, and i have my own updater software.
you on windows? I use toolbox for windows but it thinks 1.105 is up to date 
Toolbox not update nodes at all. first node updated by original storj updater it takes time when it will update. all other you need update manualy.
The new version isn’t going to change that. At best you can stop running the used space filewalker. You still need to run it once to correct the numbers but if we are lucky we can turn it off and forget about it for a few weeks maybe months.
But I guess the process that is currently active on your node is garbage collection. That one will be same costs with the new version.
There is an experimental cache you could try out. It is currently more or less untested so maybe a bit too early to try that on a bigger node.
But I saw some people said 1.107 fixed the filewalker. oh well