Bloom dont work or why my trash is empty?

Look like increasing bloom filter size to 410003 is not enough for big old nodes:

Mar 30 05:12:07 50 storagenode[11975]: 2024-03-30T05:12:07Z        INFO        lazyfilewalker.gc-filewalker        starting subprocess        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
Mar 30 05:12:07 50 storagenode[11975]: 2024-03-30T05:12:07Z        INFO        lazyfilewalker.gc-filewalker        subprocess started        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
Mar 30 05:12:07 50 storagenode[11975]: 2024-03-30T05:12:07Z        INFO        lazyfilewalker.gc-filewalker.subprocess        Database started        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode"}
Mar 30 05:12:07 50 storagenode[11975]: 2024-03-30T05:12:07Z        INFO        lazyfilewalker.gc-filewalker.subprocess        gc-filewalker started        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "createdBefore": "2024-03-20T17:59:59Z", "bloomFilterSize": 4100003}
Mar 30 05:30:26 50 storagenode[11975]: 2024-03-30T05:30:26Z        INFO        lazyfilewalker.gc-filewalker.subprocess        gc-filewalker completed        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "piecesCount": 58447240, "piecesSkippedCount": 0}
Mar 30 05:30:27 50 storagenode[11975]: 2024-03-30T05:30:27Z        INFO        lazyfilewalker.gc-filewalker        subprocess finished successfully        {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
Mar 31 04:00:10 50 storagenode[11975]: 2024-03-31T04:00:10Z        INFO        lazyfilewalker.gc-filewalker        starting subprocess        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
Mar 31 04:00:10 50 storagenode[11975]: 2024-03-31T04:00:10Z        INFO        lazyfilewalker.gc-filewalker        subprocess started        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
Mar 31 04:00:10 50 storagenode[11975]: 2024-03-31T04:00:10Z        INFO        lazyfilewalker.gc-filewalker.subprocess        Database started        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode"}
Mar 31 04:00:10 50 storagenode[11975]: 2024-03-31T04:00:10Z        INFO        lazyfilewalker.gc-filewalker.subprocess        gc-filewalker started        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode", "createdBefore": "2024-03-26T17:59:59Z", "bloomFilterSize": 4100003}
Mar 31 04:07:19 50 storagenode[11975]: 2024-03-31T04:07:19Z        INFO        lazyfilewalker.gc-filewalker.subprocess        gc-filewalker completed        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode", "piecesCount": 10097877, "piecesSkippedCount": 0}
Mar 31 04:07:20 50 storagenode[11975]: 2024-03-31T04:07:20Z        INFO        lazyfilewalker.gc-filewalker        subprocess finished successfully        {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}

For now delta between paid and unpaid storage only on one node is about 5Tb :face_with_symbols_over_mouth:

Total count of pieces stored on this node:

find /opt/storj/data/storage/blobs/ -type f -name "*.sj1" | wc -l
76451028

1 Like