# Disk usage discrepancy? **URL:** \ **Category:** troubleshooting\ **Tags:** xfs, btrfs, ntfs, storage, zfs\ **Created:** [December 19, 2023, 6:18pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715 "2023-12-19T18:18:00Z")\ **Posts on this page:** 20\ **Page:** 11
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 18, 2024, 3:58am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/237 "2024-01-18T03:58:58Z")
No. They will affect each other and you will have even higher discrepancy, but now because of concurrency of multiple nodes for the single resource - your disk. The fix likely would be implemented before you fill 22TB. ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 18, 2024, 4:02am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/238 "2024-01-18T04:02:31Z")
With disabled lazy filewalker? How many weeks? ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 18, 2024, 4:09am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/239 "2024-01-18T04:09:37Z")
No, the `forget-satellite` with a `--force` flag will delete data right away. If you skipped the `--force` flag, data may remain and will not be removed and you likely would need to remove it manually, see [Satellite info (Address, ID, Blobs folder, Hex)](https://forum.storj.io/t/satellite-info-address-id-blobs-folder-hex/17183) for folder names of the satellites in the blobs folder. If you used a `--force` flag, but still has a discrepancy, then I suggest to follow this recommendation: > [@Disk usage discrepancy?](https://forum.storj.io/t/disk-usage-discrepancy/24715/100): > > They will show up there, these files are however stored in RAM then swap in case of tmpfs. and make sure that the filewalker on start is not disabled (it’s enabled by default). ---
**Author:** ![Ottetal](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ottetal/32/18876_2.png) [@Ottetal](https://forum.storj.io/u/Ottetal)\ **Post date:** [January 18, 2024, 7:19am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/240 "2024-01-18T07:19:28Z")
Hiya friend. I’ve never made it on location - so without disabled lazy file walker, but for 9 days. It happened the night after v1.94.2, so it must have run during reset. ---
**Author:** ![Ottetal](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ottetal/32/18876_2.png) [@Ottetal](https://forum.storj.io/u/Ottetal)\ **Post date:** [January 18, 2024, 2:32pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/241 "2024-01-18T14:32:05Z")
Update on the same node: It how has additional 200GB worth of stuff to delete. Truly exiting ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/7/76bce90300db76c661dcadcdd217afa50a0b9ca3.png) ---
**Author:** ![HGPlays](https://storj.bcdn.literatehosting.com/letter_avatar/hgplays/32/5_5575768a8748004e209b776fc1b2916d.png) [@HGPlays](https://forum.storj.io/u/HGPlays)\ **Post date:** [January 19, 2024, 10:01am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/242 "2024-01-19T10:01:14Z")
I did run with --force: ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/7/79a7451e3b61ac2fa9a60df4ab9c92890f3f9a98.jpeg) Yet i still have the same issue: ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/a/ad018ed74b593aa91d44fb352efb842343b46d10.png) ---
**Author:** ![daki82](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/daki82/32/1128_2.png) [@daki82](https://forum.storj.io/u/daki82)\ **Post date:** [January 19, 2024, 9:02pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/243 "2024-01-19T21:02:22Z")
it needs some time to delete. let the window open until there are only 4 blob folders left? ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 20, 2024, 1:17am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/244 "2024-01-20T01:17:36Z")
> [@HGPlays](#): > > I did run with --force: > > ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/7/79a7451e3b61ac2fa9a60df4ab9c92890f3f9a98.jpeg) Accordingly the sign `>>`, your command either have a missing quote somewhere or you used not straight ones `"` but curly ones (`“` and `”`) or you pressed a Shift-Enter. Because such sign meaning that the command is waiting for input ---
**Author:** ![daki82](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/daki82/32/1128_2.png) [@daki82](https://forum.storj.io/u/daki82)\ **Post date:** [January 20, 2024, 6:49am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/245 "2024-01-20T06:49:49Z")
> [@How To Forget Untrusted Satellites](https://forum.storj.io/t/how-to-forget-untrusted-satellites/23821/57): > > So here is my line that did not retturn an error, as i had still blobs folders of the decomissioned test sats left. syntax sample windows GUI, powershell, admin rights, node running, cd to folder where the storagenode.exe is. with cd “C:\Program Files\Storj\Storage Node” and: .\storagenode.exe forget-satellite --force 12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB 12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo --config-dir “C:\Program Files\Storj\Storage Node” --identity-dir “C:… Refer to this windows sample, **move to the storagenode dir at first** with the cd command ---
**Author:** ![Windows7ge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/windows7ge/32/1128_2.png) [@Windows7ge](https://forum.storj.io/u/Windows7ge)\ **Post date:** [January 20, 2024, 3:06pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/246 "2024-01-20T15:06:28Z")
My node has recently started going offline without warning. I’ve started digging into it and see my ZFS array has been running a scrub for several days which might be the problem. While observing that though ZFS reports that my 40TB raidz1 (25.6TiB usable) is all out of space. But in Storj Docker I only allotted it 24TB. And in addition to that Storj reports that it’s only using 21.69TB in the WebUI. I remember reading we want to keep something like 10% free space. So I’m confused. The scrub could be causing Storj to hang, maybe the array being full is causing it to hang. But then why is the array full? I don’t keep anything else on it. It’s dedicated storage. As a temporary fix I could expand the array another 40TB and observe the behavior but I’m just confused. ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 20, 2024, 4:35pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/247 "2024-01-20T16:35:57Z")
you may reduce the allocation below the usage and disable a lazy filewalker to give it a normal priority. Perhaps it would be able to finish its work before crash/restart. ---
**Author:** ![Windows7ge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/windows7ge/32/1128_2.png) [@Windows7ge](https://forum.storj.io/u/Windows7ge)\ **Post date:** [January 20, 2024, 4:57pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/248 "2024-01-20T16:57:12Z")
I don’t understand most of what you said. If I told Storj not to use more than 24TB, it reports it’s using 21.69TB, but the file system reports it’s using 25.6TiB I have no way to measure how low I need to set Storj to get it back under control. 20TB? 21TB? 22TB? Based on what I’ve read even doing this will just prevent it from taking in more data. The only way data will shrink on the node is if the network removed data no longer needed which will taken an indeterminable amount of time. Meanwhile will that stop the crashes? I have to do manual restarts every time I get an e-mail saying my node is offline. And what is a filewalker? ---
**Author:** ![Windows7ge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/windows7ge/32/1128_2.png) [@Windows7ge](https://forum.storj.io/u/Windows7ge)\ **Post date:** [January 20, 2024, 7:41pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/249 "2024-01-20T19:41:01Z")
I just picked an arbitrary value of 20TB down frim 24TB and I’ll monitor the free space on the array. 6.6GB of free space which I’d expect to see maybe in the few 100’s of GB. After the update the WebUI now reports overused space. ![Screenshot from 2024-01-20 14-38-47](https://storj-s3.bcdn.literatehosting.net/original/2X/3/391bc098851b32b9fe41a7aa122fe3ebb75d6167.png) I’ll monitor if this slowly starts to shrink over the coming weeks and hope this plus after my ZFS scrub finishes that the crashes stop. ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 21, 2024, 3:54am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/250 "2024-01-21T03:54:54Z")
> [@Windows7ge](#): > > I have no way to measure how low I need to set Storj to get it back under control. 20TB? 21TB? 22TB? doesn’t matter, but it should be less or equal of used space (accordingly Average Disk Used Space on the left graph), this will prevent the node to receive more ingress, freeing up some IOPS to the filewalkers. We call “filewalker” the process of scanning pieces for different purposes: - scan to calculate a used space; - scan to move a garbage to the trash; - scan to delete expired pieces; - scan to delete pieces from the trash. ---
**Author:** ![HGPlays](https://storj.bcdn.literatehosting.com/letter_avatar/hgplays/32/5_5575768a8748004e209b776fc1b2916d.png) [@HGPlays](https://forum.storj.io/u/HGPlays)\ **Post date:** [January 22, 2024, 10:54pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/251 "2024-01-22T22:54:18Z")
i gave it another shot and it lookes better - but i still have the same issue with alot of used space i cant get rid of. ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/c/c470a4d8702bab2fbfb06ede83b3614d8a83aa26.png) ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [January 23, 2024, 3:13am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/252 "2024-01-23T03:13:06Z")
Now your node should finish several filewalkers without interruption from each remained satellites. ---
**Author:** ![HGPlays](https://storj.bcdn.literatehosting.com/letter_avatar/hgplays/32/5_5575768a8748004e209b776fc1b2916d.png) [@HGPlays](https://forum.storj.io/u/HGPlays)\ **Post date:** [January 23, 2024, 8:50am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/253 "2024-01-23T08:50:53Z")
Thanks - i will keep an eye on it 🙂 ---
**Author:** ![mcanto73](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mcanto73/32/1133_2.png) [@mcanto73](https://forum.storj.io/u/mcanto73)\ **Post date:** [January 31, 2024, 6:20pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/254 "2024-01-31T18:20:04Z")
Hello, I have 5 nodes running from almost 2 years in 5 linux servers and I noticed that what reported by “df” was very different compared the value I have in dashboard / graphana tools. The difference was more than 4 TB in some cases. I saw a fix and I tried to apply : \<\< Stop the storagenode Rename the piece\_spaced\_used.db Execute with sqlite3: sqlite3 F:\StorjShareV3\piece\_spaced\_used.db When you see a sqlite\> prompt, execute script: CREATE TABLE versions (version int, commited\_at text); CREATE TABLE piece\_space\_used ( total INTEGER NOT NULL DEFAULT 0, content\_size INTEGER NOT NULL, satellite\_id BLOB ); CREATE UNIQUE INDEX idx\_piece\_space\_used\_satellite\_id ON piece\_space\_used(satellite\_id); .exit Start the storagenode > > The problem is that now , it’s running since a lot of hours , I have very low values in disk space, for instance : ubuntu@hpool2:~$ df -h | grep STORJ [mcanto.ddns.net](http://mcanto.ddns.net):/STORJ3/STORJ\_NFS2 14T 12T 1.8T 87% /STORJ before the fix dashboard reported 9 TB Now : Uptime 8h25m35s ``` Available Used Egress Ingress Bandwidth N/A 2.47 TB 531.55 GB 1.94 TB (since Jan 1) Disk 9.98 TB 16.63 GB ``` Any idea ? Could I put back the old piece\_spaced\_used.db file also if it’s OLD (less than 1 day)? Best regards ---
**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\ **Post date:** [February 1, 2024, 4:20am UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/255 "2024-02-01T04:20:15Z")
1. NFS is not supported 2. The discrepancy is usually related to a failed filewalker You need to search in your logs errors related to `walk`, `retain` and `FATAL` The possible workaround could be to run a filewalker with a normal priority by disabling the lazy one with these parameters in your `config.yaml`: > [@jeremyfritzen](#): > > ```auto > storage2.piece-scan-on-startup: true > pieces.enable-lazy-filewalker: false > > ``` save the config and restart the node. ---
**Author:** ![naxbc](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/naxbc/32/1125_2.png) [@naxbc](https://forum.storj.io/u/naxbc)\ **Post date:** [February 2, 2024, 6:02pm UTC](https://forum.storj.io/t/disk-usage-discrepancy/24715/256 "2024-02-02T18:02:03Z")
Hi @Alexey just added “pieces.enable-lazy-filewalker: false” as I´m having the same issue reported: ![image](https://storj-s3.bcdn.literatehosting.net/original/2X/1/1ca31ef7533d3f5a9ee71d4a798c787c9f517f13.png) Any more advice? How long does it take to reflect on Dashboard? [Previous page](https://forum.storj.io/t/disk-usage-discrepancy/24715.md?page=10) [Next page](https://forum.storj.io/t/disk-usage-discrepancy/24715.md?page=12)