Drive is filled, but storj only reports 75% filled

I have 5 TB HD, this drive is 100% dedicated to storj and I have nothing on it except storj. In fact, this is a VM on my server that is only used for storj.

Storj is reporting that I have consumed about 3 TB of the drive, but my OS is reporting that the drive is 95% full. My node just went offline for an extended period of time because my OS is reporting that the drive is full and the service had nowhere to write to.

Is a bug that can cause this or any troubleshooting tips?

That’s is normal. My 8TB harddrive only has 7.3TB after formatting.
Read this for more information on harddrive advertisement number Why Advertised Storage Does Not Match Real Data Capacity

My apologies, I meant to say 5 TB, not 4 TB. I corrected this in my post.

Thanks for the info. I believe I already accounted for this for my 5 TB drive because I should have only allowed 4.4 TiB of space to be consumed and I believe 5 TB is ~4.54 TiB.


The above image is my drive dedicated to storj. The 4.5T is the maxed spaced and 4.3T is consumed. Albeit, I probably shouldn’t try to fill up my whole drive and leave some ‘working space’

Had to break this into two posts, sorry!
image

I leave about 50GB left on my harddrive in case I get a large spike in traffic.
Note that your database will likely get corrupted if you run out of space when it is bringing modified.

Do you think that is what happened here? Is there any way to fix it if it has? Because if you look at my images, there is a definite issue here with spacing being reported by OS or Storj

cd into your storj001, and run sudo du . -ch -d1
This will list out all directory and how much it is using. You might have other files or folder inside the storj001

I’m currently trying to index using duc right now. du was very slow, my du and duc process is being met with IO block I think because the state of the process keeps reporting back to me as D in top.

Hello @blobbert,
Welcome to the forum!

Please use --apparent-size option in your du too - it will show the physically used space. It can be significantly different from the logical size, because of used filesystem.
For example, exFAT uses more physical space for each block because it has a large cluster size, and every small file will take the whole cluster. The same can be true for improperly configured zfs pool.

Just to give an update, I am still waiting for the commands to finish running.

The commands are still running, do you know the typical run time for this?

Not exactly correct. It has almost exactly 8TB, which is 7.3TiB. they are different units. Unfortunately most OS’s display TiB units but wrongfully use the TB naming. The article you linked neglected to even mention this unit difference. Storj displays and accounts using TB, but you can set your space with TB or TiB as long as you use the correct unit.

For du -h displays TiB units --si displays TB units.

2 Likes