Question about limit display europe-west

I just received my invite and tried to upload and download a 2 GB file. working well :slight_smile:

Now, my account shows 6GB of the 25GB limit used, which is triple of what I did. Why is that?

Is it also normal that a remove of an object takes very long? It has been churning on that remove for a few minutes now…

I believe that has to do with the redundancy. It’s 2 GB uploaded to 3 nodes, for instance.

@Knowledge that was my conjecture too, storage is 300% from raw data using erasure coding if I remember correctly.

Does this also mean that we need to pay 300% from the announced prices for storage and egress in production?

Not 100% sure, but I believe so, yes. And yes, I realize that makes Storj three times more expensive on the storage side. It is something that may need to be adjusted as we get closer to production.

I think it test account limitations, as i remember when client upload file, hi not pay anything, when you download it back then you pay for trafic. But you downoad only pieces for 1 file size, may be small overhead.

@Knowledge if 3 times more expensive than advertised it is also more expensive than AWS, that is not what I understood from reading through all the docs…

1 Like

Yep, I’m aware of that. Just saying how it currently operates.

@Knowledge
I strongly hope that this will be fixed for production, it does not match with the advertisements and it will not play well with the SNOs either

1 Like

Just a quick clarification - we charge for data as uploaded to Tardigrade users and pay Storage Node Operators for data as stored.

For example, a user uploads a 1 GB file and stores it for a month. That user is charged for the storage of 1 GB.

That data is stored on the network as 2.76 GB including the erasure code overhead. That 1 GB file is broken up into sixteen 64MB segments. Each segment is broken up into eighty 2.2 MB pieces. The pieces are spread over 1280 statistically uncorrelated storage nodes. Those Storage node operators are paid for storing that data for that month.

4 Likes

@john thanks for the clarification. But how come in the test phase if I upload 2GB the limit is charged with 6GB instead of 2GB?

I have an explanation here. One moment

@champmine18 I’ve been working on some refactors regarding our live-accounting system (how we make sure you don’t exceed your free storage limit :slight_smile: )

I noticed an issue in how we calculate current project totals for display. To be brief, the gist of it is that our live accounting system was a temporary cache for keeping uploaded amounts. These were expanded values (2GB -> 6GB) whereas our actual project accounting used for invoicing does not.

If you check the console again in a bit (it might actually already have changed by now) I believe you should see it drop back to 2GB (if you haven’t made any new uploads). We’ll have this discrepancy fixed soon

Thanks for posting!

5 Likes

@cameron thanks for the explanation, storage has dropped to 0 (I deleted the file). But egress still remains at 6GB in the overview.

If I go to the detailed report, egress shows 2.4 GB which is also wrong. I just uploaded the 2 GB file, then downloaded it, and then deleted it.

Hmm, it looks like you might have just brought some more discrepancies to our attention. From poking around it looks like that 6GB figure you’re still seeing for egress could be your allocated bandwidth for the download, which is not necessarily the amount used for the download. I’ve started some conversations with the team working on the UI to see what’s going on here.

@cameron thanks for digging into this, please also let the team know that the detailed report is off, it shows 2.4 GB egress when I only downloaded the one 2 GB file once.

Sounds very close to 35/29 times the file size. Could this be because it downloads 35 pieces while 29 are needed to create the file?

@BrightSilence You’re messing things up. You are correct that this is fitting in your formular, but 35 is the repair threshold. Only when a piece is failing under this threshold the satellite will repair it by downloading these 35 pieces, but this will not shown to the customer.

A download with default RS settings will start downloading from 47 nodes (29 + 18 extra). https://github.com/storj/uplink/blob/2ae87fb1a7ffc760d4fd3a7f756a5e116e79a40f/storage/segments/store.go#L171 calculated here.

1 Like

@BlackDuck thanks for the clarification. Still egress 6 and detailed report 2.4 is wrong then in the UI / accounting

I already filed a ticket a bit ago. It is in the queue for the next sprint. I am sure that the 6 is totally wrong, with the 2.4 it is more complicated. The transfer amount could be correct in case of long tail canceling, but I am not sure it that is planned to account. We have to wait a bit.

@BlackDuck yes sure, long tail canceling figures. But this should not be paid by the tardigrade customer. I think it is important to actually do the accounting in line with the advertisements