Release preparation v1.45

This release is already deployed to satellite and binaries uploaded on GitHub.

Here the changelog:

Satellite

  • [+ 1d14e4ef] satellite/metainfo: Check ctx in download Object & Segment
  • [+ de8464e8] satellite/metabase: add GetProjectSegmentCount method
  • [+ 2c0a360a] satellite/satellitedb: migrate partner_id db column to user_agent db column
  • [+ 99deec47] satellite/metainfo: consistent error message for bucket not found
  • [+ 9c1129b4] Revert “satellite/satellitedb: migrate partner_id db column to user_agent db column”
  • [+ 984792fd] satellite/satellitedb: Add GetByEmailWithUnverified to users table
  • [+ bae8276f] satellite/admin/ui: Add missing geofencing endpoints
  • [+ bde8fb0a] satellite/admin: Move geofence endpoint to bucket info
  • [+ 16c8a080] web/satellite: new project dashboard routing updates
  • [+ b78f65e8] satellite/console, web/satellite: added object and segment count per project info to new project dashboard
  • [+ be10ce84] web/satellite: implemented charts UI for new project dashboard
  • [+ 82fb0fce] web/satellite: added charts date range picker for new project dashboard
  • [+ b3cea3d1] satellite/audit: account for piece size during audit reservoir sampling
  • [+ 73730b23] satellite/satellitedb: add segment_limit colum to projects table
  • [+ 5013dd8a] satellite/metainfo: remove error message prefix
  • [+ 3b51eea3] satellite/metainfo: store empty useragent in bucket attribution instead of throwing error

Uplink

  • [+ eec5ad4f] cmd/uplinkng: access inspect command added
  • [+ b5194762] cmd/uplinkng: share command added
3 Likes

What “segments” are we talking about?

Previously, customers had to pay a fee per stored file, and it’s been removed as part of the latest change in your pricing to make things cheaper and even easier to understand (with no hidden fees).

Is that a step back?

Large files get split up into segments (IIRC by default it’s 64MB).

1 Like

i suppose somebody could upload billions of segments which might be bad for the network, my guess would be it’s to ward against malicious, ignorance and mistaken use of some kind.

so they are limiting paid users to errr… max 64Terabyte of files? i guess that seems fair, atleast without any sort of notice.

2 Likes

That’s what I thought but I wasn’t sure it was about those same segments.
Well then it’s even worse? Counting files is easy and makes sense to users, but counting segments? Segments are a technical thing behind the scene, how is a customer supposed to know how many segments their files count?

I don’t see why billions of files would be bad, and in fact many projects have in immense number of files in the development world, because of wonderful things line NodeJS for instance…

What I would be afraid of would be the use of file names only for storing data for instance: that would be a way to abuse the system, I mentioned it long ago already:

And that’s why initially Storj charged “$0.0000022 Per Object Per Month”.
But they decided to remove it later on!

1 Like

I think this is key.
I think it is of dubious consumer protection legality to charge an amount for a product which is arbitrary and where the cost is not clear enough.
I can decide how may objects or files to store, I have no control (or indeed knowledge) of the number of segments.

I’m sure there is a good reason for this change, but I too am struggling to see what it could be.

2 Likes

The segment size is not fixed. We have customers that upload with default 5MB multipart size. That will create 10 times more segments for the same file size. There is also one person that has chosen to upload a segment size that is even lower in order to game our system. The segment fee is there as an incentive to use 64MB segments whenever possible. We are talking about a very low fee. Let us come back with some numbers for you later.

3 Likes

Well then I think what would be a better move would be to have 0 segment fees for customers who choose to keep segments above a certain size (>=16MB for instance), and make pay only customers who choose a lower value?

Why would one want a very small segment size by the way?

2 Likes

It should be close to 0. I am currently trying to get some numbers about the the amount we would charge in different situations.

I like that kind of an idea. Why not 0 fee for those who don’t game the system and use the default 64 MB size for their data? And then surcharge those who upload with different sizes.

1 Like

I have an update for you. The segment fee is $0.0000088 per Segment-Month. That means an additional 14-15 Cent per TB with perfect file sizes. Maybe 20 Cent per TB if we throw typical file sizes at it.

I am a bit concerned about this pricing increase. If one of you has a paid account maybe take a look at how many segments you are currently storing and calculate how expensive that would be for you. It would be awesome to get some real-world examples to understand the impact.

5 Likes

This is what my EU1 account shows:

I have never knowingly chosen a segment size. The billing page shows:

I have been aware of the per segment cost but never assumed it would affect my minimal use. It is mentioned on the main pricing page (and has been for some time)…

4 Likes

108 Segments should be less than 1 Cent.

1 Like

My largest bucket currently has about 777gb in it and has 11,731 segments. That works out to 0.1 cents.

2 Likes

Sounds like I was more concerned than necessary. It looks like the segment fee is low enough to not hurt but also high enough to be a meaningful incentive.

Thank you all for your feedback.

5 Likes

so if we do the math in reverse.

1mil max segments, at 0.0000088 pr segment pr month.
so that becomes 8.8$ max extra cost pr month.

and 1 segment being 64MB so thats when storing 64TB or in the case of 5MB it would then be 5TB obviously.

seems reasonable, and if we then compare it to what the data storage itself would be for 5 and 64TB.

5 TB would be 20$ + 8.8$ + bandwidth or 64 would be 256$ + the 8.8$ for segments and the bandwidth on top.

meaning at peak its less than 5% extra costs while at 5MB segment size it would be near 50% of the total storage cost.
so yeah it would certainly work as an incentive to configure it correctly for large scale usage.
unless if one has certain benefits from running smaller segments which one likes to pay for…

like say the ability to scrub video files more accurately or something… duno how it works but i would assume a smaller segment size would allow for a more accurate data retrieval process.
for seek in videos or such, ofc making the assumption that people download per segment.

yeah seems a very reasonable incentive.

ofc one might want to review the implications and pricings of the segments on a yearly basis or whenever other prices change, so that the incentive remains, as prices for storage change.

3 Likes

The new project dashboard is currently disabled on the QA satellite. If you want to get an early demo you can still reach it with QA-Satellite

Be aware this early demo is showing mostly mock data.

2 Likes

Alright, faire enough.
As long as StorjLabs make sure customers don’t discover overnight prices changed and get noticed beforehand, I guess that’s fine.

But whatever a customer’s activity/mileage/mindset, it’s rarely good news when a service’s prices go up, so I hope StorjLabs will know how to introduce that smoothly, with tact :slight_smile:

1 Like