Bandwidth cost is increasing very quickly

I tried to setup a demo account to test the Storj DCS, and uploaded 3 Videos ( ~80 MBs each ) of total combined size of ~300MBs, while testing I noticed just watching each Video once or twice, it’s adding more than 1GBs in my Bandwidth usage and increasing my cost on single view significantly.

How it this possible, and what can I do to optimise it? Could using a CDN work to optimise Bandwidth usage?

1 Like

That seems excessive, but will be affected by a few factors. The two which I’ve experienced are:

  • The bandwidth shown is the ‘max’ usage. Over the next 72 hours the usage will reduce to be just the amount actually downloaded. This is due to the decentralised nature of the network, the nodes supplying the data only report usage rollups to the satellites on a schedule.

  • Some video codecs are not good for streaming in certain browsers. The buffering and loading process can be very wasteful with bandwidth. This is harder to pin down but you should be able to check this using a browser’s developer tools.

You could definitely look at Fastly as CDN, with StorJ as backend storage.

2 Likes

I don’t think first one is the reason will try to second option, as you can see my final billing shows 9 GBs of Egress generated from 300MBs

Hi @im_ayushtiwari! Could you clarify how these files are being accessed? Are you using uplink CLI, libuplink or linksharing?

1 Like

I’ve setup a test account so right now it is only Link Sharing

Does it makes any difference on Egress ? What should we choose while setting this up with our native application ?

@im_ayushtiwari It should not make any difference. We are just gathering more information for further investigation.

So I found one thing during experimenting, Mp4 format is what makes the egress to increase significantly, while I tried webM, it works fine

1 Like

We figured it out, it’s the front end. When uploading a file via the object browser you can select to see details which loads the preview. If you do this the edge service that build the preview downloads the file, download # 1. If you then click to create a share the preview comes up again, that’s download # 2. Finally, you open the share link in another tab, that’s download # 3.

How can we do better when using the satellite GUI? We can implement caching so these repetitive downloads do not occur. Tickets have been created and we will sort this soon.

How can a user avoid this? You could use the CLI (Uplink) to create both the Linkshare and upload the file. In addition when streaming the video changing the share url from /s to /raw will directly download the file.

Normal Linkshare with map and preview
https://link.eu1.storjshare.io/s/jwwlss3g44lwvp5etxkxcn6oo6ya/newdemo/Big%20Buck%20Bunny%20Demo.mp4’

Direct Download
https://link.eu1.storjshare.io/raw/jwwlss3g44lwvp5etxkxcn6oo6ya/newdemo/Big%20Buck%20Bunny%20Demo.mp4’

When using the CLI tools you will not load previews and thus not incur unexpected egress bandwidth.

6 Likes

Got it but if this is the case why is there a bandwidth usage difference between two different file extension, as I mentioned MP4 consumed more bandwidth than Webm

Great question, likely the browser is not pre-loading the Webm content behind the preview. Behaviors via different browsers may differ. The team is continuing to test. When I have more updates I’ll be sure to post here.

Thank you!

3 Likes