Here is a file sharing app I built, what could I be doing better with my Storj integration?

Hi all,

I wanted to share an app that I’ve recently started to launch and ask for some feedback about how I’m using Storj.

So, for context, I have an open source video player that is designed for esports coaches to use when reviewing game footage with their team (called “Vodon Pro” - you can read about it here if you’re interested: GitHub - Rodeoclash/vodon-pro: Vodon Pro is a video player designed for esports coaches to review footage of players.)

One of the common problems that the coaches and players have is transferring the footage that’s been recorded on the players computer to the coaches machine so that they can load it into Vodon Pro. This are quite often ~4gig video files as minimal compression is performed on them to ensure that the coach is working with the maximum amount of detail (they’re usually 60fps+ framerates to boot).

To solve this, I put together a small app called FileYeet which allows the coaches (and anyone really) to easily create portals, a.k.a a website page, which can be used to upload files (https://www.fileyeet.io/ if anyone is interested :upside_down_face:)

The site is built with Elixir / Phoenix and a small amount of React for the file upload interface. To integrate with Storj I’ve leveraged an AWS S3 library which seemed to play quite nicely with Storj’s S3 compatibility layer. However! After browsing around the forum a bit, it seems that I’m leaving quite a bit of functionality on the table by using this compatibility layer. Given my use case (generating signed URLs so that the uses can upload and download) what am I missing out on with this approach? Does anything spring to mind for people that would be a better solution (and can you link me to the docs about it :laughing:)

Finally, a couple of small bits of feedback after working with Storj for a while. I couldn’t see if ACLs are used by Storj and if they are, they didn’t seem to be visible in the UI interface and I couldn’t figure out how to use uplink to query for the current ACLs on an object. As I grant access only via signed URLs this didn’t affect me too much as I was setting everything to private anyway.

7 Likes

Sounds very interesting.
I am just a SNO (so maybe I am even storing some of your data already, who knows? :smile:).
As far as I understand the user uploads a file through the portal and the coach downloads it to view it in Vodon Pro?
Have you ever thought of an option so that the files can be streamed directly from Storj DCS into Vodon Pro without downloading it first.

Let me also point you to https://transfer.sh/ which uses Storj DCS as storage backend as far as we know. Maybe this gives you some additional ideas for your portals.

Streaming is tricky because of the file sizes and that a coach might have 5 simultaneous viewpoints loaded at once (even some SSDs struggle to deliver this, let alone over an internet connection) - that said, I’m planning on integrating both a bit more closely, having Vodon Pro manage the downloads into a library of videos for the coach automatically.

Thanks for the link to transfer.sh - very interesting. Similar concept but I suppose mine is a bit more user friendly, although I am a big fan of developer focused tools that can work on the CLI like that.

Yes I understand that streaming might face technical limits. The advantage however would be clear, a coach would simply need a link and could start reviewing immediately without the need to download and to save those big files locally.
Let’s see, maybe someone from Storj will jump on this thread and can add some thoughts what might be possible and what not.
From what I understand is that Storj is currently focused on these kind of use cases and put a lot of optimization into performance through massive parallelism. If that would be enough for such a use case like yours, I cannot tell. But it sounds very interesting I have to say.
@Dominick

2 Likes

Hello @Rodeoclash ,
Welcome to the forum!

It’s good to hear that your app is already working as expected!

Yes, if you could use a native integration, files can be downloaded much faster, see Hotrodding Decentralized Storage
But upload could be slower (consumers ISP plans usually have asymmetric streams - downstream is faster than upstream), so for upload you can use S3 compatibility but for downloads use a native integration (see Storj Client Libraries | Storj Docs).

In case of low bandwidth (like 3G or H+) the S3 compatible layer will be more preferred though, because the file will be uploaded to the one endpoint instead of 110. And download will happen from one endpoint instead of 39.

3 Likes

Great! Thanks for the link, I’ll check it out. Any plans on an official Elixir client library? (and to follow up, do you use some kind of codegen to produce each client library?)

1 Like

The uplink library is implemented in Golang: GitHub - storj/uplink: Storj network Go library
We also maintain the C binding to the uplink library: GitHub - storj/uplink-c: Uplink C library
Bindings in other languages are implemented as wrappers of the C binding. Examples:

Some of the bindings are maintained by Storj, but others are maintained by community members.

Storj currently has no plans for an Elixir binding. If you have the expertise in Elixir, it shouldn’t be a problem to implement such binding as a wrapper to the C binding.

2 Likes

Does not work anymore??

Dumb mistake I made with my Caddyfile config - should be running fine now.

Spent a bit of time trying to improve speeds after some complaints. Converted my upload system to use the split uploads feature in S3 and now I upload 3 parts at one time (with file splitting based on the size of the file). Can confirm that I’ve got the speed to about 95% of what I get uploading to Google Drive which is cool.

I’d be interested in seeing other peoples speeds compared to Google Drive as well if anyone can be bothered testing :laughing:

Hi,

Basically, I would like to do something similar. A simple and free file-sharing website, however, I can’t do that myself. Would your website copy be for sale at one point?

Many thanks!