JS library for the browser

Developer Discussions
1

Is there JS library that can be used on client side in web browsers to access files on storj network?

2

Yes, we have a binding to NodeJS:

All other Storj-backed bindings you can see there: https://documentation.tardigrade.io/api-reference/storj-client-libraries
Community bindings: https://documentation.tardigrade.io/api-reference/libraries

1 Like
3

Is it client side?

4

NodeJS is server-side @Alexey. Im looking of a way for chrome and firefox to download file directly from storj network without a proxy on my or someone else’s server. Are you saying that if I was to add that nodejs lib to my react project, it should run in the browser?

5

At the present time, we don’t have a native browser-based implementation. One of the things we haven’t done yet is to implement a way for a browser to trust the Storage Nodes from which pieces of files are directly downloaded peer-to-peer.

So, you need to have a server anyway.

6

So it is not on the roadmap yet?

7

Hey friends!

We currently do not have client-side in-browser Javascript (or WASM) bindings on the 2020 roadmap, but we have made some exciting and relatively unexpected progress just this week that make it much more feasible.

At a high level, there are a couple of challenges we need to overcome for this to work:

  1. We need the code running inside the browser. This is relatively straightforward these days to be honest - even though the Uplink library is large, Go natively can target WASM modules, which can then be used to interoperate with the browser inside Javascript. Recent testing just this week suggests that this WASM module can be delivered compressed in a much smaller package than we thought it would, so while it won’t be the smallest Javascript library, it is well within the range of reasonableness now. We have some more testing and work to do before this is part of our build process, which may not happen for some time because:
  2. We need the browser to be able to talk to nodes directly over SSL which requires browser trusted certificates for all storage nodes. For a flavor of how this works, please check out https://blog.filippo.io/how-plex-is-doing-https-for-all-its-users/. It’s an exciting solution that will require we provide DNS entries to all storage nodes automatically. Unlike that blog post though, we probably can’t afford the same agreement with Digicert. Luckily, Let’s Encrypt might be able to help us here! We are working on some approaches to combine Let’s Encrypt CA signed certs with our existing decentralized certificate authority node identity system.
  3. We need to support proxying traffic over HTTP requests to storage nodes. Our current protocol does not use HTTP.

Once we’ve tackled these three things, then browsers will be able to talk to storage nodes directly with a Javascript library. Sadly it’s not quite as straightforward as we’d like, which is why we’re still expecting a long time estimate on it (it’s not slated for 2020), but we do believe all of these problems are directly tractable and solveable. If anyone wants to help contribute to the project to get these steps across the line sooner than 2020, please let me know!

5 Likes
8

Thanks for the thorough answer. It seems that #3 can be solved if storagenodes runs http to grpc proxy. grpc already has production ready browser lib which requires http to grpc proxy and there is a go implementation of that proxy for serverside. Same http proxy could take care of letsencrypt verification so storagenode can obtain certs for each satellite.

all is required is for satellite to provide dns service for storagenodes. It can be something as simple as service that returns ip address derived from subdomain. Something like: dns A lookup for 1-2-3-4.proxy.satellite-fqdn.com can be translated to ip 1.2.3.4. Only caveat there is that storagenode needs to detect when it’s public ip changes and ask for new cert from letsencrypt. But since ips dont change as often, it might not be that much of an issue, especially since there are other nodes already for redundancy to provide same piece to the web client, while some nodes are updatimg their certs.

So if satellite provides dns service like above, and storagenode implements http server for http to grpc proxy and for cert verification from letsencrypt, then it should be easy to have libuplink client in browser.

9

This post kind of made my day. I love clever solutions and yeah this stuff takes time to develop. But I’m always enlightened by the references Storj uses for solving specific problems. The Plex link is a great example of that and the white paper has a lot of that stuff too.

This post has kind of switched my perspective from thinking this is something really hard to crack, to something doable with enough time and engineering effort. :+1:

2 Likes
10

Thanks for the thorough answer. It seems that #3 can be solved if storagenodes runs http to grpc proxy. grpc already has production ready browser lib which requires http to grpc proxy and there is a go implementation of that proxy for serverside. Same http proxy could take care of letsencrypt verification so storagenode can obtain certs for each satellite.

We actually do not use gRPC! One of the things drpc ripped out for improved simplicity was any HTTP layer: GitHub - storj/drpc: drpc is a lightweight, drop-in replacement for gRPC. That isn’t to say this isn’t solvable, but we won’t be able to use gRPC methods.

In terms of DNS, this is probably not something we will provide per Satellite, as we would then need each Satellite to be registered as a public domain suffix as per Let’s Encrypt. That said, it is still a service we can run with the right public domain suffix registrations.

This post has kind of switched my perspective from thinking this is something really hard to crack, to something doable with enough time and engineering effort. :+1:

Yay!

11

Out of curiosity, has there been any progress regarding an uplink implementation that would work inside a web browser?

1 Like
12

Just to throw it out there, if they want a purely JavaScript implementation, it would also solve this problem.

14

@jtolio would storj consider switching to https://buf.build/blog/connect-a-better-grpc ? It has full grpc compatibility and also would allow connecting to nodes from browsers. You could have full blows JS client that has same capabilities as the command line one.

15

We do not use gRPC, we use DRPC, see Introducing DRPC: Our Replacement for gRPC

2 Likes