Signed URL 500 error

Developer Discussions
1

Hi all,

I’m using signed URLs to allow users to upload files from their browser. I just had a user get a 500 error on a PUT request signed URL upload (some fields redacted)

https://gateway.storjshare.io/uploads/11/20/62/file?ACL=private&Content-Disposition=attachment%3B%20filename%3Drune-vod%20review.mp4&Content-Type=video%2Fmp4&key=11%2F20%2F62%2Ffile&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=REDACTED%2Fgateway%2Fs3%2Faws4_request&X-Amz-Date=20220803T082146Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=REDACTED

I’ve probably close to hundreds of other uploads go without issue. Any idea why this one may have failed?

EDIT: It seems that I’m getting the issue as well. My upload seems to be stuck without completing, I also can’t seem to open any of my buckets in the UI, they’re just timing out with a spinner. Is Storj having an outage??

1 Like
2

The network request in the UI eventually returned after ~2m with:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: rpc: tcp connector failed: rpc: dial tcp 35.221.198.23:7777: i/o timeout)</Message><BucketName>uploads</BucketName><Resource>/uploads</Resource><RequestId>1707CC67C0F00544</RequestId><HostId></HostId></Error>
3

The outgoing request which is timing out with the error is:

https://gateway.storjshare.io/uploads?delimiter=%2F&prefix=

If that helps in any way

4

Ok, service now seems to be restored.

The downtime by my estimate was from 08:25:45 UTC time until 10:08 am (now).

What was going on??

5

Shoot, sorry this affected you! A victim of our own success to some degree - we’ve had a lot of load recently! Some error rates have been a bit higher (largely due to internal timeouts).

We’ve been working on scaling up but I’m always bummed when scaling challenges are noticeable. We’re bringing on a lot more Satellite and gateway capacity soon. Thanks for your patience!

3 Likes
6

All good. It’s a bit stressful when stuff like this happens but it’s unclear that Storj knows theirs an issue / the status page isn’t updating :grimacing:

3 Likes