Problems Hosting Static Sites with Storj

Hi there.

My personal websites is built with a static site generator (Hugo) and deployed to Amazon S3 with Cloudfront’s CDN in front of it. This has worked well for a while.

I’d like to replicate similar functionality with Storj and have been playing around with it. While the documentation is great, I’ve been unsuccessful in getting this to work even in simple scenarios.

Here’s what I’m doing – I’d love to get some help so someone can tell me what I’m doing wrong.

But first, here’s the tools I’m using (I’m running all of this from a Mac):

  • Uplink CLI
    • Release build
    • Version: v1.105.4
    • Build timestamp: 29 May 24 04:33 PDT
    • Git commit: e20f78fa5d0e07eecfb615aa8b74075c7ae25b92
  • rclone v1.67.0
    • os/version: darwin 14.5 (64 bit)
    • os/kernel: 23.5.0 (arm64)
    • os/type: darwin
    • os/arch: arm64 (ARMv8 compatible)
    • go/version: go1.22.4
    • go/linking: dynamic
    • go/tags: none.

What I’m doing

  • I created a new Storj project in the website console
  • I created a new bucket in the website console
  • I created an access key for uplink with the proper grants
  • I created an access key (s3 credential style) for rclone so I can sync my website files into my bucket
  • I configured rclone to use the s3 credentials above and synced my files (confirmed working)
  • I used uplink to make my bucket publicly viewable for website purposes by running uplink share --dns www.mywebsitename.com sj://mybucketname --tls --not-after none
  • I received the uplink CLI output (sanitized version below)
Sharing access to satellite xxx@us1.storj.io:7777
=========== ACCESS RESTRICTIONS ==========================================================
Download     : Allowed
Upload       : Disallowed
Lists        : Allowed
Deletes      : Disallowed
NotBefore    : No restriction
NotAfter     : No restriction
MaxObjectTTL : Not set
Paths        : sj://mybucket/ (entire bucket)
=========== SERIALIZED ACCESS WITH THE ABOVE RESTRICTIONS TO SHARE WITH OTHERS ===========
Access       : xxxx
========== GATEWAY CREDENTIALS ===========================================================
Access Key ID: xxxkey
Secret Key   : xxxsecret
Endpoint     : https://gateway.storjshare.io
Public Access: true
=========== DNS INFO =====================================================================
Remember to update the $ORIGIN with your domain name. You may also change the $TTL.
$ORIGIN example.com.
$TTL    3600
www.mywebsite.com    	IN	CNAME	link.storjshare.io.
txt-www.mywebsite.com	IN	TXT  	storj-root:mybucket
txt-www.mywebsite.com	IN	TXT  	storj-access:xxxkey
txt-www.mywebsite.com	IN	TXT  	storj-tls:true
  • I then went into NameCheap (my DNS provider) and added the necessary records:
    • A CNAME for www that points to link.storjshare.io.
    • A TXT record for txt-www that points to storj-root:mybucket
    • A TXT record for txt-www that points to storj-access:xxxkey
    • A TXT record for txt-www that points to storj-tls:true
  • I then waited a while for DNS to propagate and verified my DNS records were accurate by running:
    • dig @1.1.1.1 txt-www.mywebsite.com TXT

OK, that was a lot, but now that I did all that here’s my problem:

  • When I go to my website using a browser (www.mywebsite.com), I’m able to see my website but it has no CSS/JS/images loaded.
  • I’m able to click between links on my site but all sub-pages also have no CSS/JS/images.
  • If I inspect the browser web console I can see that all of my images/CSS/JS are responding with a 400 and failing to load
  • If I visit or curl one of my assets directly, ex: curl https://www.mywebsite.com/static/css/style.css I get a 400 with a ‘Malformed request’ error from Storj.

This is very confusing to me because I’m somehow able to render the HTML pages just fine, and I can verify the links are pointing to the correct paths that exist within my Storj bucket.

Any ideas of what I’ve screwed up? Greatly appreciate your time! <3

1 Like

Hello @rdegges,
Welcome to the forum!

Are you using absolute paths to these CSS/JS/Image files or are they all relative in the HTML pages?

Do you see these files in the bucket with the specified prefix?

uplink ls sj://mybucketname/static/css/

Also, what is attached content type?

curl 'https://link.storjshare.io/s/accesskeyhere/mybucketname/static/css/style.css' -v --output style.css

See also: edge/docs/linksharing.md at 82d843c4a692e511ef10611042743aed5c205447 · storj/edge · GitHub

Thank you!

To answer your questions…

When I run:

uplink ls sj://mybucketname/static/css/

I do see the file listed. Here’s my real-world output:

uplink ls sj://mybucket/static/css/
KIND    CREATED                SIZE    KEY
OBJ     2024-07-20 17:58:53    3584    style.css

When I run:

curl 'https://link.storjshare.io/s/accesskeyhere/mybucketname/static/css/style.css' -v

I get the following output (very odd):

* Host link.storjshare.io:443 was resolved.
* IPv6: (none)
* IPv4: 136.0.77.2
*   Trying 136.0.77.2:443...
* Connected to link.storjshare.io (136.0.77.2) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=link.storjshare.io
*  start date: Jun 15 10:43:09 2024 GMT
*  expire date: Sep 13 10:43:08 2024 GMT
*  subjectAltName: host "link.storjshare.io" matched cert's "link.storjshare.io"
*  issuer: C=US; O=Google Trust Services; CN=WR1
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://link.storjshare.io/s/accesskey/rdegges-www/static/css/style.css
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: link.storjshare.io]
* [HTTP/2] [1] [:path: /s/accesskey/rdegges-www/static/css/style.css]
* [HTTP/2] [1] [user-agent: curl/8.6.0]
* [HTTP/2] [1] [accept: */*]
> GET /s/accesskey/rdegges-www/static/css/style.css HTTP/2
> Host: link.storjshare.io
> User-Agent: curl/8.6.0
> Accept: */*
>
< HTTP/2 200
< access-control-allow-headers: *
< access-control-allow-methods: GET, HEAD
< access-control-allow-origin: *
< x-request-id: DEocTGzm55W
< content-type: text/html; charset=utf-8
< date: Sun, 21 Jul 2024 18:05:56 GMT
<
<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <title>style.css | Storj</title>

  <meta property="og:url" content="https://link.storjshare.io" />
  <meta property="og:title" content="style.css | Storj" />
  <meta property="og:description" name="description" content="Shared content - Storj" />

  <meta property="og:image" content="https://link.storjshare.io/static/img/file.png?v=82d843c" />


  <meta name="twitter:card" content="summary" />
  <meta name="twitter:site" content="@storj" />
  <meta name="twitter:title" content="style.css | Storj" />
  <meta name="twitter:description" content="Shared content - Storj" />

  <meta name="twitter:image" content="https://link.storjshare.io/static/img/file.png?v=82d843c" />


  <link rel="shortcut icon"
    href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiYCaTi9xzAn4rPPtLpPH7jaMBSDA4DE43AIFWAwYdBLApUf0L/OyFJ2/oWT28/wKd3Tw/RIdB4nLYnAjpa5gxzI8Mm6TRRlxQ8ZSuPQSbjYEGjEaICBHpegagww7gBpOJTm6HY+lMp/2/DBdQRfcA7bmIkEIaC+Vexqq3ZLPTkJOGndT6g6rfUf5V1FqIB+wqRkefag5hBFXP7yHKvoCQRHEFPBeOxgz8oZeNYvHNKGrqMeE48bAdRdj/zyglVBFApLM3EZJpXSHyy3K4PXTjMZpzMTCN+DIfwrJH6HILV7uDks4gbKLTFejxCIsEFYG6T6jPoKjd9MHmu3vpA4RUCrLQViumWb2VSeVv4/M2tuHivaSz0KtGn2W8hopAQZm4vrVSRDd9o1PlqM8Ukvdqffz7L3B+JkXm4OtmX6riu4KSd2UKAl39ZDpIoVnU4cBWO8XmixIv2EZHrivIWd4X0mfguVs44qLObozIwZj9ZO8Zf3uxl0dFUxt0dMHaFRQdwSprqPkjZG+h56VQkqfoqxPpULiPQgys5A6rriNjGNuwEjq7gNMlyAL2XnANQpxJhCXHs3IrXDrMEswQ7u2AMXLKalZ6lJ0/QIhQVSfMnIKDYO8ZC/Ft5E52ugA8EKIX7uR4sSbBrz4xOYK9XWaYJ4iCroQsYAgGpoPbNVD/WRk44TTsotoGFqyDneuFz9EPEAScLkEWWBoF4aHQ8p21tEuRE47ElkVTUS5sXMU8KCaVIRQ4XYI8i1xuaG4nMTX9IZBBzCIWuVDE8d5s+Ar8N4hdwrRBCqYsoHBOx0H+i6YjpyLHnICiHGqHuRKpVUVhm/0l+gsfrGM3Nzju5wAAAABJRU5ErkJggg=="
    type="image/x-icon">

  <link rel="stylesheet" href="https://link.storjshare.io/static/css/bootstrap.min.css?v=82d843c">
  <link async href="https://fonts.googleapis.com/css?family=Inter:300,700" rel="stylesheet">
  <link rel="stylesheet" href="https://unpkg.com/leaflet@1.7.1/dist/leaflet.css"
    integrity="sha512-xodZBNTC5n17Xt2atTPuE1HxjVMSvLVW9ocqUKLsCC5CXdbqCmblAshOMAS6/keqq/sMZMZ19scR4PsZChSR7A=="
    crossorigin="" />

  <link rel="stylesheet" href="https://link.storjshare.io/static/css/style.css?v=82d843c">
  <script src="https://unpkg.com/leaflet@1.7.1/dist/leaflet.js"
    integrity="sha512-XQoYMqMTK8LvdxXYG3nZ448hOEQiglfqkJs1NOQV44cWnUrBc8PkAOcXy20w0vlaXaVUearIOBhiXZ5V3ynxwA=="
    crossorigin=""></script>
  <meta name="viewport" content="width=device-width, initial-scale=1.0, shrink-to-fit=no" />
</head>

<body>


<div class="container-fluid">
  <div class="row">

    <div class="col-lg-8 col-xl-9 bg-grey px-lg-4 px-xl-5 content">

      <div class="row pt-4 pb-2">
        <div class="col-12 col-md-5 text-center text-md-left">
            <div class="file">
              <img src="https://link.storjshare.io/static/img/icon-file.svg?v=82d843c" class="d-block d-md-inline-block mx-auto" alt="File icon">
              <p class="file-title-sidebar font-weight-bold mt-3 mt-md-0 ml-0 ml-md-1">style.css</p>
            </div>
            <p class="mt-3">3.58 KB</p>
        </div>
        <div class="col-12 col-md-7 text-center text-md-right">

            <a href="?download=1" class="btn btn-primary btn-lg mb-3 mx-1 mx-md-2 mt-1" download>Download <img src="https://link.storjshare.io/static/img/icon-download-white.svg?v=82d843c" alt="Download" class="ml-2"></a>
            <button type="button" onclick="openModal()" class="btn btn-outline-primary btn-lg mb-3 mt-1 btn-share">Share <img src="https://link.storjshare.io/static/img/icon-share.svg?v=82d843c" alt="Share" class="ml-2"></button>
        </div>
      </div>

      <div class="row justify-content-center">
        <div class="col-12 pb-4">
          <object class="embed-responsive embed-responsive-4by3" id="pdfTag"></object>
          <img class="embed-responsive embed-responsive-4by3" id="imgTag" alt="preview image">
          <video class="embed-responsive embed-responsive-4by3" id="videoTag" controls></video>
          <audio class="embed-responsive embed-responsive-4by3" id="audioTag" controls></audio>
          <img class="embed-responsive embed-responsive-4by3 d-none" id="placeholderImage" src="https://link.storjshare.io/static/img/file-drop.svg?v=82d843c" alt="placeholder image">
        </div>
      </div>

    </div>

    <div class="col-lg-4 col-xl-3 px-lg-4 px-xl-5 sidebar active d-block">
      <div class="row mb-3 mt-3">
        <div class="col text-center text-md-left">
          <a href="https://www.storj.io/"><img src="https://link.storjshare.io/static/img/logo.svg?v=82d843c" class="logo mt-4" alt="Storj Logo"></a>
        </div>
      </div>
      <div class="row">
        <div class="col">
          <div class="row justify-content-center">
            <div class="col py-3 py-lg-4 text-center text-md-left">
              <h3 class="mb-3">You’re getting this file from all over the world.</h3>
              <p>Storj splits files into smaller pieces, then distributes those pieces over a global network of nodes and recompiles them securely on download. This map shows the real-time locations of this file’s pieces.</p>
              <p class="nodes-count w-100 mt-2 text-center">0 Storj nodes are storing a piece of this file</p>
            </div>
          </div>
          <div class="row">
            <div id="map-img" class="col-12 col-lg-12 text-center map mt-4 mb-5">
              <img src="?map=1&width=800" style="width:100%;" />
            </div>
          </div>
        </div>
      </div>
      <p><img class="collapser" src="https://link.storjshare.io/static/img/collapse.svg?v=82d843c" onclick="toggleSideBar()" alt="collapse icon"></p>
    </div>
  </div>
  <p><img class="expander" src="https://link.storjshare.io/static/img/expand.svg?v=82d843c" onclick="toggleSideBar()" alt="expand icon"></p>
</div>


<div class="modal fade" id="shareModal" tabindex="-1" aria-labelledby="shareModalLabel" aria-hidden="true">
  <div class="modal-dialog modal-dialog-centered">
    <div class="modal-content text-center p-2 p-sm-4 p-md-5">
      <div class="modal-header border-0">
        <div class="copy-notification" id="copyNotification">
          <p class="copy-notification-text">Link Copied!</p>
        </div>
        <h5 class="modal-title mx-auto" id="shareModalLabel">Share style.css</h5>

      </div>
      <div class="modal-body pt-0">
        <p>Just copy and paste the link to share this file.</p>
        <input class="form-control form-control-lg mt-4 input-url" type="url" id="url" readonly>
        <button type="button" name="copy" class="btn btn-light btn-copy" onclick="copy()" id="copyButton">Copy</button>
      </div>
      <div class="modal-footer border-0">
        <button type="button" class="btn btn-primary btn-block btn-lg" data-dismiss="modal" onclick="closeModal()">Done</button>
      </div>
    </div>
  </div>
</div>
<div class="modal-backdrop fade show" id="backdrop" style="display: none;"></div>

<script type="text/javascript">
  const pdfExtensions = 'pdf'
  const imageExtensions = ['bmp', 'svg', 'jpg', 'jpeg', 'png', 'ico', 'gif']
  const videoExtensions = ['m4v', 'mp4', 'webm', 'mov', 'mkv']
  const audioExtensions = ['mp3', 'wav', 'ogg']

  function openModal() {
    if(!navigator.clipboard) {
      document.getElementById("copyButton").disabled = true;
    }
    document.getElementById("backdrop").style.display = "block"
    document.getElementById("shareModal").style.display = "block"
    document.getElementById("shareModal").className += "show"
    input.value = window.location.href;
  }

  function closeModal() {
      document.getElementById("backdrop").style.display = "none"
      document.getElementById("shareModal").style.display = "none"
      document.getElementById("shareModal").className += document.getElementById("shareModal").className.replace("show", "")
      document.getElementById("copyNotification").style.display = "none"
  }

  function copy() {
    navigator.clipboard.writeText(input.value)
    document.getElementById("copyNotification").style.display = "block"
  }

  function setupPreviewTag(id) {
      const previewURL = `${window.location.origin}${window.location.pathname}?wrap=0` + ""

      const el = document.getElementById(id)
      el.style.display = 'block'
      if (el.tagName === 'OBJECT') {
        el.data = previewURL
      } else {
        el.src = previewURL
      }
  }

  let modal = document.getElementById('shareModal');
  let input = document.getElementById('url');

  window.onclick = function (event) {
      if (event.target == modal) {
          closeModal()
      }
  }

  function toggleSideBar() {
      const sidebar = document.querySelector('.sidebar');
      const contentArea = document.querySelector('.content')
      const expander = document.querySelector('.expander');

      if (!(sidebar && contentArea && expander)) {
          return;
      }

      sidebar.classList.toggle('active');
      expander.classList.toggle('active');

      if (sidebar.classList.contains('active')) {
          sidebar.classList.replace('d-none', 'd-block');
          sidebar.classList.replace('col-xl-0', 'col-xl-3');
          sidebar.classList.replace('col-lg-0', 'col-lg-4');
          contentArea.classList.replace('col-lg-12', 'col-lg-8')
          contentArea.classList.replace('col-xl-12', 'col-xl-9')
      } else {
          sidebar.classList.replace('d-block', 'd-none');
          sidebar.classList.replace('col-xl-3', 'col-xl-0');
          sidebar.classList.replace('col-lg-4', 'col-lg-0');
          contentArea.classList.replace('col-lg-8', 'col-lg-12')
          contentArea.classList.replace('col-xl-9', 'col-xl-12')
      }
  }

  function setPlaceholderImage() {
    document.getElementById('placeholderImage').classList.remove('d-none');
  }

  window.onload = async function () {
      let fileExtension = "style.css".split('.').pop();
      if (fileExtension) {
        fileExtension = fileExtension.toLowerCase();
      }

      switch (true) {
          case fileExtension === pdfExtensions:
              setupPreviewTag('pdfTag')
              break
          case imageExtensions.includes(fileExtension):
              setupPreviewTag('imgTag')
              break
          case videoExtensions.includes(fileExtension):
              setupPreviewTag('videoTag')
              break
          case audioExtensions.includes(fileExtension):
              setupPreviewTag('audioTag')
              break
          default:
              setPlaceholderImage()
      }
  }
</script>

<footer>
    <div class="container-fluid">
        <div class="d-flex col-12 px-lg-4 px-xl-5 align-items-center justify-content-between footer">
            <div class="d-flex row align-items-center justify-content-start">
                <a href="https://www.storj.io/" class="d-block d-lg-inline-block"><img src="https://link.storjshare.io/static/img/logo.svg?v=82d843c" class="logo mt-0 mb-4 mb-lg-0" alt="Storj Logo"></a>
                <p class="m-0 ml-5">Sign up for a 30 day trial today.</p>
            </div>
            <div class="d-flex row align-items-center justify-content-end buttons-area">
                <a href="https://storj.io/signup" class="btn btn-primary btn-lg d-block d-md-inline-block">Sign Up</a>
                <a href="https://www.storj.io/" class="btn btn-outline-primary btn-lg ml-4 px-4">Learn More</a>
            </div>
        </div>
    </div>
</footer>

</body>
</html>

* Connection #0 to host link.storjshare.io left intact

NOTE: I’ve left my public key and real bucket name in the output above so you can see what I see.

It looks to me like the file being returned is a Storj webpage and not the CSS file I’m expecting. And this is weird because if I directly copy the file using uplink and open it, I get the expected CSS file directly:

uplink cp sj://mybucket/static/css/style.css test.css
cat test.css
# <expected CSS file here>

Yes, please do the same for /raw/ instead of /s/:

curl 'https://link.storjshare.io/raw/accesskeyhere/mybucketname/static/css/style.css' -v --output style.css

then it should download the file, not showing our preview page, so it is accessible and working correctly. You do not need to post result here, because I already used your key to see a content-type, it’s correct:

* Request completely sent off
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Access-Control-Allow-Headers: *
< Access-Control-Allow-Methods: GET, HEAD
< Access-Control-Allow-Origin: *
< Content-Disposition: attachment; filename=style.css
< Content-Length: 3584
< Content-Type: text/css; charset=utf-8
< Last-Modified: Sun, 21 Jul 2024 00:58:53 GMT

I now hided the public key in your message.

So, the problem seems appearing only for the custom domain, I suppose?
If so, could you please check your HTML files - how the paths to the images, CSS and JS are specified - as an absolute links or as a relative?

Ok, I found your site and checked, they are relative. I would ask the team, because everything looks correct and it should work.

@rdegges I’m sorry for problems you have. We checked it on our side and we think this’s a bug. Could you fill bug report under Sign in to GitHub · GitHub?

Oh wow, I’ll file a report. Thanks so much for looking into this @Alexey and @michaln! I just assumed I was doing something dumb =D

For those following along, here is the GitHub issue: Hosting a Static Website on Storj: Potential Issue? · Issue #458 · storj/edge · GitHub

3 Likes