Here I would like to share my experience of moving my static website hosted on GitHub to Storj.
I have hosted the website since a decade on GitHub Pages. The website has been built with Hugo locally and uploaded there to be served from there, so it is a pure static website (ie. without Jekyll dynamic template etc. which requires dynamic compilation). As I have known that a website can be served via a bucket on Storj, I finally decided to give it a try.
Requirements
What I wanted to achieve here are:
- Track the website history with Git
- Serve the website with Cloudflare CDN
Version control is the main reason why I have hosted the site on GitHub, but I realized GitHub was not the only service where you could use version control in the first place. Git repositories can be managed locally, and its backup can be well maintained thank to a tool such as Duplicati with granular configurations, and this indeed makes much more sense to me, as I have already maintained other Git repositories, for example, one for Radicale (CalDAV server), in that way.
Procedure
Here’s what I have done to move the website to Storj, ensuring to keep both Git and CloudFlare CDN:
- Create a git repository locally
- Clone the repository from GitHub to the local repository
- Build the site with Hugo
- Create a bucket to host the site data
- Set up
rclone, following the instruction available at Configure Rclone Natively - Storj Docs - Copy the built website data to the bucket with
rclone - Follow the instruction available at Setting Up Custom Domains - Storj Docs
- Go to CloudFlare to set up DNS records by adding the records provided as DNS info in the output of the last step
- Set up a Duplicati backup task to ensure the git repository is properly backed up on Storj
Root domain setting on Cloudflare
(Please correct me if I am wrong; I did not find the instruction about this on the documentation page.)
One tricky thing about hosting the website is that it does not seem to be possible to host it directly on the root domain, such as example.com, unless you would set up a reverse proxy by yourself. With the combination of Storj and Cloudflare, the website seems to be required to have a subdomain such as www, as the two TXT records (txt-www.example.com) expect them. My website has been provided with the subdomain www from the start, so this was not an issue for me anyway.
In this case, you might want to set up a redirect so that accessing to example.com is redirected to www.example.com. Otherwise, the website cannot be reached with example.com.
For that matter, Cloudflare DNS dashboard recommends you to add an A, AAAA, or CNAME record for root domain. GitHub Pages provides A records to redirect the root domain, but as the website is no longer served from GitHub, you can remove them, and add CNAME record instead as below thanks to CNAME flattening:
(cropped from the image above)
If the website cannot be reached with the root domain yet after adding the CNAME record, you might want to wait for a couple of minutes until DNS update is shared or try with another computer as the DNS cache can be stored on the computer.
Update the website with rclone
To update the website, simply rebuild the website and use rclone to synchronize the local directory for the website with the remote bucket.
This documentation should help you to understand how to synchronize them: Rclone with S3 Compatibility Guide - Storj Docs. As warned on the documentation, do not forget to test with --dry-run to ensure the files are synchronized as expected.
Feedback
So far the website seems to be working as expected.
This example on the instruction did helped me to understand how the bucket and DNS records should be configured. By following the instruction it was not difficult to move my website from GitHub to Storj.
I followed the Cloudflare’s recommendation, but explanation about root domain setting regarding the way of redirecting the base domain would make the migration experience smoother.
Performance optimization with Cloudflare
The website started to be rendered in the same way as before, but I felt a little bit of lag.
I ran a performance test on webpagetest.org several times, and I noticed that the time to first byte to a computer in my country was around 1.1 second on average, and starting to render required around 2.3 - 2.5 seconds, which I find are not optimal if I am honest. I think it is a matter of node distribution; the metrics would be better if there were more nodes around me (Asia-Pacific), especially in the East Asia. I guess the performance should be fine when you host your website in European network and most of the requests come from there as well.
Based on my risk profile, I configured Cloudflare to proxy DNS records:
I ran another performance test on webpagetest.org, and I noticed the performance has been greatly improved (the time to first byte was 0.9 second, and starting to render required 1.2 second). E2EE was sacrificed in favor of the performance, but I think it is better anyway to have my website as origin distributed over the network than storing it on a centralized service.