Can we use MINIO_CACHE with Gateway?

Developer Discussions DCS
1

Question is simple: can MINIO_CACHE be leveraged while using the gateway? Not only for static websites but for other uses.

Thanks.

2

Hi there,

We are using it for cncted to speed up access to smallish jpg files. Can you tell us more about what other use cases you are hoping it will help with?

3

Well to avoid contacting Storj network (and egress fees) for a configured amount of time for every file in our bucket, just like we do with Minio gateway and any S3 provider :slight_smile:

4

Hi, thanks for providing the additional information. You should be able to use MINIO_CACHE with our gateway. Here’s a reference on how we did it for our cdcted (https://github.com/Razva/storj/blob/31dff66a1494b792226896c9ff7295e750409b77/service/storj-gateway.service).
You can try to:

  • create a mount point just for the minio cache
  • configure the various flags - some of which only made sense after we read the code
  • export those flags

Let me know if you are able to get it to work.

Thanks

1 Like
5

Seems fine! Will give it a try in the next few hours and let you know. Thanks. :wink:

6

Hi, it works great; perhaps in our case, as we are proxying to many gateways trough nginx, i’ve implemented the cache at nginx (proxy_cache) as it best fits our needs (cache at the proxy, not at each gateway node). :+1:

If it can help someone:

# Configure out caches
proxy_cache storj;

# Cache all 200 OK's for 30 days
proxy_cache_valid 200 30d;

# Use stale cache file in all errors from upstream if we can
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;

# Lock the cache so that only one request can populate it at a time
proxy_cache_lock on;

# Do not convert HEAD into GET
proxy_cache_convert_head off;

# GET and HEAD are cached by default
proxy_cache_methods GET HEAD;

# Since head conversion is disabled, the request_method is included in the proxy_cache_key
proxy_set_header Host $host;
proxy_cache_key "$scheme$request_method$proxy_host$host$request_uri";
2 Likes
7

@Yingrong we’ve tested MINIO_CACHE further and there is something i can’t understand.

With a standard MinIO server, when CACHE is enabled, cache path populates very fast and rapidly grows.

# du -sh /mnt/cache/minio
12G    /mnt/cache/minio

Here, with your example, after some traffic, folder is only a few KB size…

# du -sh /var/cache/storj
884K    /var/cache/storj

Here is my test service:

[Unit]
Description=Storj Gateway service
After=syslog.target network.target
Wants=network-online.target

[Service]
User=storj
Group=storj
ExecStart=/usr/local/bin/gateway run --config-dir=/etc/storj/gateway
Restart=always
RestartSec=5s
Type=simple
NotifyAccess=main
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=yes
LimitMEMLOCK=infinity
LimitNOFILE=65535

StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=storj-gateway
Nice=10

Environment=MINIO_CACHE=on
Environment=MINIO_CACHE_DRIVES=/var/cache/storj
Environment=MINIO_CACHE_EXPIRY=3650
Environment=MINIO_CACHE_QUOTA=60
Environment=MINIO_CACHE_WATERMARK_LOW=80
Environment=MINIO_CACHE_WATERMARK_HIGH=95
Environment=MINIO_CACHE_AFTER=1

[Install]
WantedBy=multi-user.target

Is gateway only caching something like metadata only? :thinking:

8

that’s strange. Can you please check the permission on /var/cache/storj (does the storj user and group have write access there):
sudo -u storj
ls /var/cache/storj
touch /var/cache/storj/test
rm /var/cache/storj/test
Do you see any new files showing up in /var/cache/storj? Clear out the cache, capture the output from find /var/cache/storj, download an object through the gateway, capture the output again and compare it.

9

Hi, it seem it was just a bad configuration,
after rebuild, it is OK. Thanks.

1 Like