[Tutorial] Run your own satellite (part 7) - Satellite api

The api subcommand in satellite-modular, it runs the customer and node-facing layer — it’s the front door of the satellite. It handles all real-time requests but does not do any heavy background processing (repair, auditing, accounting rollups, etc.).

See a fun diagram explain how api sign bandwidth on page 53 https://static.storj.io/storjv3.pdf.


For ./satellite-modular api to work, we need at least (it have gazillion flags):

--identity-dir
--database-options.url
--metainfo.database-url
--live-accounting.storage-backend
--orders.encryption-keys
--server.use-peer-ca-whitelist

What are those flags?

--identity-dir ./id/satellite/: if you are using my script on [Tutorial] Run your own satellite (part 3) - Generate certificates.

For --database-options.url and --metainfo.database-url, see [Tutorial] Run your own satellite (part 6) - Database migration, at this time, we don’t do split table yet, if you split it now, you will miss out on tutorial how to hot swap database/table on production.

--live-accounting.storage-backend redis://127.0.0.1:6379, you need to install redis, and for now, we will use redis standalone, I’ve a PR to use redis cluster, but we will see if it merged or if it even work.

--orders.encryption-keys: to generate this key, use this script

--server.use-peer-ca-whitelist: in storj/storj commit 1d63395fd, comment mention to disabling this on satellites entirely, but for reason, no follow up commit ever realize this, you must set this to false.

echo $(openssl rand -hex 8)=$(openssl rand -hex 32) # keep these keys safe

Put it together

./satellite-modular api \
--identity-dir ./id/satellite/ \
--database-options.url "cockroach://my_role:123456@rendezvous.example.com:26257/my_database?sslmode=verify-full&sslrootcert=ca.crt" \
--metainfo.database-url "cockroach://my_role:123456@rendezvous.example.com:26257/my_metainfo_database?sslmode=verify-full&sslrootcert=ca.crt" \
--live-accounting.storage-backend "redis://127.0.0.1:6379" \
--orders.encryption-keys 88a2ab18c619535b=e9bbe3c658d595d5c40dba994152abc07109cdd4425fec627194bdec32c43333 \ # not a real key
--server.use-peer-ca-whitelist=false

For now, this should be put in a systemd file /etc/systemd/system/satellite-api.service:

[Unit]
Description=Satellite API
Requires=network.target

[Service]
Type=simple
WorkingDirectory=/my_satellite/
ExecStart=/my_satellite/satellite-modular api --identity-dir ./id/satellite/ --database-options.url "cockroach://my_role:123456@rendezvous.example.com:26257/my_database?sslmode=verify-full&sslrootcert=ca.crt" --metainfo.database-url "cockroach://my_role:123456@rendezvous.example.com:26257/my_metainfo_database?sslmode=verify-full&sslrootcert=ca.crt" --live-accounting.storage-backend redis://127.0.0.1:6379 --orders.encryption-keys 88a2ab18c619535b=e9bbe3c658d595d5c40dba994152abc07109cdd4425fec627194bdec32c43333 --server.use-peer-ca-whitelist=false
TimeoutStopSec=300
Restart=always
RestartSec=10
User=root

[Install]
WantedBy=default.target

Of course, you have to change to fit your need.

To see your satellite base58-check node ID, use this command:

./identity certificate-authority id --ca.cert-path id/satellite/identity.cert # thanks @alpharabbit

It shall look similar to node ID in https://static.storj.io/dcs-satellites, thing are coming together.

By default, satellite api will listen publicly (0.0.0.0) on port 7777 and privately on port 7778 (127.0.0.1) to communicate with admin service.

If you want to use hashstore (see ./satellite-modular api -h | grep hashstore), set these flags to true:

--contact.hashstore-rollout.current.write-to-new=true \
--contact.hashstore-rollout.next.write-to-new=true \
--contact.hashstore-rollout.next.read-new-first=true \
--contact.hashstore-rollout.next.write-to-new=true \

IMPORTANT UPDATE: if you are using caddy to proxy, it is important to self built it with layer4 support (not the default caddy binary ship with your OS) to proxy tcp on port 7777.

The way I do that on debian/ubuntu - install via apt then use dpkg-divert and mark /usr/bin/caddy to never update let you handle that file, but of course, you can do it however you want.

See you on part 8.

Protip: if you don’t understand what specific flag mean, copy the explaination and paste to github (using “” to search exact), and see the code, if still don’t understand, copy code content and ask AI. It’s 2026!

Update Jul 27:

Q: What to do if your new satellite have enough nodes but not enough unique /24 subnet?
A: There are many knots to try, but personally I use this:

overlay.node.distinct-ip # for satellite-modular api
overlay.node.new-node-fraction # for satellite-modular api and repair

After a while, use this sql to confirm if you have enough unique subnets then remove config above:

SELECT count(DISTINCT regexp_replace(last_ip_port, '\.\d+:\d+$', ''))
    AS distinct_24s, count(*) AS nodes FROM nodes
    WHERE last_contact_success > now() - interval '4 hours'
    AND disqualified IS NULL
    AND last_ip_port NOT LIKE '[%';

Update Aug 3:

If you are running a test satellite, you should set price config so storagenode operator don’t get supprised about it:

# config.yaml
compensation:
  rates:
    at-rest-gb-hours: "0"
    get-audit-tb: "0"
    get-repair-tb: "0"
    get-tb: "0"
    put-repair-tb: "0"
    put-tb: "0"

So, I’ve spend sometime reading about the numerous flags on satellite-modular api -h, right now, focusing on cache and it a bit confusing:

./satellite-modular api -h | grep cache | grep 1000
        --bucket-eventing.cache.capacity string                                              maximum number of entries in the in-memory config cache (default "10000")
        --orders.flush-batch-size string                                                     how many items in the rollups write cache before they are flushed to the database (default "1000")
        --orders.public-project-id-cache-capacity string                                     capacity of the public project ID LRU cache used for eventkit tracking (default "100000")
        --accounting.retention-remainder-recorder.cache-capacity string                      capacity of the retention remainder recorder (default "10000")
        --database-options.api-keys-cache.capacity string                                    satellite database api key lru capacity (default "10000")
        --database-options.revocations-cache.capacity string                                 macaroon revocation cache capacity (default "10000")
        --console.ghost-session-cache-limit string                                           maximum number of ghost session email timestamps to keep in memory (default "10000")
        --metainfo.rate-limiter.cache-capacity string                                        number of projects to cache. (default "10000")
        --metainfo.upload-limiter.cache-capacity string                                      DEPRECATED. number of object locations to cache. (default "10000")
        --metainfo.user-info-validation.cache-capacity string                                user info cache capacity (default "10000")
        --metainfo.project-entitlement.cache-capacity string                                 delete objects hook cache capacity (default "10000")
        --metainfo.api-key-tails-config.cache-capacity string                                API key tails cache capacity (default "10000")
        --contact.rate-limit-cache-size string                                               the number of nodes or addresses to keep token buckets for (default "1000")

These are satellite in-memory cache (LRU) for various services, it’s good, just one problem: how to get an insight of these cache when operating the satellite? I try to get that info via debug server (--debug.addr) but unable to find that information, the need is about: cache hit, cache miss, current number of entries (if cache self-eviction is a feature), eviction count number…

It seem that info is not available yet? It’s alright, just adding some word here to remind myself one more entry to TODO list.

You may try to ask AI agent for the Storj repository.
But seems these metrics are not available.

Hi, turn out it not as simple to just turn redis into redis-cluster:

Some command, for now: MGET and SCAN, it work differently when using on redis and redis-cluster (I could provide patch to make it work consistently – for example: projectID[:]) become "{" + projectID[:]) + "}", so it will be on the same shard, but it require all future code written be aware of this different too).

The hope is that StorJ will agree with new mental model about redis-cluster, I know that a big ask, so for now I’ll skip on this and write on the next tutorial.

Also, I now know redis can use username/password, it is important because implementation only need 2 new fields in config:

  • mode: standalone or cluster.
  • cafile: path to ca.crt file.