This is an advance feature of satellite, the engine behind storage tier.
You can have as many tier as you like (65536 limit for now) with extraordinary placement rule - the placement rule itself is a mini programming language.
What is placement?
A placement is a named policy for bucket that answers four questions about a piece of data:
- Which nodes are eligible to hold it? (filter)
- How do we pick from among the eligible nodes? (selector)
- What must remain true after repair? (invariant)
- How many pieces do we make, and when do we repair? (EC overrides)
Every bucket on your satellite carries a placement ID. Placement 0 is the default and is used whenever nothing else is specified.
Have a look at the legacy placement.
To ensure maximum compatibility, start your own placement id at 11 or higher.
Side story: config
Storj gave you many way to config your satellite:
- ENV variable
- binary runtime flags
- config.yaml
My rule of thumb: use config.yaml for config that contain a long string or sensitive information or satellite specific content, use binary flags for everything else.
Never use ENV variable - it only suitable for an unforgiving env like container where it hard for you to debug on production (some team might disagree). One benefit of ENV variable is that it more secure - but it also hard for you to debug.
Put everything on a single config.yaml seem like good practice - until you realize, you don’t know which service actually use that config and feel the pain every time to read through the yaml file. Of course, the inverse is also true, now you have to copy the same config for 2 or more services - but hey, at least you know that service actually use it.
Preparation
To use this feature you need to enable a few things:
--overlay.geo-ip.db=/tmp/GeoLite2-City.mmdb on satellite-modular api
--console.placement.self-serve-enabled=true: this one is nasty, you need to this on satellite-modular console BUT you also need this on satellite-modular api — If you want only admin can change bucket tier, then don’t enable this feature.
Side lore: delete doesn’t really mean delete
While researching this, I notice I don’t like few things when operate the satellite - delete doesn’t really mean delete.
- Delete account mean mask your user record, email become
deactivated+%s@storj.io - Delete your bucket and
value_attributionis still there, I understand about calculating billing, paying for partner and SNO, but there are no way to delete those records. - Delete your project just mean it will become
status = 0
…
Maybe more, I didn’t check it all… To me, database is a sacred place: those records will not live there rent free forever.
Example config for placement
This one should be on config.yaml:
console:
placement:
self-serve-details: |
- id: 0
id-name: "GLOBAL_0"
name: "Global"
title: "Globally Distributed"
description: "The data is globally distributed."
- id: 11
id-name: "US_SELECT_11"
name: "US Select 11"
title: "US Select 11"
description: "Store data only on Select nodes in the United States."
allowed-placement-ids-for-new-projects: '[0]' # if only one choice, front end will remove choice UI for you
# maybe you don't need this
#payments:
# products: |
# - id: 1
# name: "Global"
# short-name: "GLOBAL_0"
# storage: "4"
# egress: "7"
# segment: "0.0000088"
# - id: 2
# name: "US Select 11"
# short-name: "US_SELECT_11"
# storage: "8"
# egress: "10"
# segment: "0.0000088"
# placement-price-overrides: |
# 1: [0]
# 2: [11]
# short form placement
placement: '0:annotation("location","GLOBAL_0");11:annotation("location","US_SELECT_11")'
# or long form placement: /etc/storj/placement.yaml
# placement: /etc/storj/placement.yaml
What is possible with placement rule?
This entire section is out of my reach, Claud, can you take the mic? Sure thing:
# /etc/storj/placement.yaml
templates:
SIGNER: 12Q8q2PofHPwycSwAVCpjNxxzWiDJhi8UV4ceZBo4hmNARpYcR7
NO_DATACENTER: exclude(tag("$SIGNER","datacenter","true"))
placements:
- id: 0
name: global
filter: $NO_DATACENTER
selector: attribute("last_net")
invariant: maxcontrol("last_net", 1)
- id: 1
name: eu-1
filter: country("EU") && $NO_DATACENTER
upload-filter: exclude(country("DE"))
selector: attribute("last_net")
invariant: maxcontrol("last_net", 1)
download-selector: random
ec:
minimum: 29
repair: "+5"
success: 80
total: 110
There are also cohort in placement I think?
To test placement, there is a tool in cmd/tools/placement-test.
This topic is a bit advance for me, so the quality drop a bit, see you in part 18.
