In the past, I’ve use ansible and similar tools to deploy, still love ansible but there are problem with it: it does not reuse hard fought learned unix command that ingrain in the brain of mass population.
A lot of tool nowaday seem to be doing that, inventing their own domain specific language to do something without trying to reuse the resource user already have.
I just discover (actually, know it for a while but never have the chance to use it) a new tool for deployment, the author taste seem excellent to me.
# this is actually php code
task('prepare', function () {
run('mkdir -p /opt/tutor');
run('apt update');
run('apt install caddy -y');
upload('Caddyfile', '/etc/caddy/Caddyfile'); # from local to remote host
run('systemctl enable caddy');
run('systemctl start caddy');
});
What was that ?!?
Have not seen this level of succinct in any other tools, it took me 1 day to learn this and I think it will save ton of engineer time and resource in the long run.
The tool is only a binary file ~600kB (though it require install php apt install php-cli to run).
Quick update on the tutorial so far: I’m deploying a tutorial satellite in a few days, some part of tutorial required real knowledge of interaction between satellite/storagenode/uplink.
Zero-downtime deployment meaning: no request will be drop during transition between old software version and newer software version.
How to achieve that?
There are many ways to do that, today, I’m using a method supported by Caddy web server (as reverse proxy).
The main idea is this: Caddy support graceful reload config, when you reload config, old request still being served to old backend, new request will be served by new backend, no request dropped during the change.
But here is the trick: every step while executing our plan need to be power outage proof, otherwise if something happen during this and the system will be in an inconsistent state.
Step by step idea:
upload green binary (as in blue/green deployment)
enable and start green service (and ensure it work)
symlink caddy config to green config
reload caddy config
copy green binary to a temp file then move tmp file to replace normal binary
restart normal service
symlink caddy config back to normal
reload caddy
disable and stop green service
It is important that we use move and symlink instead of copy (not an atomic operation). Also, move only atomic if on the same filesystem.
If power outage happen, you would be fine, just run deployment script again.
Note that, green service is should be on different port with normal service, if normal service is on port 8080, green should be on 8081 for example.
And green service should be disable by default, only enable (start together with operating system) normal service.
If you don’t understand any step, feel free to ask or suggest me with better idea.
I dabble with it a little then move on and now forgot almost all of it, right now, it is not enough time for me to relearn it and finish this tutorial, I estimate it would take me 2-3 months to get familiar with k8s ecosystem and know the rope, so this is it for now. Anyone know how satellite work can improvise deployment to their liking.