1. Ingeniería

Deploying CoreOS with Foreman
Johannes Maximilian Scheuermann, Nils
Domrose
Gent , 02.02.2015
Agenda
‣ About us
‣ CoreOS
‣ CoreOS components
‣ How to deploy CoreOS with pre 1.8 foreman
‣ Building a POC
‣ Create a PR
‣ Deploying a CoreOS Cluster
About us
A brief introduction
Nils:
● Senior Linux Systems Engineer / Architect at inovex GmbH
http://github.com/endyman
IRC @__endy__
Johannes:
● Computer Science Student working at inovex GmbH
http://github.com/johscheuer
IRC @johscheuer
CoreOS
Linux for Massive Server Deployments
●
●
●
●
●
●
●
●
●
ChromeOS Fork
Minimal Linux – uses ~40% less RAM
Painless update – update OS as single unit
Docker Container
Clustered By Default
Cluster Management with fleet
fleet - a distributed init system
Service Discovery with etcd
Systemd as init-system
image: https://coreos.com/assets/images/media/Host-Diagram.png
CoreOS Components
How is CoreOS shipped
●
●
●
coreos_production_pxe_image.cpio.gz (initrd)
coreos_production_pxe.vmlinuz (kernel)
cloud-config (cloudinit)
●
●
●
●
PXE boot file has to provide a cloud-config
Install options (in memory, disk)
Boot options (images, iso, PXE)
coreos.autologin opens a Shell without prompting for a password ->
troubleshooting
cloud-config-url for provisioning
Post-Boot Actions via systemd units defined in cloud-config
●
●
CoreOS Components
How is CoreOS configured
●
●
●
●
●
Cloud-Config is used to configure CoreOS during each boot
○ Invalid Cloud-Configs won’t be processed but will be logged in the
journal -> sadly you can’t access the journal
CoreOS validater https://coreos.com/validate is a helpful tool to check your
Cloud-Config syntax
YAML-Style
Has to start with #cloud-config or #!
Example:
#cloud-config
coreos:
etcd:
name: node001
discovery: https://discovery.etcd.io/<token>
addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001
●
Pass Cloud-Config as parameter e.g. PXE Boot cloud-config-url or in
foreman as provision template
source: https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config
How to deploy CoreOS with pre 1.8 foreman
●
●
●
●
Create a Dummy OS
Select Red-Hat as OS family
manually downloaded kernel + ram disk to your TFTP Servers /boot dir
create custom templates for your cloud configs
Drawbacks:
now proper versioning, no auto download of coreos, no out-of-the box
templates….
Building a POC
...in 4 hours
●
●
●
●
●
●
●
➔
➔
➔
➔
look at other OS implementation in the source
play around with current foreman installation
modified the source on disk
challenge to get information passed to the CoreOS installer script
developed nested cloud config concept
fun to get patches out :-)
started using git with proper remotes at this point
CoreOS downloaded from the inet (everytime)
Full auto install (with minor glitches)
ssh access
deployment of multi nodes available in fleet (fleetctl list-machines w/o
manual steps)
create Patches and PR
●
●
●
created patch from POC
applied Patch to fork
created external templates for Patch
●
PR for foreman
○ created app/models/operatingsystems/coreos.rb
○ added CoreOS on multiple places in foreman
○ added unit test for CoreOS
○ added CoreOS logo after triggering the CoreOS Team to add License
PR for communinty-templates
○ provision.erb (cloud-config)
○ PXELinux.erb (PXE boot)
○ disklayout_CoreOS.erb (not supported)
○ coreos_cloudconfig.erb (snippet, cloud-config applied at installation
PR for coreos :-)
○ allow to override the base-url
○ CoreOS created their own Patch based on the idea
●
●
optimizing the PR
getting the code in
●
●
●
●
comments
code style
re-factoring
template optimization
●
●
support for coreOS mirror
only in CoreOS > 557.0.0
Source: http://github.com
Deploying a CoreOS Cluster
CoreOS discovery 101
●
●
●
Tested with etcd 0.4.6, 2.0.0 release at 28.01 not yet in CoreOS
discovery token
○ get new token from https://discovery.etcd.io/new
○ set token in Cloud-Config
○ start cluster
○ + easy setup for tests/fast deployment
○ - needs internet access
○ - everybody could see the cluster at https://discovery.etcd.io/$token
○ - and could also connect to the cluster
(static) etcd node as entrypoint
○ start up an etcd node (entrypoint)
○ enter IP-Address of entrypoint into Cloud-Config as -peers
○ start up other etcd nodes
○ + only intern communication is necessary
○ - entrypoint has to be available if a new etcd node will join
Deploying a CoreOS Cluster
CoreOS discovery logical workflow
1.
if log data is found by etcd:
a. Try to join via peers in previous cluster
b. Try to join via peers found in discover URL
c. Try to join via peers in peer list
d. Restart the previous cluster which is down
e. return
2.
If discover URL is found:
a. Fetch peers through discover URL
b. If Success:
i. Join peers found
ii. return
3.
If peer list is given:
a. Join as follower via peers in peer list
b. return
4.
Start as the leader of a new Cluster
source: https://github.com/coreos/etcd/blob/v0.4.6/Documentation/design/cluster-finding.md
Deploying a CoreOS Cluster
●
Create a host group coreos_cluster
○ host param etcd_discovery_url for an easy setup with an discovery
URL
○ or create an etcd node as entrypoint
○ add install-disk as hosts parameter pointing to your target device
○ add ssh_authorized_keys as host param
■ if you forget this step the host root password will be deployed
for the core user
■ the CoreOS way is to work with SSH keys
Deploying a CoreOS Cluster
Deploying a CoreOS Cluster
Demo ?!
Deploying a CoreOS Cluster
●
Validate that the cluster is deployed
○ fleetctl list-machines -> list of all deployed CoreOS nodes
○ journalctl shows entire system log
○ journalctl -u test.service show system log for specific service
○ good for debugging
○ Quick debugging with toolbox -> installs a Fedora Docker container
What’s next
SkyDNS
… and many more simply via systemd-units in your
cloud-config template
Logos: http://deis.io, http://kubernetes.io, http://mesos.apache.org, http://mesossphere.com
Thank you for listening!
Kontakt
Nils Domrose
Johannes Maximilian Scheuermann
[email protected]
[email protected]
inovex GmbH
Office Karlsruhe
Ludwig-Erhard-Allee 6
76131 Karlsruhe