P U P P E T


    http://docs.puppetlabs.com/


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
What is Puppet?

* Language to define how machines should be configured
* The programs that apply that definition

Make sure that:
* packages are/are not installed
* config files have the correct contents
* services are/are not running
* ...


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Why should you use Puppet?

* Automation
* Documentation
* Structure
* Reuse
* Test installations
* Re-installations


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Compared to:	rdist/rsync + ssh/pdsh

- Only affects machines that are up.
- No traceability or cloneability.  You still don't know
  how the machines are or should be configured.


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Compared to:	Re-install machines from master image

- Requires downtime
- In its purest form, only works for identical machines.
  Must be supplemented with scripting if the machines
  differ.
- Gives cloneability, but not traceability.  You can
  build identical machines, but it doesn't help if you
  want to change OS version or OS distribution.
- Must take care to not re-install over important data.
+ Gives a guaranteed identical state.


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Compared to:	Re-install with a long %post in kickstart

- Requires downtime
- Not portable: every OS distribution has its own
  kickstart equivalent.
- One kickstart file per machine type; can be tricky to
  keep all the kickstart files in sync with each other.
- Must take care to not re-install over important data.
+ Gives a guaranteed identical state.
+ Gives you both cloneability and traceability.


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Compared to:	Document

- Boring.
- Easy to forget writing down some details, to write too
  vague descriptions, no time to do it "right now", ...
- Still lots of manual work to clone.
- Easy to make mistakes when following the documentation.
+ Often easier and more natural to describe not only what,
  but also why.
± Can give you a gold star with your PHB.


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Concepts

* Idempotency
* Convergence

* Declarative language, not imperative


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/secret.txt':
	owner => 'root', group => 'wheel', mode => 0600;
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/nsc':
	ensure => directory;
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/sysconfig/selinux':
	content => "SELINUX=permissive\nSELINUXTYPE=targeted\n";
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/ntp.conf':
	source => '/config/files/ntp.conf';
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
type {
    title_a:
	param_1 => value_1a,
	param_2 => value_2a,
	...;
    title_b:
	param_1 => value_1b,
	param_2 => value_2b,
	...;
    ...
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/secret.txt':
	owner => 'root', group => 'root', mode => 0600;
    '/etc/nsc':
	ensure => directory;
    '/etc/sysconfig/selinux':
	content => "SELINUX=permissive\nSELINUXTYPE=targeted\n";
    '/etc/ntp.conf':
	source => '/config/files/ntp.conf', mode => 0444;
    '/etc/rndc.key':
	ensure => link,
	target => '/var/named/chroot/etc/rndc.key';
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
package {
    'ntp':
	ensure => installed;
    'dhcp':
	ensure => absent;
    'torque':
	ensure => installed, provider => rpm,
	source => '/pkg/local/torque-2.2.1-1cri.x86_64.rpm';
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
yumrepo {
    'smokerings':
	descr => 'Smokerings local repository',
	baseurl => 'file:///pkg/local/smokerings/x86_64/',
	gpgcheck => 0, enabled => 1;
    'nordugrid':
	descr => 'NorduGrid - stable',
	baseurl => 'http://ftp.nordugrid.org/repos/redhat/el5/x86_64/stable',
	gpgkey => 'http://ftp.nordugrid.org/RPM-GPG-KEY-nordugrid',
	gpgcheck => 1, enabled => 1;
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
service {
    'dhcpd':
	enable => false, ensure => stopped;
    'ntpd':
	enable => true, ensure => running;
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
* augeas       * package              * nagios_command
* cron         * resources            * nagios_contact
* exec         * schedule             * nagios_contactgroup
* file         * service              * nagios_host
* filebucket   * ssh_authorized_key   * nagios_hostescalation
* group        * sshkey               * nagios_hostextinfo
* host         * tidy                 * nagios_hostgroup
* k5login      * user                 * nagios_hostgroupescalation
* mailalias    * yumrepo              * nagios_service
* maillist     * zone                 * nagios_servicedependency
* mount        * zfs                  * nagios_serviceescalation
* notify       * zpool                * nagios_serviceextinfo
                                      * nagios_servicegroup
                                      * nagios_timeperiod


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
package {
    'ntp':
	ensure => installed;

}
file {
    '/etc/ntp.conf':
	source => '/config/files/ntp.conf',
	require => Package['ntp'];
}
service {
    'ntpd':
	enable => true, ensure => running,
	subscribe => File['/etc/ntp.conf'];
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
package {
    'ntp':
	ensure => installed,
	before => File['/etc/ntp.conf'];
}
file {
    '/etc/ntp.conf':
	source => '/config/files/ntp.conf',
	notify => Service['ntpd'];
}
service {
    'ntpd':
	enable => true, ensure => running;

}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
class ntp-generic
{
    file {
	'/etc/ntp':  # Ntpd needs to be able to store /etc/ntp/drift.
	    ensure => directory, owner => 'ntp', group => 'ntp', mode => 0755;
    }
    package { 'ntp': ensure => installed; }
    service { 'ntpd': enable => true, ensure => running; }
}

class timeserver
{
    include ntp-generic
    file { '/etc/ntp.conf': source => '/config/files/ntp.conf-server'; }
}

class timeclient
{
    include ntp-generic
    file { '/etc/ntp.conf': source => '/config/files/ntp.conf-client'; }
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
node armstrong
{
    include timeserver
    file { '/etc/nodetype': content => "system-server\n"; }
}

node goodman, 'miller.smokerings.nsc.liu.se'
{
    include timeclient
    file { '/etc/nodetype': content => "system-server\n"; }
}

node  n1,  n2,	n3,  n4,  n5,  n6,  n7,	 n8,  n9, n10, n11, n12,
     n13, n14, n15, n16, n17, n18, n19, n20, n21, n22, n23, n24,
     n25, n26, n27, n28, n29, n30, n31, n32, n33, n34, n35, n36,
     n37, n38, n39, n40, n41, n42, n43, n44, n45, n46, n47, n48
{
    include timeclient
    file { '/etc/nodetype': content => "computenode\n"; }
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/var/named/chroot/etc/named.conf':
	source => '/config/Files/named.conf';
    '/var/named/chroot/etc/rndc.conf':
	source => '/config/Files/rndc.conf';
    '/var/named/chroot/var/named/192-168-116.zone':
	source => '/config/Files/192-168-116.zone';
    '/var/named/chroot/var/named/192-168-117.zone':
	source => '/config/Files/192-168-117.zone';
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
$pupfiles = '/config/Files'
$namedroot = '/var/named/chroot/'
$namedetc = "${namedroot}etc"
$zonedir = "${namedroot}var/named"

file {
    "$namedetc/named.conf":
	source => "$pupfiles/named.conf";
    "$namedetc/rndc.conf":
	source => "$pupfiles/rndc.conf";
    "$zonedir/192-168-116.zone":
	source => "$pupfiles/192-168-116.zone";
    "$zonedir/192-168-117.zone":
	source => "$pupfiles/192-168-117.zone";
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
file {
    '/etc/smurf.conf':
	source => ".../smurf.conf-${operatingsystem}";
    '/etc/gnapp.conf':
	source => ".../gnapp.conf-${hostname}";
    '/usr/bin/gargamel':
	ensure => ".../gargamel.${architecture}";
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
$bindpkg = $operatingsystem ? {
    'Fedora' => ['bind', 'bind-chroot'],
    'Gentoo' => ['net-dns/bind'],
}
package {
    'bindpackage':
	name => $bindpkg, ensure => installed;
}
service {
    'named':
	enable => true, ensure => running,
	require => Package['bindpackage'];
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
case $operatingsystem {
    'CentOS', 'Fedora': {
	package {
	    'bind': ensure => installed;
	    'bind-chroot': ensure => installed;
	}
	service {
	    'named':
		enable => true, ensure => running,
		require => [ Package['bind'], Package['bind-chroot'] ];
	}
    }
    'Gentoo': {
	package {
	    'net-dns/bind': ensure => installed;
	}
	service {
	    'named':
		enable => true, ensure => running,
		require => Package['net-dns/bind'];
	}
    }
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
import "util.pp"
import "dns.pp"
import "rootuser.pp"
import "osfixes.pp"
import "puppet.pp"
import "filesystems.pp"
import "network.pp"
import "time.pp"
import "installation.pp"
import "cluster.pp"

import "manifests/*.pp"



¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
$cluster_domain = 'smokerings.nsc.liu.se'
$n_computenodes = 66

file {
    '/etc/hosts':
	ensure => file,
	content => template("$pupfiles/hosts.erb");
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
127.0.0.1	localhost
::1		localhost6

192.168.116.1	armstrong.<%= @cluster_domain %>	armstrong
192.168.116.2	goodman.<%= @cluster_domain %>		goodman
192.168.116.3	miller.<%= @cluster_domain %>		miller
192.168.116.4	ellington.<%= @cluster_domain %>	ellington
192.168.116.5	basie.<%= @cluster_domain %>		basie

# 192.168.116.129-200 compute nodes
<% 1.upto(@n_computenodes.to_i) do |i| -%>
192.168.116.<%= i+128 %>	n<%= i %>.<%= @cluster_domain %>
<% end -%>


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
* alert		* fqdn_rand()		* search
* crit		* generate()		* sha1()
* debug		* include		* sprintf()
* defined()	* info			* tag
* emerg		* inline_template()	* tagged()
* err		* notice		* template()
* fail		* realize		* warning
* file()	* regsubst()


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
# Mirror a directory tree using rsync.
#
# The key in the file $sshkey must not be encrypted.

define rsync_mirror($source, $target, $sshkey='', $unless='',
		    $onlyif='')
{
    if $sshkey {
	$sshcmd = "ssh -i${sshkey}"
    } else {
	$sshcmd = 'ssh'
    }

    exec {
       "rsync-${target}":
	  command => "rsync -aRqv --no-implied-dirs --delete --delete-excluded '${source}' '${target}'",
	  env => "RSYNC_RSH=${sshcmd}",
	  path => '/bin:/usr/bin',
	  unless => $unless ? { '' => undef, default => $unless },
	  onlyif => $onlyif ? { '' => undef, default => $onlyif };
    }
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Extensions ("plugins")

* Your own resource types
* Your own functions
* Your own facts

* Plugins are written in Ruby


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Client-server

* 'puppet master' on a server
* 'puppet agent' on all clients
* Encrypted connections (TLS)
* Two-way authentication using X.509 certificates
* Functions evaluated on server (during compilation)
* Resources applied by client.

file {
    '/etc/ntp.conf':
	source => 'puppet:///timekeeping/ntp.conf';
}


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Other similar tools:

- CFEngine 2	- "Grandfather" of config management
- CFEngine 3	- Total rewrite, new language
- Quattor	- Was used by CERN
- Bcfg2		- XML-based config
- Chef		- Uses Ruby as its DSL


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Good and bad points about Puppet

- Documentation is of varying quality
+ "High level operations"
- File editing is complicated
+ Macros ("definitions")
+ Extendable (in Ruby)
+ Powerful templating for file contents
+ Active development
- Active development - things change
- Somewhat slow


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Pitfalls:

* Avoid global variables
* Scoping rules have changed 2.6 -> 3.0
* Be careful about parametrized classes


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯


	Usage at NSC



¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Systems using Puppet:

- Internal servers
- External servers (Babylon)
- Smokerings (grid compute cluster)
- Serenity (grid storage)
- Skylord (old SAAB cluster)
- Starkiller/Skywalker (new SAAB cluster)


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
* No manual config; everything managed by Puppet
  - With some exceptions:
    o Storage provisioning
    o ssh host keys, certs, TSM passwords

* Backup are for wimps; just re-install from scratch with
  kickstart
  - Kickstart files for VM guests generated by Puppet

* Except for data...
  - Decide what is data and what is config
    User homedirs, web pages, DNS zones, DHCP machine lists, ...

* Separate data onto separate filesystems, so a re-install
  does not overwrite important data
  E.g. /var/lib/mysql, /var/named/chroot/var/named, /home


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Environments

Each sysadmin has her own "environment" where she can develop
manifests and test them, independently of others
- /config/e/bellman
- /config/e/jens
(Some of us have more than one environment...)

When finished, push to central Git repository
- /config/git/puppet.git

Then pull to production, /config/e/production


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Wrapper script 'runpuppet', run from cron every 4 hours.

   # runpuppet env=bellman

   # runpuppet env=bellman --noop


¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
/config/private/${hostname}

* ssh host keys
* TSM passwords
* Host certificates
* Some other host-specific configs
  - Flexlm license files
  - email alias files
  - wiki user lists
  - htpasswd files