Bare Metal Chef

How to automate your hardware

by Maxime Brugidou / @brugidou

Clouds

knife ec2 server create

Prerequisites

Network
Firmwares
BMC/IPMI

Chicken-Egg Problems

How do we install the first server?

Chef Server

DHCP


# Generated by Chef with DHCP cookbook
subnet 10.10.0.0 netmask 255.255.255.0 {
  next-server 10.10.0.3;
  option domain-name-servers 10.10.1.1, 10.10.1.2;
  option routers 10.10.0.1;
  range 10.10.0.128 10.10.0.255;
}

Initial Diskless Image

Built with Debirf. Modified a bit to support:

CentOS images using Mock
rootmemsize kernel arg to limit the Diskless memory to 2GB
firstboot_script kernel arg to run script after boot

PXE Boot


# TFTP cookbook
default['tftp']['pxelinux.cfg']['default']['netboot'] = {
  'kernel' => node['tftp']['files']['netboot_kernel']['name'],
  'append' => "initrd=#{node['tftp']['files']['initrd']['name']} 
  rootmemsize=2097152k console=tty0 console=ttyS1,9600n8 
  firstboot_script=tftp://#{node[:ipaddress]}/default/setup.sh"
}

Default Hostname

MAC Addresses are unique identifiers and deterministic. Example: 6c-3b-e5-a8-0f-34.test.org


cat <(cat /sys/class/net/*/address) \
<(cat /proc/net/bonding/* 2>/dev/null |\
    /bin/grep "Permanent HW addr:" |\
    /bin/awk '{print $4}') |\
/bin/grep -v 00:00:00:00:00:00 |\
/bin/sort | /usr/bin/uniq  |\
/usr/bin/head -n1 | /usr/bin/tr [A-Z] [a-z] |\
/usr/bin/tr : -

Firstboot Script

Install Chef
Register Node
Set run_list to role[firstboot]

Discover New Hardware


$ knife status 'roles:firstboot'
33 minutes ago, 2c-59-e5-3b-60-c4.test.org, 2c-59-e5-3b-60-c4.test.org, 10.12.160.74, centos 6.4.
31 minutes ago, 2c-59-e5-47-84-0c.test.org, 2c-59-e5-47-84-0c.test.org, 10.110.160.40, centos 6.4.
30 minutes ago, 74-46-a0-f4-1e-0c.test.org, 74-46-a0-f4-1e-0c.test.org, 10.110.160.54, centos 6.4.
30 minutes ago, f0-92-1c-10-17-04.test.org, f0-92-1c-10-17-04.test.org, 10.10.160.46, centos 6.4.
30 minutes ago, f0-92-1c-10-5a-b4.test.org, f0-92-1c-10-5a-b4.test.org, 10.12.160.79, centos 6.4.
...

Provision a Node

Just give it a run_list!

RAID Controller


# raid cookbook
default[:raid][:config][:arrays] = [
  {
    :raid => "1+0",
    :disk_count => 8,
    :disk_size => "600 G",
    :disk_type => "SAS"
  },
  {
    :raid => "1"
    :disk_count => 2,
    :disk_size => "200 G",
    :disk_type => "Solid State SATA"
  }
]

Disk Partitioning


# os cookbook
# fai-setup-storage syntax
default['os_install']['disks']['/dev/sda']['options'] = 'fstabkey:label disklabel:gpt-bios bootable:1'
default['os_install']['disks']['/dev/sda']['volume'] = [
  {
    :type => 'primary',
    :mountpoint => "/mnt/os_install",
    :size => '20GB-',
    :filesystem => 'ext4',
    :mount_options => 'defaults',
    :fs_options => "createopts=\"-L ROOT -b 4096 -m 5 -O extent\""
  },
  ...
]

FAI Setup-Storage

Part of the Debian FAI Project
Works from our CentOS diskless image
Can do anything (LVM, mdadm, ntfs...)
We use it to partition and format on all platforms

OS On-Disk Installation


# os cookbook
default['os_install']['diskless'] = false
# supports: debian, centos, windows
default['os_install']['platform'] = 'centos'