ZFS, booting from

Note: THIS ARTICLE IS FOR 9.0-RELEASE AND UP! If you're using FreeBSD 7.x or 8.x, please see ZFS, booting from (pre 9.0-RELEASE).

Installing FreeBSD Root on ZFS (Mirror) using GPT

Creating a bootable ZFS Filesystem

Boot FreeBSD install DVD or USB Memstick, and choose Install. Choose your keyboard mapping, hostname, and distribution components (doc, games, lib32, ports, and src - I'd highly advise you to leave ports selected!) as normal. At the Partitioning stage, select <Shell>.

Create GPT Disks

In this stage we'll assume you have two drives, ada0 and ada1 - check your device names (ls /dev/ada*) and adjust as appropriate, and of course if you have more drives, copy these commands for those drives as well.

# gpart create -s gpt ada0
# gpart create -s gpt ada1

Create the boot, swap and zfs partitions

Create 3 partitions on both drives ada0 and ada1. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (20GB). NOTE: you'll greatly decrease your chance of fat-finger mistakes if for each new command, you use the up-arrow key to duplicate the previous line, then just edit the parameters as appropriate!

# gpart add -b 34 -s 128 -t freebsd-boot -l boot0 ada0
# gpart add -b 34 -s 128 -t freebsd-boot -l boot1 ada1
# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap0 ada0
# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap1 ada1
# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root0 ada0
# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root1 ada1

Notes: 1. While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume. 2. Sizes and offsets are specified in sectors (1 sector is typically 512 bytes). 3. You may issue a gpart show command to see the correct location and size for further partitions you might need.

Install the Protected MBR (pmbr) and gptzfsboot loader to both drives

# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1

This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:

# sysctl kern.geom.debugflags=0x10

Create the root ZFS Pool

# zpool create -o altroot=/mnt zroot mirror gpt/root0 gpt/root1
# zpool set bootfs=zroot zroot

Install FreeBSD to zroot

Optional: Create ZFS datasets specifically for a FreeBSD system

If you wish, you may set up ZFS datasets within the zroot pool specifically optimized for the FreeBSD system.

I wish to stress that this is optional, and many admins may prefer to simply install to the single 20GB zroot pool already set up - among other things, following the guide linked to will cause sixteen new filesystems to show up in the output of df!

If this sounds like more trouble than it's worth, skip this step completely and move on.

Optional: Change the checksum algorithm

The fletcher4 algorithm should be more robust than the fletcher2 algorithm.

# zfs set checksum=fletcher4 zroot

Installing the system

# exit

Now you'll be back in the normal installer, and it will start decompressing and copying files as usual. This shouldn't take too long. Enter a root password as prompted. Configure your network interface(s) and system time as usual. Choose your system boot services (usually sshd and ntpd). Enable crash dumps (recommended). Add a user account if you want to. Now, back at the "Final Configuration" menu, Exit. Now at "Manual Configuration", when asked if you would like to open a shell to make any final manual modifications, select No. This is a lie, but it's a lie you have to tell! NOW, at the "Complete" menu, when asked "Would you like to reboot into the installed system now?" select <Live CD>.

Log in as root - you will not be prompted for a password - and make some necessary final configuration changes. Your system will not boot if you forget these steps!

Initial configuration of the new system

configure /etc/rc.conf

hostname# echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf

configure /boot/loader.conf

hostname# echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
hostname# echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf

Check your loader.conf, and make CERTAIN you the quotes in "zfs:zroot" showed up - your system will not boot if they're not there!

Install zpool.cache to the ZFS filesystem

hostname# zpool export zroot
hostname# zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot
hostname# cp /tmp/zpool.cache /mnt/boot/zfs/

Finishing the installation

Using swap

Create /etc/fstab to use both swap partitions

hostname# cat << EOF > /mnt/etc/fstab
# Device                       Mountpoint              FStype  Options         Dump    Pass#
/dev/gpt/swap0                 none                    swap    sw              0       0
/dev/gpt/swap1                 none                    swap    sw              0       0
EOF

Reboot into your new system

hostname# reboot

The system will now reboot into your new installation. Enjoy!