Revision as of 10:16, 13 August 2012

Note: this article was taken very nearly verbatim from Scott Hetzel's excellent guide at http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror.

Installing FreeBSD Root on ZFS (Mirror) using GPT

Creating a bootable ZFS Filesystem

Boot FreeBSD install DVD or USB Memstick, and choose the Fixit option in sysinstall.

Create GPT Disks

Fixit# gpart create -s gpt ad0
Fixit# gpart create -s gpt ad1

Create the boot, swap and zfs partitions

Create 3 partitions on both drives ad0 and ad1. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (20GB).

Fixit# gpart add -b 34 -s 128 -t freebsd-boot -l boot0 ad0
Fixit# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap0 ad0
Fixit# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root0 ad0

Fixit# gpart add -b 34 -s 128 -t freebsd-boot -l boot1 ad1
Fixit# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap1 ad1
Fixit# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root1 ad1

Notes: 1. While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume. 2. Sizes and offsets are specified in sectors (1 sector is typically 512 bytes). 3. You may issue a gpart show command to see the correct location and size for further partitions you might need.

Install the Protected MBR (pmbr) and gptzfsboot loader to both drives

Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad0
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad1

This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:

Fixit# sysctl kern.geom.debugflags=0x10

Create the root ZFS Pool

Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
Fixit# kldload /mnt2/boot/kernel/zfs.ko
Fixit# mkdir /boot/zfs
Fixit# zpool create zroot mirror /dev/gpt/root0 /dev/gpt/root1
Fixit# zpool set bootfs=zroot zroot

Install FreeBSD to zroot

Optional: Create ZFS datasets specifically for a FreeBSD system

If you wish, you may set up ZFS datasets within the zroot pool specifically optimized for the FreeBSD system.

I wish to stress that this is optional, and many admins may prefer to simply install to the single 20GB zroot pool already set up - among other things, following the guide linked to will cause sixteen new filesystems to show up in the output of df!

If this sounds like more trouble than it's worth, skip this step completely and move on.

Optional: Change the checksum algorithm

The fletcher4 algorithm should be more robust than the fletcher2 algorithm.

Fixit# zfs set checksum=fletcher4 zroot

Installing the system

Fixit# cd /zroot
Fixit# cd /dist/8.0-*
Fixit# export DESTDIR=/zroot
Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
do (cd $dir ; ./install.sh) ; done
Fixit# cd src ; ./install.sh all
Fixit# cd ../kernels ; ./install.sh generic
Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/

Initial configuration of the new system

chroot into the new system

Fixit# chroot /zroot

create a /home directory

If you didn't create a dataset for home in the optional steps above, you'll need to create a directory for it now.

Fixit# mkdir /usr/home
Fixit# ln -s /usr/home home

configure /etc/rc.conf

Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf
Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf
Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf

Note: Replace re0 with the name of the Network interface for the new system

configure /boot/loader.conf

Fixit# echo 'zfs_load="YES"' > /boot/loader.conf
Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf

Check your loader.conf, and make CERTAIN you the quotes in "zfs:zroot" showed up - your system will not boot if they're not there!

Change root's password

Fixit# passwd

Set the local time zone

Fixit# tzsetup

Create /etc/mail/aliases.db

Fixit# cd /etc/mail
Fixit# make aliases

Installing ZFS aware /boot/loader (Required for 8.0-RELEASE and 7.{0-2}-RELEASE)

Note: This step is obsoleted in FreeBSD 8.0-STABLE, FreeBSD 9.0-CURRENT, FreeBSD 7.2-STABLE, and newer.

Fixit# echo 'LOADER_ZFS_SUPPORT=YES' > /etc/src.conf
Fixit# mount -t devfs devfs /dev
Fixit# export DESTDIR=""
Fixit# cd /usr/src/sys/boot/
Fixit# make obj
Fixit# make depend
Fixit# make
Fixit# cd i386/loader
Fixit# make install
Fixit# umount /dev

Exit from the /zroot

Fixit# exit

Install zpool.cache to the ZFS filesystem

Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

Finishing the installation

Using swap

There are 2 ways to use the gpt/swap0 and gpt/swap1 partitions. Only choose one!

Create /etc/fstab to use both swap partitions

Fixit# cat << EOF > /zroot/etc/fstab
# Device                       Mountpoint              FStype  Options         Dump    Pass#
/dev/gpt/swap0                 none                    swap    sw              0       0
/dev/gpt/swap1                 none                    swap    sw              0       0
EOF

Use gmirror to mirror the swap partitions

Fixit# kldload /mnt2/boot/kernel/geom_mirror.ko
Fixit# gmirror label -b prefer swap gpt/swap0 gpt/swap1

Note: The 'prefer' balance algorithm can be replaced by 'round-robin'. See the gmirror(8) man page about problem using the 'round-robin' balance algorithm and kernel dumps

Fixit# cat << EOF > /zroot/etc/fstab
# Device                       Mountpoint              FStype  Options         Dump    Pass#
/dev/mirror/swap                       none                    swap    sw              0       0
EOF

 Fixit# echo 'geom_mirror_load="YES"' >> /zroot/boot/loader.conf

Setting mountpoints before first boot

Fixit# export LD_LIBRARY_PATH=/mnt2/lib 
Fixit# cd /
Fixit# zfs unmount -a
Fixit# zfs set mountpoint=legacy zroot

If you set up the full hierarchy of ZFS datasets for the system in the optional steps above, you need to set their mountpoints as well:

Fixit# zfs set mountpoint=/tmp zroot/tmp
Fixit# zfs set mountpoint=/usr zroot/usr
Fixit# zfs set mountpoint=/var zroot/var

Exit Fixit mode and sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.