ZFS, booting from (pre 9.0-RELEASE)
Note: this article was taken very nearly verbatim from Scott Hetzel's excellent guide at [1].
Installing FreeBSD Root on ZFS (Mirror) using GPT
Creating a bootable ZFS Filesystem
Boot FreeBSD install DVD or USB Memstick, and choose the Fixit option in sysinstall.
Create GPT Disks
Fixit# gpart create -s gpt ad0 Fixit# gpart create -s gpt ad1
Create the boot, swap and zfs partitions
Create 3 partitions on both drives ad0 and ad1. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (20GB).
Fixit# gpart add -b 34 -s 128 -t freebsd-boot ad0 Fixit# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap0 ad0 Fixit# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root0 ad0 Fixit# gpart add -b 34 -s 128 -t freebsd-boot ad1 Fixit# gpart add -b 162 -s 8388608 -t freebsd-swap -l swap1 ad1 Fixit# gpart add -b 8388770 -s 41943040 -t freebsd-zfs -l root1 ad1
Notes: 1. While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume. 2. Sizes and offsets are specified in sectors (1 sector is typically 512 bytes). 3. You may issue a gpart show command to see the correct location and size for further partitions you might need.
Install the Protected MBR (pmbr) and gptzfsboot loader to both drives
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad0 Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad1
This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:
Fixit# sysctl kern.geom.debugflags=0x10
Load ZFS kernel module
Fixit# kldload /mnt2/boot/kernel/opensolaris.ko Fixit# kldload /mnt2/boot/kernel/zfs.ko
Create ZFS Pool zroot
Fixit# mkdir /boot/zfs Fixit# zpool create zroot mirror /dev/gpt/disk0 /dev/gpt/disk1 Fixit# zpool set bootfs=zroot zroot
Optimizing the ZFS filesystem for a FreeBSD system
NOTE: these steps are entirely optional. You may choose to omit them entirely and simply run FreeBSD from a single 20GB pool if you wish.
Changing the checksum algorithm
The fletcher4 algorithm should be more robust than the fletcher2 algorithm.
Fixit# zfs set checksum=fletcher4 zroot
Creating ZFS datasets for the system
Fixit# zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp Fixit# chmod 1777 /zroot/tmp Fixit# zfs create zroot/usr Fixit# zfs create zroot/usr/home Fixit# zfs create -o compression=lzjb -o setuid=off zroot/usr/ports Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
Note: If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.
Fixit# zfs create zroot/var Fixit# zfs create -o exec=off -o setuid=off zroot/var/empty Fixit# zfs create -o exec=off -o setuid=off zroot/var/db Fixit# zfs create -o exec=off -o setuid=off zroot/var/run Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log Fixit# zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp Fixit# chmod 1777 /zroot/var/tmp
Notes: Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compresion ratio. gzip is equivalent to gzip-6). Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.
Install FreeBSD to zroot
Creating home directory
If you didn't create a dataset for home in the optional steps above, you'll need to create a directory for it now.
Fixit# mkdir /usr/home Fixit# ln -s /usr/home home
Installing the system
Fixit# cd /zroot Fixit# cd /dist/8.0-* Fixit# export DESTDIR=/zroot Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \ do (cd $dir ; ./install.sh) ; done Fixit# cd src ; ./install.sh all Fixit# cd ../kernels ; ./install.sh generic Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/
Initial configuration of the new system
chroot into the new system
Fixit# chroot /zroot
configure /etc/rc.conf
Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf
Note: Replace re0 with the name of the Network interface for the new system
configure /boot/loader.conf
Fixit# echo 'zfs_load="YES"' > /boot/loader.conf Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
Change root's password
Fixit# passwd
Set the local time zone
Fixit# tzsetup
Create /etc/mail/aliases.db
Fixit# cd /etc/mail Fixit# make aliases
Installing ZFS aware /boot/loader (Required for 8.0-RELEASE and 7.{0-2}-RELEASE)
Note: This step is obsoleted in FreeBSD 8.0-STABLE, FreeBSD 9.0-CURRENT, FreeBSD 7.2-STABLE, and newer.
Fixit# echo 'LOADER_ZFS_SUPPORT=YES' > /etc/src.conf Fixit# mount -t devfs devfs /dev Fixit# export DESTDIR="" Fixit# cd /usr/src/sys/boot/ Fixit# make obj Fixit# make depend Fixit# make Fixit# cd i386/loader Fixit# make install
Exit from the /zroot
Fixit# umount /dev Fixit# exit
Install zpool.cache to the ZFS filesystem
Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache
Finishing the installation
Using swap
There are 2 ways to use the gpt/swap0 and gpt/swap1 partitions. Only choose one!
Create /etc/fstab to use both swap partitions
Fixit# cat << EOF > /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0 EOF
Use gmirror to mirror the swap partitions
Fixit# kldload /mnt2/boot/kernel/geom_mirror.ko Fixit# gmirror label -b prefer swap gpt/swap0 gpt/swap1
Note: The 'prefer' balance algorithm can be replaced by 'round-robin'. See the gmirror(8) man page about problem using the 'round-robin' balance algorithm and kernel dumps
Fixit# cat << EOF > /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/swap none swap sw 0 0 EOF
Fixit# echo 'geom_mirror_load="YES"' >> /zroot/boot/loader.conf
Shutting down and booting into the new system
Fixit# export LD_LIBRARY_PATH=/mnt2/lib Fixit# zfs unmount -a Fixit# zfs set mountpoint=legacy zroot
If you set up the full hierarchy of ZFS datasets for the system in the optional steps above, you need to set their mountpoints as well:
Fixit# zfs set mountpoint=/tmp zroot/tmp Fixit# zfs set mountpoint=/usr zroot/usr Fixit# zfs set mountpoint=/var zroot/var
Exit Fixit mode and sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.
