How to install FreeBSD 8.0 using only ZFS partitions? (Newbie proof, I hope)
Ok, so I want to install the newly released FreeBSD 8.0 and only have ZFS partitions.
Questions I had that I am answering here for you:
- Does Sysinstall support ZFS?
No. You have to install manually using Fixit.
- Can I boot off a ZFS partition?
Yes, you can. Takes some effort to set it up though.
- Is it easy?
No, it isn’t easy it is hard. Unless you have used FreeBSD for some time, then it is just tedious and not really easy or hard.
- Is there a guide?
Yes, it is here:
http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
But hopefully by the time I am done, the best guide will be this post.
Ok, so here it goes, I am following this guide: http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
I am going to improve upon it and try to make it newbie proof and not skip steps in my guide when this guide skips steps. Why am I making it newbie proof? I don’t know, you would think if you are doing ZFS you aren’t a newbie, but who knows. Better safe than sorry.
Steps for How to install FreeBSD 8.0 using only ZFS partitions?
Step 1. Creating a bootable ZFS Filesystem
- Download the FreeBSD 8 DVD ISO from here: http://www.freebsd.org/where.html
Ok, so this isn’t exactly a download link, but it takes you to where you choose your architecture. Click on the [ISO] link next to your Architecture type.
If you are a newbie and don’t know your architecture, you probably want i386. If you just bought a new 64 bit machine then you want amd64.
Ok, so the actually link you want is the one that looks as follows:
For amd64 – 8.0-RELEASE-amd64-dvd1.iso.gz
For i386 – 8.0-RELEASE-i386-dvd1.iso.gz - Extract it as it is zipped.
- Burn the ISO to DVD disk (or if you are using VMWare just point your VM’s CD-ROM at the ISO).
- Boot off the FreeBSD 8 DVD and let it boot all the way up.
- Choose your country/region.
You should now be at the Sysinstall Main Menu. Remember, we cannot use Sysinstall because it doesn’t yet support ZFS. (I am hoping for a new installer over fixing this old one.) - Choose Fixit. You will be promted to Please choose a fixit option.
- Choose CDROM/DVD.
You are now at a Fixit command prompt. And if you are following the wiki guide, you are only at that guide’s step 3. Create MBR disk - Create an MBR disk by running the following command:
gpart create -s mbr ad0
However, what the guide assumes you already know is that
ad0is the name of your hard drive and can be different for each installation. I am installing on VMWare using SCSI and the name of my hard drive isda0. So I would run:gpart create -s mbr da0You can find out your by looking at a directory listing of /dev if you are familiar with common names, otherwise, you can start a Standard install and see what name is used when you get to the Disk label editor section of the install.
I am going to use da0 or ad0 in the rest of my document interchangeably, so you need to remember to always use the correct disk name for your drive.
- Run the following command to show the information for your disk
gpart show ad0
There are two columns of numbers. The first column number is a start block for the slice and the second is the size in blocks for the slice. 63 is the start of every first slice. However, the end of a slice is variable depending on the size of your hard drive.
A slice is not exactly a partition. On FreeBSD you first create a Slice and then you create your partitions on your slice. The next steps will show you have to do both.
- Create a new Slice using the entire disk.
Obviously the guy who wrote the wiki I am following already had two slices for windows, so he must have been dual booting. I am assuming that you are not dual booting and that you are installing a new server and you plan to use the entire disk.
To create a slice using the entire disk, run this command replacing the value after the -s with the value you saw when you ran the previous command.
gpart add -b 63 -s 104857515 -t freebsd da0It will output something like “da0s1 added”. If you look at the da0s1 string, it makes sense. da0 (your disk name) s1 (slice 1 – the slice you just created).
- Now lets create our slice. No, I am not sure why you have to both add the slice and create the slice, but I am sure there is a good reason.
gpart create -s BSD da0s1
- Lets make our slice the active slice. Sounds like this is only sometimes needed. Better to do it and not need it than to not do it an find out you need it.
gpart set -a active -i 1 da0
You can run the gpart show da0 command again to make sure it is set as active.
- Look at the slice.
gpart show da0s1
Again, you will have two rows of numbers. This time the first number is 0 and the second number is the size of the slice.
We want at least two partitions, one for
/and one as a swap partition. So we need to determine how much space we want for theswappartition. I want 3 GB.Now we have to convert the desired size from GB to sectors.
1 kilobyte = 1024 bytes or 2 sectors (Sectors are normally 512 Bytes)
1 megabyte = 1024 kilobytes
1 gigabyte = 1024 megabytesSo to get the number of sectors in 1 GB, we need to use the following equation:
Gigabytes in Sectors = NumberOfGB * numberOfMBInAGB * NumberOfKBInAMB * NumberOfSectorsInAKB
1 GB in sectors = 1 * 1024 * 1024 * 2 = 2097152
3 GB in sectors = 3 * 1024 * 1024 * 2 = 6291456So take the total size of your slice in sectors and subtract 6291456 and you will have the size of your
/partition. And ourswappartition will be 6291456. - Create your
/partition.gpart add -i 1 -b 0 -s 98566059 -t freebsd-zfs da0s1 - Create the
swappartition.gpart add -i 2 -b 98566059 -s 6291456 -t freebsd-swap da0s1 - Load the ZFS kernel module.
kldload /mnt2/boot/kernel/opensolaris.ko
kldload /mnt2/boot/kernel/zfs.ko - Create your zpools.
mkdir /boot/zfs
zpool create zroot /dev/da0s1a
zpool set bootfs=zroot zroot - Install the boot manager.
gpart bootcode -b /mnt2/boot/boot0 da0
- Install ZFS boot.
zpool export zroot
dd if=/mnt2/boot/zfsboot of=/dev/da0s1 count=1
dd if=/mnt2/boot/zfsboot of=/dev/da0s1a skip=1 seek=1024
zpool import zroot
Yeah we are done with step 1. Stay tuned for step 2 and step 3 coming.
… I am back for round 2…er uh…step 2 that is.
Step 2. Installing FreeBSD to the ZFS filesystem
Ok for those of you who skipped the details above, I am reading this wiki:
http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
My intention is to make a more thorough and newbie proof version of this wiki. So here we go, diving into step 2.
- Create the ZFS hierarchy.
Wow, this is going to be a lot of tedious typing. You know, while FreeBSD didn’t make an installer for all this, how hard would it have been to create a couple of scripts and include them on the CD so this would be easier.
zfs set checksum=fletcher4 zroot
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
chmod 1777 /zroot/tmpzfs create zroot/usr
zfs create zroot/usr/home
cd zroot ; ln -s /usr/home homezfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packageszfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
zfs create zroot/var
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /zroot/var/tmpcd /dist/8.0-RELEASE
export DESTDIR=/zroot
for dir in base catpages dict doc games info lib32 manpages ports; \
do (cd $dir ; ./install.sh) ; done
cd src ; ./install.sh all
cd ../kernels ; ./install.sh generic
cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/
zfs set readonly=on zroot/var/emptychroot /zroot
echo ‘zfs_enable=”YES”‘ > /etc/rc.conf
echo ‘hostname=”zfs.mydomain.local”‘ >> /etc/rc.conf
echo ‘ifconfig_em0=”DHCP”‘ >> /etc/rc.confecho ‘zfs_load=”YES”‘ > /boot/loader.conf
echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.confecho ‘LOADER_ZFS_SUPPORT=YES’ > /etc/src.conf
mount -t devfs devfs /dev
export DESTDIR=””cd /usr/src/sys/boot/
make obj
make depend
make
cd i386/loader
make installpasswd
tzsetup
cd /etc/mail
make aliases
umount /dev
exit
cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache
Warning! There is only one line that might catch a newbie off-guard. Every other line you can type in as is but this one.
On FreeBSD this is how you setup your network card to use FreeBSD. However, while my card is em0, not all cards are em0. Run the ifconfig command on FreeBSD to see your card type and replace em0 with the type for you card.
Step 3 – Finish
I followed the guide almost exactly except I had to do a cd / before unmounting. So I added that command where it needs to be, so this should be very newbie proof.
- Run these commands.
cat < /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/ad0s3b none swap sw 0 0 EOF export LD_LIBRARY_PATH=/mnt2/lib cd / zfs unmount -a zfs set mountpoint=legacy zroot zfs set mountpoint=/tmp zroot/tmp zfs set mountpoint=/usr zroot/usr zfs set mountpoint=/var zroot/var
I made some mistakes but finally got it to work.
Some really select content on this website, saved to fav.
my blog post :: sell music online (liverpoolsu.com)
autozone auto parts online store truck parts
How to install FreeBSD 8.0 using only ZFS partitions? (Newbie proof, I hope) | Rhyous
john lewis advert girl growing up
How to install FreeBSD 8.0 using only ZFS partitions? (Newbie proof, I hope) | Rhyous
So to conclude about the Casio Gshock Mudman Solar Gw9000A-1.
In simple terms, you might be not likely to find another
line of watches that keeps to the same standard.
Casio has always been into high quality electronic goods.
You need to combine practicality and personal touch in the gifts.
People who tend to overpack their personal
effects are always caught up in this dilemma. Fortunately, these days
baby bags come in manly colors and designs.
This didn't work for me. I didn't understand the usage of "cat" in the script (nor the cryptic comments by users; my experience and experiments with the cat command tell me it can't work) and replaced that part by creating the (nonexistent) fstab file with two echo... lines.
Booting resulted in:
F1 FreeBSD
F6 PXE
Boot: F1
but no further progress.
Thanks anyway, jh
i get this far too, it seems somehow the bootloader isn't correctly installed. (after "F1 FREEBSD F6 PXE", you're supposed to get the list with boot normal, single user mode, etc to the left and the FreeBSD logo on the right, right?)
i followed all steps with no error, but in end I can't boot the system with no error showing, I test this in VirtualBox 3.2.12 with 8.2-RELEASE-AMD64.
To those didn't find lib32 directory, because this directory available only in AMD64 and IA64 I guess.
Great howto!
May could help someone, or improve this post:
Unfortunately, trying to install the 8.1-RELEASE got no ATA DVD drive in old PC whose have to be the home router and NAS. Also, no USB boot at BIOS. So, it's impossible to boot from DVD or memstick image.
Booted from Live FS CDROM, then, after creating pool (RAIDZ1) and ZFS hierarchy, tried to mount the memstick image on my 1Gb USB drive (/dev/da0a) to any dir under /mnt, then started extracting distributions. Kernel panic - low kmem, I have only 512Mb RAM on the box.
We brake for nobody!
So, next boot we set vm.kmem_size="330M"
vm.kmem_size_max="330M"
vfs.zfs.arc_max="40M"
vfs.zfs.vdev.cache.size="5M" at the loader prompt and then boot. (Don't forget to place that to the /boot/loader.conf).
And, after boot and diving to fixit shell, zpool import -f zroot to get where we were when crashed. Then, followed your manual, all goes like a charm.
Thanks a lot.
Ok, there is now an actually easy way of installing FreeBSD root on ZFS using mfsbsd livecd. I did a complete write up on my blog: http://mpietruszka.com/2010/08/14/freebsd-on-zfs-update-1/, but all you really need to get it going is just the mfsbsd livecd and two commands:
Mount the cd:
mount_cd9660 /dev/acd0 /cdrom
Install ZFS version 15/FreebSD:
zfsinstall -d /dev/ad0 -t /cdrom/8.1-RELEASE-amd64.tar.xz -s 4G -V 15
Just to let you know, pjd has started testing ZFS version 28 so we might see that trickle down real soon as well.
Not exactly the same, but Debian GNU/kFreeBSD has an installer with ZFS support: http://robertmh.wordpress.com/2010/09/06/debian-installer-with-zfs/
That should be: cat < /zroot/etc/fstab as in the docs. Sorry.
Thanks! Worked like a charm with 8.1
According to the docs, installing the bootloader (the steps "cd /usr/src/sys/boot/" ... make, et c.,)
are not required because 8.1 ships with a zfs aware bootloader
However, it tried it just to see, and it does no harm.
To n00b-proof your script, you might want to change:
"cat < /zroot/etc/fstab" to
"cat < /zroot/etc/fstab" _I_ know that ^D will get me an EOF, but someone just typing the script might get 'EOF' as the last line of /etc/fstab which might result in some interesting, though harmless, errors.
Once again, thanks for a really great post. You rock, and now so does my Beastie.
nice job, many thks ... just one question, I tried with success a slight updated version of this how-to with a mirrored zpool but I try now to add a second (non mirrored) zpool on the same disk, ... is there a way to do this ?
I would like to second what the poster above me (ben) has said. I do receive the same error messages upon boot.
...I should also add, one odd thing: my disk device was called ad3 (not da0, or ad0, or whatever). I partitioned it correctly (after some messing around), and the above commands all worked perfectly and gave the expected output. I had a slice for the full disk, partition one was 180-something GB (200GB disk) and then 4GB of swap; all the zfs creation commands worked; all fine.
I have only one HDD, a DVD burner (internal), and that's it.
Full disclosure: I have used linux for some time, but I am completely new to FreeBSD. Wanted to try ZFS natively for a project I'm working on.
I followed your guide and everything worked smoothly - however when I reboot, I get the following after the kernel loads/recognises devices:
...
uhub4: 8 ports with 8 removable, self powered
Root mount waiting for: usbus4
Trying to mount root from zfs:zroot
ROOT MOUNT ERROR:
If you have invalid mount options, reboot, and first try the following from the loader prompt:
set vfs.root.mountfrom.options=rw
and then remove invalid mount options from /etc/fstab.
Loader variables:
vfs.root.mountfrom=zfs:zroot
vfs.root.mountfrom.options=
Manual root filesystem specification:
: Mount using filesystem
eg. ufs:/dev/da0s1a
eg. cd9660:/dev/acd0
This is equivalent to: mount -t cd9660 /dev/acd0 /
? List valid disk boot devices
Abort manual input
My keyboard (a logitech usb one with backlit keys) doesn't light up and is inoperable at this point. I thought the 'uhub' stuff might be referring to the kb, and it was ruining the boot process, so I disconnected it (and the usb mouse) - no effect, same error. uhub/usbus stuff still comes up at the end of the output preceding the boot failure.
I also tried booting up, entering option 6 to go into bootloader prompt, followed the system's advice and set vfs.root.mountfrom.options=rw - this does nothing, it just shows that option as having the value 'rw' when you boot up and the process fails again in exactly the same way.
I have no idea what's going on, can anyone help me?
I was thinking perhaps I wrote information to fstab incorrectly, (it was the only file I didn't check with a 'cat /etc/fstab' afterr creating it, while writing out the commands in the tutorial), but that only has an entry for 'swap', so I don't know how this could affect things.
I thoroughly checked and I believe I wrote out every other command in the tutorial correctly.
Really disappointed this didn't work for me, and appears this hasn't happened to anyone else - can someone tell me what might be wrong?
Possibly a faulty DVD? I have lib32 among others under /8.0-RELEASE on my amd64 DVD.
Thanks for the info david, but when I cd to the /dist/8.0 on the CD I did a ls command I found all the directories base, catpages, dict, doc, games info, manpages ports except for lib32. and I'm using 8.0-RELEASE-i386-dvd1.iso downloaded from freebsd
Frank, the lib32 error could occur if you made a typo or omitted lib32 when typing this line:
for dir in base catpages dict doc games info lib32 manpages ports; \
Whatever the reason, the installer is complaining that there's no lib32 directory.
BTW I've also found this other howto very helpful for ZFS root setups:
http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ1
Hi,
Awesome post, I have some questions, after cd it to //8.0-release and did the
do (cd $dir; ./install.sh) ; done
I got some error messages pop up saying
can't cd to lib32
./install.sh: not found
anyone encounter this?
[...] following How to install FreeBSD 8.0 using only ZFS partitions numerous times to install FreeBSD on ZFS I decided automate things. So I created a pair of shell [...]
[...] How to install FreeBSD 8.0 using only ZFS partitions is a step-by-step guide for install FreeBSD 8.0-Release on to ZFS at rhyous.com that works. [...]
bah...looks like you did have the right command but wordpress filters it and messes it up...some of it got cut off. It just did the same thing in my previous commend. The command that's pasted here currently doesn't let the user input the required lines and create the file...for people running into the same issue with fstab, the full command is on step 3 in http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
Hi, great post, thanks for the detailed instruction! I've run into a problem though; on Step 3 when I run 'cat < /zroot/etc/fstab', it says the file does not exist.
I unmounted everything and reboot; I get the boot loader but when I try to start FreeBSD, it fails to mount zfs:zroot and leaves me on the manual root mount prompt.
I can see all the files on the file system through the boot loader command line, so everything seems to be intact, I'm guessing it's just the missing /etc/fstab that's causing the problem. Can you please tell me how I can fix this? I've looked over all the steps multiple times and can't find where I made my mistake.
Thank you
Slava
OK, I got it...the command you have posted up there didn't work for me, I needed to use the command from the other tutorial 'cat < /zroot/etc/fstab'
Also, after 'cd zroot ; ln -s /usr/home home', it keeps you in the zroot directory. So you should add a cd.. or cd / to go back to /. Not sure if that's required, but I wanted to make sure I ran everything from / unless otherwise specified.
My FreeBSD 8 system boots in ZFS now, thank you!!
Slava
OK, I think I got this working. Mirroring two disks in the zroot partition involves the following:
1. Follow all steps above.
2. On the second disk (da1 in my case) follow steps 9-14 above. Allocate the entire disk to the root partition. (Also, don't follow step 8; you do NOT install an MBR on the second disk.)
3. Run this command to add the second disk to the zroot pool:
zpool attach zroot da0s1a da1s1a
4. Verify that you now have two disks in zroot:
zpool status zroot
That's it.
Thanks David. That will be a great help to others that come here.
1. Yup, that appears to be the case. The machine is clearly running an amd64 kernel. Thanks for the sanity check.
2. Yes, I have multiple disks in this machine. I would like to mirror the first two of them for the zroot pool.
Is this doable using the method described here, and if so how?
thanks again
Worked for me on 8.0-RELEASE amd64, thanks.
Two questions:
1. In step 2.1, there is no 'amd64' directory under /usr/src/sys/boot. Is i386 OK?
2. ZFS newbie question: How to add disks to this zroot pool?
The command 'zpool add zroot da1' returns this error:
cannot add to 'zroot': root pool can not have multiple vdevs or separate logs
Thanks again.
1. Thanks for pointing that out. I believe I did using i386 and it was fine. Maybe they haven't made amd64 boot environment because i386 boot environment can boot and am64 machine.
2. Are you trying to add multiple physical disks and you are getting this error? I am not sure, I only had one disk when I did this.
Thank you! I've been using FreeBSD since 3.1 but still wouldn't have ever gotten it to work without your instructions
Great instructions...worked well for me. The line at the end for fstab reads...
/dev/ad0s3b none swap sw 0 0
It should be...
/dev/ad0s1b none swap sw 0 0
This isn't instantly obviously since it still boots ok...but it won't use a swap partition unless you give it the correct device name.
You can use qualifiers such as M or G in gpart's size parameter. That should be easier to read for you, and it's up to gpart to do the messy calculations.
(I would only for the last partition specify the size in sectors, so you fill it all up)
Also, step 18 and 18 can be merged into one:
gpart bootcode -b /mnt2/boot/boot0 -p /mnt2/boot/zfsboot da0
Wow, perfect! Thank you. I've "messed" with FreeBSD a few times over the years, but I am really needing something with ZFS support, and OpenSolaris is, uh, well, not ok for my needs, so FreeBSD it is. So, newb to FreeBSD, but been using Unix (and then Linux) for more than 20 years, so this post is EXACTLY what I needed. Again, thank you very much!