FreeBSD diskless clients.

Written by: Nathan Boeger
email: nathan (you know what goes here) khmere.com
Date: June 4, 20000

Note: I must note that I did this with very little documentation so if you happen to see errors or have a better solution please feel free to let me know and I will be happy to update my docs.....

The steps to follow:

  1. Make a boot disk
  2. Setup a Bootp & NFS server
  3. Create a client kernel
  4. Create an Image file system
  5. Update the rc scripts for diskless clients

1. Boot disk.

In order to boot freebsd you will need to download a package like etherboot ( http://etherboot.sourceforge.net/ ) you will have to pay attention to the notes on building this package on FreeBSD. When I compiled it, I needed to play with some of the assembler options ( basically I commented out some option flags.... ) but on Linux is compiles with little configuration. (here is what I did for etherboot 4-6-1.)

NEWGAS:= $(shell $(AS) --version | grep -q '2\.9\.1' || echo -DGAS295)

NEWGAS:= $(shell $(AS) --version | grep -q '2\.9\.1' || echo "" )

 (Just removed the echo -DGAS295, but there is a better way to do this!)
After you compile then you should be able to:

cat floppyload.bin.pre (your driver).lzrom > /dev/fd0.1440

This will give you the boot floppy.

* Note that with the FreeBSD kernel you do not need the mknbi. Also that the instructions to make the floppy could change consult the etherboot docs....

2. Bootp & NFS server

In order to boot a diskless FreeBSD box you will need to choose bootp or dhcp. I found that bootp is the best of the two for just simplicity . Now to be honest I never got dhcp to send the root-path info to the client... So I cannot be totally non-biased. With bootp you will need to note or have ready the following things.

  1. Mac (hardware ethernet address)
  2. Fixed IP or dynamic (but fixed is simpler)
  3. Kernel that has the right options.. (see below)
  4. Bootp entry for each client
  5. NFS server with a directory for the client (basically an image of a real / )
  6. NFS swap image (but if you have a enough ram then you most likely you won't need this )

If you don't have your mac address don't fret, after you make your boot floppie if you boot from it, you should see your mac address (it will print it out) this was true for etherboot-4.6.2 Then you should note this and make sure you don't use capitol letters when you enter it into your bootptab.

Now the bootp entries will be pretty easy. What you will need to provide to the client is just some basic info. Like their IP, Default Gateway, root dir to mount etc.... Here is a sample entry: 

.default:\
        :ht=ether:\
        :sm=255.255.255.0:\
        :ds=192.168.1.78:\
        :gw=192.168.1.216:\
        :to=auto:

d95.blabla.com:\
        :tc=.default:\
        :ha=00d0b7583d8a:\
        :ip=192.168.1.95: \
        :hd=/tftpboot/kernel:\
        :rp="192.168.1.21:/opt/diskless/client":

** I will only cover a few options if you need more (man bootptab)

You use the :tc=.<name> this tells bootp to go look at that flag .<name>

If you create the .default you can put all of the common options for each server their (save a bunch of typing..... But then if you are shooting for carpel tunnel then..... go on type it)

the :rp=<NFS dir> is very important ! you should make sure that you double quote it, and I would suggest that you make sure that it is mountable from another box before you start your diskless client. Thier is very little debug info from them once they boot!

:hd=<dir> is where the server will get its kernel image.

Now the swap image can be made by using dd... (like dd if=/dev/zero of=<image name> bs=512 count=<size> ) if you need it.

After you have all that done then what you need to do is try and boot.

3. Client Kernel.

When you compile the kernel for the client you will need to have several options compiled in.

Here is a sample: 

options NFS #Network File System
options KERNFS #Kernel filesystem
options NFS_ROOT #NFS usable as root device
options BOOTP # use bootp to obtian ip
options BOOTP_COMPAT # workaround
options MFS
options NULLFS
options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
options BOOTP_NFSV3 # Use NFS v3 to NFS mount root

You also need to make sure that you compile in support for your nic card ! I would advise trying to make the kernel as small as you can. If you get rid of all scsi and even ide support (if you don't need it) and other unused stuff it will help make your box a little more tighter. Make sure that you just do make depend and make only. Do not make install for this kernel or it will replace your current kernel.

After you make this kernel you should place it in a directory on the bootp server that tftpd will be able to access. Most often you should place it in /tftproot (note that this can be a symbolic link to somewhere else). After that you should make sure that the name is kernel (note that you can change this but, no real need) When the client boots it will look for the file /tftproot/kernel to boot from by default. Make sure that this file is readable by who ever you run the tftpd from (see inetd) or just world readable should be fine.

4. Root filesystem

Now you will need to make a root image. A good solution is to make a clean install from FreeBSD-Release and then from an empty partition tar up the whole box from / ( * Note that you will have to exclude the /proc dir as well as your current tar file :-) and make sure you have enough room !! ). Then what you need to do is make a directory like /opt/diskless and exported that as read only. then make a client dir ( so... /opt/diskless/client/ ) and un-tar the image into the /opt/diskless/client dir (don't forget to make a proc dir inside this new dir) .

One note that should be made. After I created my images and I logged in. I noticed that if I had a password for root I could not login on the console.. ? But I could ssh into the box ... ? Also if I tried to change the password for anyone I got an error "cannot get lock on pwd.db" ..? I did not have much time to research this matter so I just worked around it by using the tar image where root had no password from a clean install (this may not work for everyone !) So if anyone has time to research this I would really like to know ! I did however get this to work properly... so its kinda weird ? I do however suspect that it might have been that a file or two did not get added to the tar archives... ?

5. Edit rc scripts

Now the cool thing about FreeBSD is that when the client boots it reads the /etc/rc.diskless1 and then makes a fake MFS mount ( memory file system) then it copies from the /etc directory into this MFS partition and then mounts it. This will override the current /etc and allow more than 1 server to boot off the same system image. Now make sure that you do not place the ethernet ip address into the rc.conf ... or you might have a problem (it should use the bootp ip instead). After the rc.diskless1, it then reads in the rc.diskless2 and creates another MFS mount for its /var. These scripts look for a /conf/<ip>/etc or /conf/etc dir so you will need to make a /conf (and or) /conf/<ip>/etc place them inside your image dir as well. What I suspect the /conf/<ip>/etc does, is it allows you to override the rc.conf or any other file for that specific box (which has that ip) so if all your box's will be similar don't worry about it.

For my boxes I had to modify the rc.diskless2, here is my updated rc.diskless2 


# S T A R T
# $FreeBSD: src/etc/rc.diskless2,v 1.5 2000/01/06 18:17:38 luigi Exp$
#
# rc.diskless2
#

mount_mfs -s ${varsize:=65536} -T qp120at dummy /var
var_dirs="log cron at run dev db msgs tmp spool cron/tabs at/jobs spool/mqueue \
	  spool/lpd spool/output spool/output/lpd "

for i in ${var_dirs}
do
	mkdir /var/${i}
done

chmod 755 /var/run
chmod 755 /var/db
chmod 755 /var/spool
chmod 755 /var/log
chmod 1777 /var/tmp

chown -R root.daemon /var/spool/output
chgrp daemon /var/spool/lpd

# now make the files for the logging deamons...
touch /var/log/messages
touch /var/log/all.log
touch /var/log/security
touch /var/log/wtmp
touch /var/tmp/vi.recover

#
# XXX make sure to create one dir for each printer as requested by lpd
#

if [ ! -h /tmp -a ! -h /var/tmp ]; then
	mount_null /var/tmp /tmp
fi
if [ -r /etc/defaults/rc.conf ]; then
	. /etc/defaults/rc.conf
elif [ -r /etc/rc.conf ]; then
	. /etc/rc.conf
fi

# extract a list of device entries, then copy them to a writable partition
(cd /; find -x dev | cpio -o -H newc) > /tmp/dev.tmp
mount_mfs -s 4096 -i 512 -T qp120at dummy /dev
(cd /; cpio -i -H newc -d < /tmp/dev.tmp)
# E N D


I only made minor changes.. First is the way the /var dir is made I added a few dirs like log ! And then I got a pesky error for the at/jobs... so I added that and then I touched some files so the start up doesn't complain (like syslogd)

Then you should be good to go. You will have to make sure that you have all the daemons running on the server (inetd, nfs.... ) and then make sure that your server and client are on the same network... Or if they have to cross a router then your on your own.

I must note once again that my goal was not to make a good diskless workstation but rather a diskless install image. So thier are certain things I have not mentioned (like the xf86config ! ) if you have more or better info please feel free to contact me I would be happy to update my docs