Infrastructure Product Review | Is FreeBSD a Superior Server Platform to Linux? (Web Techniques, Jan 2001)

Is FreeBSD a Superior Server Platform to Linux?

FreeBSD 4.1
www.freebsd.org
Free software

By Nathan Boeger
I switched to FreeBSD from Linux last year not because Linux isn't flexible enough, but because FreeBSD is richer and more mature. There are various flavors of BSD available. FreeBSD has remained consistent on i386 (meaning that it hasn't ported the OS to other platforms, for the most part), and now Alpha architectures. NetBSD has been ported to almost anything with a CPU. OpenBSD also has many ports to different architectures.
However, the main difference lies in its extensive code review. OpenBSD's team reviews all code, looking for possible exploits as well as any other userland exploits. OpenBSD has bragging rights in that it has had "three years without a remote hole in the default install!" This may not seem much of a feat, but compared to other OS default installs that have had serious holes on the system, this is quite impressive.

The reason I've stuck with FreeBSD is that, for the most part, it has very good support for Intel hardware. It lets me use many different types of hardware like RAID controllers and network cards. And FreeBSD also has good support for low level system calls, and a very large port tree.

I've also found that almost any program that runs on Linux will have a FreeBSD port available. If not, FreeBSD's Linux compat support lets me run many Linux programs almost natively. The Linux compat port or package must be installed-similar to emulation, but FreeBSD handles it rather differently. For more information, visit www.freebsd.org/handbook.

So What's New?
The differences between FreeBSD 4.0 and 4.1 are somewhat minor-they're mostly bug fixes. With the switch from 3.x to 4.x there was an extensive redesign of the VM system, /dev/kmem libs, more hardware support, and SMP support changes. The 4.1 version represents nine months of various fixes, which is more significant than minor add-ons and such.
I've chosen to compare selected server features of FreeBSD to those of Red Hat, which is the Linux distribution I've seen most often on production Web servers. Of course, Red Hat isn't the only Linux distribution out there, Red Hat is just a very vocal company at the fore of the Linux community. Debian is very popular, but with the GNU Hurd, it seems to be moving in a different direction than most other mainstream Linux distributions. So, I'll look mainly to Red Hat as the Linux representative.

System Boot and Startup Scripts
FreeBSD has a three-stage boot process. The first stage boot is very small (512bytes) and resides on the MBR. It loads the next boot stage (boot1 or another OS bootstrap program). The boot1 is very small (again 512bytes); its main purpose is to understand disk slices and load the next stage, boot2. This final stage is the most complex of the three. The loader program runs after boot2 and you can adjust a number of system options from the loader program. For example, you can specify the console (serial or video), load a different kernel, and load any needed kernel modules.

While this boot process seems complex compared to Linux lilo, remember that if you make a new kernel in Linux and forget to update lilo, or if the kernel is no good and you don't have a backup entry, you're left with a system that won't boot. With FreeBSD, even if your kernel is no good, you can simply unload it from the loader program, then load the old kernel that's backed up by default (provided you haven't manually deleted it).

This loader program also assists in remote server box management. From either the /boot/loader.conf or /boot.config you can specify that you want the system to boot up and use the serial console as the system console. Now all you need to do is serial into the box from remote and you have complete control of booting the system. This comes in handy when you want to choose a different kernel (like a diskless kernel) or if you're experimenting with different kernel modules.

After the system boots, the next stage is to run the system startup scripts. Most Linux distributions have their System V startup scripts symbolically linked from an init.d directory to the rc(n).d directories, for example, /etc/rc.d/init.d to/etc/rc.d/rc(n).d in Red Hat. This is a potentially confusing scheme, and you can't customize these scripts unless you properly add the runlevel info at the top of the file. You can try adding your custom scripts to the /etc/rc.d/rc.local, but this makes debugging and runlevel control difficult. And, if a script stalls, you can only wait.

FreeBSD takes a simpler approach. Most system services and initializations are inside the /etc/rc.conf (which overrides the /etc/defaults/rc.conf). From this script, you set the IP addresses for your various daemons, such as Ethernet devices, routes, hostname, services and so on. After this script is processed, it runs any programs or scripts inside /usr/local/etc/rc.d, where you can easily add a new service to the system. All you need to do is make a simple start script and put it in that directory, and FreeBSD will run all executable scripts there.

This flexibility can be used to ensure that all of your systems are consistent with regard to the running processes, and restart the same processes with little effort. All you need to do is have all your systems rsync over the contents of a master /usr/local/etc/rc.d. Of course, you could try the same with Red Hat but then you'll have to allow them to rsync over the entire /etc/rc.d/ directory or just the /etc/rc.d/rc.local.

But, if you have a large Web farm and need to change IPs or adjust static routes for all machines, under Red Hat you have two options. You can edit several files in the /etc/sysconfig/network-scripts/ directory, or run linuxconf for all machines, which is potentially insecure-good sys admins should remove it. In FreeBSD you edit only one file per box. It's a subtle distinction, and maybe I'm just being lazy, but there's more.

The File System
FreeBSD uses the UFS (Unix File System), which is a little more complex than Linux's ext2. It offers a better way to insure filesystem data integrity, mainly with the "sofupdates" option. This option decreases synchronous I/O and increases asynchronous I/O because writes to a UFS filesystem aren't synced on a sector basis but according to the filesystem structure. This ensures that the filesystem is always coherent between two updates. In my informal performance updates, softupdates showed significant improvement.
I used two identical boxes, one with Linux and the other with FreeBSD 4.0-RELEASE. I moved a 1.2GB file between two mount points, back and forth. I found that FreeBSD, without the sofupdates, performs a little slower than Linux. This speed changed after I added the softupdates to the FreeBSD kernel and then updated the mount point (via tunefs). Only then did I notice that FreeBSD's performance was marginally better (10 percent, or so).

These performance tests aren't perfect or anywhere near conclusive. The Linux filesystem can be tweaked for performance; however, currently ext2 gets its performance from having an asynchronous mount. This is great for speed, but if your system crashes it could take out the filesystem, its data, and its current state. Often, a hard crash permanently damages a mount. FreeBSD with sofupdates can sustain a very hard crash with only minor data loss, and the filesystem will be remountable with few problems.

Besides performance, FreeBSD UFS also has one major advantage over Linux in security. FreeBSD supports file flags, which can stop a simple script kiddie dead in his tracks. There are several flags that you can add to a file, such as the immutable flag. The immutable (schg) flag won't allow any alteration to the file or directory unless you remove it. Other very handy flags are append only (sappnd), cannot delete (sunlnk), and archive (arch). When you combine these with the kernel security level covered below, you have a very impenetrable system.

System Security
FreeBSD and Linux both have runlevel states. With Linux you have runlevels 0Ð6; none of which have security levels associated with them. With FreeBSD you have single-user mode and multi-user mode. However, you can run your kernel at different security levels. These levels vary from -1 (insecure) to 2 (very secure).
If you run your server at level -1 or 0, then the kernel security level doesn't do much. However, increasing the security level to 1 puts the system into secure mode. This means /dev/mem or /dev/kmem can't be opened for writing and prevents some attacks to your system by using exploits on these files. Also, the file flags cannot be turned off. Running in level 2 gives you all of the level 1 features plus it doesn't allow any disks to be opened for writing except by mount.

The kernel security levels give the system admin more tools. For example, if you set your system's file flags in strategic places and run it in secure level 2, you can create an almost read-only system. A cracker who tries to create a back door by altering a binary-like login, or even sshd, will fail. In addition, employing the sappnd flag on your log files means they can only be appended. So if a cracker tries to clean up after herself, she won't be able to clear your logs or wtmp files. If the cracker wants to really do damage by unmounting the file system and running newfs on it, her attempt will also fail in level 2, because she won't be able to write to the disks.

Another FreeBSD security feature is the log_in_vain option, which you set on boot by specifying it in the /etc/rc.conf. This feature logs any attempts to connect to your server on an unopened port. So if you're being port scanned, you'll see multiple entries into your /var/log/messages or dmesg. These entries include the port and the remote machine so you can track attempts. Most people who are scanning use a random pause between scans, which prevents software that's looking for scans from detecting them. However, the log_in_vain entries have a timestamp if logged to syslogd, so all attempts are logged regardless.

System Programming and /proc
Linux has a handy /proc filesystem. Some of its features are pretty nice, like the /proc/self/status file. Any process can get its virtual memory information, signal information, groups, user ID, and more from this file. Normally this is done by getrusage and other system calls. The problem is, when a process like top wants to get this information for the machine processes it has to use these files. So if you have a very busy file server, top will crawl because it has to start all the files in the /proc dir and the kernel has to generate them.
It may be a minor performance detriment, but the main problem with Linux's /proc file system is that there's too much dependency on it. As a system programmer, I'm very concerned about using expensive parsing routines to gather information. Another example of performance detriments for program ability is the routing table. On Linux I haven't found a good way to get the routing table from the kernel. (Of course, if you're an admin, there are two easy user space programs, route and netstat, which you can use in Perl or PHP, but not in C/C++.) I can parse out the /proc/net/route file, but this action takes up my resources and the kernel's resource when it generates the file. On FreeBSD, I can just open up a raw socket and dump the routing table into a buffer that I can use.

Conclusion
These differences may seem minor, perhaps even petty. But for speed and security, FreeBSD is the way to go. In no way am I trying to say that Red Hat, or Linux, is inferior. Linux has very good desktop uses, and works well in some server situations. But, the FreeBSD 4.x tree is by far the fastest OS running on Intel hardware that I've ever seen. This translates into all areas, including compilation, file access, network stack, or just running X. And for something to look forward to: A lengthy list of goodies are in the works with the 5.x tree and are due out sometime next year.

Nathan is the senior system administrator and systems programmer for GetRelevant (www.getrelevant.com), an online promotions company. You can contact him at nathan (at someplace like) khmere.com .

| | | | | | | | | | | | |