BSDCon's kernel internals tutorial
A review of Dr. Marshall Kirk McKusick's tutorial at BSDCon 2000, covering data structures, algorithms and networking.

By Nathan Boeger

Dr. Marshall Kirk McKusick needs no introduction. If you are pretty new to BSD, then I will fill you in. Dr. McKusick received his Ph. D. in computer science from UC Berkeley in 1984. Since then he has been a very strong contributor, influence and voice for both BSD and Unix. He oversaw the development of 4.3BSD and 4.4BSD, as well as brought us the 4.2BSD fast filesystem. He is a dynamic speaker and glows much like a proud father boasting about his child that can do no wrong. This glow is very well deserved. BSD has grown and matured into an elegant simple design that so far cannot be duplicated. Not even by the "L" word as he refers to it.

Trying to cram a good, thorough review of the kernel, virtual memory system, network implementation and filesystems into a two-day tutorial is by no means easy. I must say that Dr. McKusick had his work cut out for him. When we arrived, we received our red binders that included his entire slide presentation, as well as some selected man pages and some miscellaneous tutorials.

The class for the most part ran at a pretty good pace. You could sense the urgency after each break. The people would rush in, bumping into each other, dropping their cookies on floor and instead of going back to get a new one and miss a bit of the lecture, they just brush them off and eat them anyway (true story, I sat by the door and watched it).

Monday morning: Kernel Resource Management

Topics covered: basic kernel services, process structure, scheduling, signals and virtual memory management. This was a lot of ground; what happened was that Dr. McKusick first skimmed over the entire system and then made two more rounds going into more and more detail each time. For the most part this was a pretty high-level look at these topics.

Unfortunately due to the short time, Dr. McKusick was unable to go into great detail. So if you had not been through a good OS class or read his book ("The Design and Implementation of the 4.4BSD operating system" from Addison Wesley) then you where a little lost. Fortunately, I have read his book and so I was able to follow along.

Monday afternoon: Kernel I/O structure

Topics covered: special files, terminal handling, multipexing I/O, auto configuration strategy and structure of a disk device. This was almost a history of these services. The tale of the Process Group implementation, revealed an interesting note: This originally was a serious security threat since any process could claim to be the session leader for any other process (oops).

In this lecture, Dr. McKusick gave us an important glimpse into the future. He noted that the FreeBSD 5.x tree will finally be doing away with block devices. Also discussed was why most Unices have not implemented device drivers. For the most part it is due to them being a serious security threat. This makes sense when you think about it, all device drivers run in kernel mode. So a bad driver or a malicious one could reek havoc on a system very quickly.

Tuesday morning: Filesystems

Topics covered: filesystem services, block I/O system, filesystem implementation, support for multiple filesystems and NFS. This is one of Dr. McKusick best areas -- after all, he implemented the fast filesystem. Most of the lecture was of the basics of a filesystem: how disk's work and then softupdates.

The softupdates discussion was where we got another glimpse into the future. From the softupdates development, there is steady work on "snapshots". Snapshots are just basically block recovery. So snapshots will allow the system to take a snapshot of the filesystem. Then have a mount for them somewhere common. So if a file is lost or removed then the user can go into a snapshot and recover the file. This would be great for file servers and I think would bring a huge smile to a lot of system admins.

The lecture ended with a short history of NFS and then its basic implementation. There was also some discussion about the next generation of NFS (version 4). Apparently, NFS4 is moving toward not being stateless and will use some type of lease on files. So clients and servers will not have so many problems with orphaned open files.

Tuesday afternoon: Network Implementation

Topics covered: concepts and terminology, basic IPC services, system layers and interfaces, rbuting issues, internet protocols (TCP/IP) -- again a lot of ground to cover. The lecture opened up with a basic introduction to the socket-layer and its interface. Then outlined the different protocols, UDP, TCP, IP and ICMP. There was some mention of IPV6, but Dr. McKusick seems to think that it will be a very long time before we see IPV6 running in the real world. A very good look at the TCP window and how the BSD stack deals with slow re-transmissions and how it avoids retransmitting entire windows of data was also discussed. Most of this lecture was very in-depth in theory. So it would be difficult for me to try and sum it up in a paragraph. However this final lecture was very good given the time constraint.

Overall, I would say that if you have any interest in the inner workings of BSD or Unix in general, I would suggest attending a lecture by Dr. McKusick. Even if you have little understanding you will still walk away with some good understanding. However if you have taken an OS class or read a book, then this class will give you a good perspective on the theory of the BSD system -- not to mention the little look at what is to come for the BSD's. Besides just watching Dr. McKusick rant and rave about his little fights and struggles in creating the BSD daemon child is more than entertaining.

Nathan Boeger is a senior system admin / systems developer for GetRelevant, a promotional Internet marketing company.