04.16
So a friend gave me a Soekris 4801 (in exchange for an Alpha DS10l) and I decided to use it as a PF based firewall. The very first hurdle, obviously, was installing and running OpenBSD off the CF card.
Now this isn’t a new idea by any means but it turns out that pretty much all the web info on doing something like this is old, and assumes tiny CF sizes (32 – 128 megs). Since this is 2007 and 512 meg CF cards are butt-cheap, there is no need to strip an OpenBSD install to fit. A simple baseXX.tgz and a kernel fit well within 512 megs and that’s for the most part all that’s needed for a full system (c’mon, you don’t need man pages, do you?).
Now the thing about CF cards and flash memory in general is that they support only a limited number of erase/write cycles, which means that eventually you’re gonna write and/or erase your CF card out of existence. Also, the Soekris box doesn’t have a “power off” mode or switch. Its for these reason I wanted to run this whole operation read-only so I don’t have to worry about killing the CF early or a hard power-cycle of the box.
Now you can’t just go marking all the volumes “ro” in /etc/fstab, yes I tried that. No, it really doesn’t work.
Enter the “memory file system”
The first thing to address is /tmp. A lot of stuff is written to and read from /tmp so it’s an easy fix to just use an mfs volume for /tmp. You can even see an example of this usage in the man page for fstab. Solved.
Next up was /var. /var is where logfiles are stored as well as other system databases and such. On *BSD systems it’s also the home of all the web files for use by the web server. I figured it would be just easiest to mount /var in it’s own mfs volume as well. This is where the -P option to mfs would come in handy, and it did.
Of course you immediately run into a chicken and egg scenario, you can’t populate an mfs /var partition from an existing /var partition while its mounted. I got around this by originally setting up /var on it’s own disk partition and then leaving that unmounted in /etc/fstab while mounting an mfs -P version pointing to it. End result, a copy of /var on a mfs partition. Solved.
Not there yet
So we’re done, right? We mark the remaining partitions “ro” in /etc/fstab and we’re good, right? Sure! And you reboot and all is great! And then you notice that the root partition is still read-write, what the hell?
Deep inside /etc/rc is this little gem:
mount -uw / # root on nfs requires this, others aren't hurt
Yeah, comment that out. Reboot and… holycrapwhatareallthoseerrors!
What about /dev?
Close but no cigar. OpenBSD fiddles with a number of devices under /dev and really these need to be read-write, unless you like all those errors. No, you can’t do the same trick that you did with /var and put /dev on its own partition. Seriously… think about it.
For /dev I still took advantage of mfs -P but I had to add a few lines to /etc/rc to make it all work:
mount_mfs -s 8192 -i 2048 -P /dev swap /altroot
mount_mfs -s 8192 -i 2048 -P /altroot swap /dev
umount /altroot
What this does is creates a little mfs partition, then copies all the devices to it, then creates the “real” mfs partition and then copies the devices to it. It’s gross, but it works well enough, and you only do it at boot. Solved.
Fin
Now you can go mark root as read-only and have it all work.
In the end I have a working Soekris running happily all read-only off a CF card. My /etc/fstab looks like this:
/dev/wd0a / ffs ro 1 1
/dev/wd0e /usr ffs ro,nodev 1 2
swap /var mfs rw,nodev,nosuid,-s=32768,-P=/dev/wd0d
swap /tmp mfs rw,nodev,nosuid,-s=16384
Cool, no?
