The fine folks at OpenBSD have the song up for the 4.2 release on the lyrics page. So go forth and rock.

It sounds like they went for a Rush feel this time and its not bad. Its been a while since I’ve actually liked a release song, so this is refreshing.

And of course there’s the requisite moral in the lyrics too….

You may remember a while back I pontificated about using a 3rd party host for lectroid.net stuff. Well I looked around and finally just gave up on that idea. Face it, after five+ years of running my own stuff, 3rd party hosts suck by comparison.

But I did make some changes:

• I now have a separate Soekris based firewall/router – I was running the one-machine-does-everything approach, but now I can blow up my web server and still get on the ‘net to google for help.
• I killed most of my OpenBSD dev boxes – Ok, face it, I’m not porting much anymore, so it was time for the sparc64 and — yes — the vax to go.
• OpenBSD 4.1 install. No not an upgrade, a whole new install. After 10 or so OpenBSD upgrades on a web/mail server you collect a lot of cruft. Time to decruftify.
• Web/Email updates – Dovecot 1.0, WordPress 2.2 probably others.
• Web site cleanup – old stuff like my old javascript pages and such are finally gone. My OpenBSD ports pages too.

As I said, I only blew it twice. Yeah, once on the new firewall I screwed up a rdr rule for DNS. I basically bled my internal DNS to the outside world. That broke lectroid.net for a bit but since I’m such a high-traffic site no one noticed. Then when doing some tweaks on a temp web server I blew up the real one, that kinda gets you rolled over real fast.

Finally though, over the weekend I rolled back onto my real web server (which went flawlessly) and here we are.

Now this isn’t a new idea by any means but it turns out that pretty much all the web info on doing something like this is old, and assumes tiny CF sizes (32 – 128 megs). Since this is 2007 and 512 meg CF cards are butt-cheap, there is no need to strip an OpenBSD install to fit. A simple baseXX.tgz and a kernel fit well within 512 megs and that’s for the most part all that’s needed for a full system (c’mon, you don’t need man pages, do you?).

Now the thing about CF cards and flash memory in general is that they support only a limited number of erase/write cycles, which means that eventually you’re gonna write and/or erase your CF card out of existence. Also, the Soekris box doesn’t have a “power off” mode or switch. Its for these reason I wanted to run this whole operation read-only so I don’t have to worry about killing the CF early or a hard power-cycle of the box.

Now you can’t just go marking all the volumes “ro” in /etc/fstab, yes I tried that. No, it really doesn’t work.

Enter the “memory file system”

The first thing to address is /tmp. A lot of stuff is written to and read from /tmp so it’s an easy fix to just use an mfs volume for /tmp. You can even see an example of this usage in the man page for fstab. Solved.

Next up was /var. /var is where logfiles are stored as well as other system databases and such. On *BSD systems it’s also the home of all the web files for use by the web server. I figured it would be just easiest to mount /var in it’s own mfs volume as well. This is where the -P option to mfs would come in handy, and it did.

Of course you immediately run into a chicken and egg scenario, you can’t populate an mfs /var partition from an existing /var partition while its mounted. I got around this by originally setting up /var on it’s own disk partition and then leaving that unmounted in /etc/fstab while mounting an mfs -P version pointing to it. End result, a copy of /var on a mfs partition. Solved.

Not there yet

So we’re done, right? We mark the remaining partitions “ro” in /etc/fstab and we’re good, right? Sure! And you reboot and all is great! And then you notice that the root partition is still read-write, what the hell?

Deep inside /etc/rc is this little gem:

mount -uw / # root on nfs requires this, others aren't hurt

Yeah, comment that out. Reboot and… holycrapwhatareallthoseerrors!

What about /dev?

Close but no cigar. OpenBSD fiddles with a number of devices under /dev and really these need to be read-write, unless you like all those errors. No, you can’t do the same trick that you did with /var and put /dev on its own partition. Seriously… think about it.

For /dev I still took advantage of mfs -P but I had to add a few lines to /etc/rc to make it all work:

mount_mfs -s 8192 -i 2048 -P /dev swap /altroot
mount_mfs -s 8192 -i 2048 -P /altroot swap /dev
umount /altroot

What this does is creates a little mfs partition, then copies all the devices to it, then creates the “real” mfs partition and then copies the devices to it. It’s gross, but it works well enough, and you only do it at boot. Solved.

Fin

Now you can go mark root as read-only and have it all work.

In the end I have a working Soekris running happily all read-only off a CF card. My /etc/fstab looks like this:

/dev/wd0a / ffs ro 1 1
/dev/wd0e /usr ffs ro,nodev 1 2
swap /var mfs rw,nodev,nosuid,-s=32768,-P=/dev/wd0d
swap /tmp mfs rw,nodev,nosuid,-s=16384

Cool, no?

Most people on asynchronous DSL connections like mine use queues to prioritize TCP ACK packets. As I’d said before my DSL seems unaffected by that particular issue which is interesting in and of itself because it seems to indicate that 1) my DSL is a fatter pipe than they advertise and 2) the bandwidth limiting is happening farther upstream and TCP ACK packets are being prioritized there.

Well that’s all well and good, but I recently discovered that during a large upload (*cough*bittorrent*cough*) my SSH sessions became laggy and generally a pain to use… and we can’t have any of that.

Enter ALTQ

I set up altq on my firewall with two queues:

altq on $ext_if priq bandwidth 327Kb queue { q_pri, q_def }
queue q_pri priority 7
queue q_def priority 1 priq(default)

This is a pretty vanilla queue set up, two priority queues one high and one default and a bandwidth setting at 327Kb. That’s important, that bandwidth number, but we’ll come back to that.

Then we add stuff to our queues like so:

pass out on $ext_if keep state queue (q_def, q_pri)

And voile, we’re pretty much done.

When you specify two queues the second one is used for packets with a “lowdelay” TOS and TCP ACK packets with no data payload and so in this case we’re assigning them to a higher priority queue, see?

Now there are plenty of examples around the net showing special queues for SSH traffic, but with SSH (at least OpenSSH) interactive traffic is set to lowdelay by default so there’s nothing more to do, it gets placed in the higher priority queue.

It’s the bandwidth, stupid

Where I had screwed up early on was in the bandwidth setting. See, queues only work when there’s something actually queued (imagine that) so basically you’ve got to be approaching your maximum bandwidth to see any effect. When I first started messing with queues I set the bandwidth to 350Kb or 91% of my 384Kb/s upload speed since that was my demonstratable maximum upload speed. That was too high and the queues were ineffective.

I dropped it to 338kb thinking that surely 88% would be good. I mean I see issues at speeds above that so surely it must be a good speed to kick in at. I could see improvement, but there was room to do better.

Finally, at 327kb, or 85% of my maximum upload speed, things hummed along, even at full tilt on the upload side. Now I can SSH to my heart’s content with my bandwidth is pegged.

Humppa Negala is an aquired taste for sure, but it brings back the Hackathons I’ve been to like nothing else. Gotta love the circus torture .

And who can argue with, “Be Open, Be Vocal, Stay Open, Stay Vocal”?

It sucks but it was also way past time.

I first got into OpenBSD at version 2.8 because at the time I was all into infosec and OpenBSD was the OS of choice. I soon started fixing ports and then maintaining them. I started helping out on porting Gnome 2.0 to OpenBSD and in 2002 was offered commit access.

Gnome was my thing at the time. It was (and still is), however, huge and unwieldy and had its fingers into everything. Generally a huge pain in the ass, but it also spawned quite a few bug fixes across the ports tree and even some in the source tree.

Gnome, the ultimate regression test.

Over time my interest in porting Gnome waned (owing a lot to my purchase of a Mac Mini as my primary desktop) and maintaining it became a chore. The killing blow came around Chrstmas when someone discovered a bug in a prereqesit to GTK+ 2.8 that made GTK+ apps (like Mozilla) die on certain 8-bit displays and it was my fault for not testing this.

Yeah, 8-bit displays, all two of them.

It’s a long, sad story and with a direction being taken I was not all together happy with, it also made it painfully obvious that my interest was at an all time low.

So currently my commit account is only disabled, which is a good thing as I’m sure I’ll be back when time and intrest permit. I still have nothing but respect for the OpenBSD developers and the goals they have. OpenBSD is still in my mind the only open source OS that does it right.

So go buy some CDs just don’t look for my name in the liner notes .

• Down with the sickness: for almost two weeks the family and I were down with the Flu. It was awful. I missed a whole week of work and probably should have missed two.
• GTK+ 2.8 issues: Someone discovered that GTK+ 2.8′s Cairo requirement makes GTK apps blow up on certain 8-bit displays. The GTK folks attitude is pretty much “We don’t support 5-year-old displays.” Theo’s attitude is “revert to 2.6″. What a pain.
• Component cable love: Among the standard Christmas goodies that abounded on the 25th, I got a PlayStation2 for the family (read: me and my son). In the process I reexamined the TV hookups and discovered that almost everything was one hookup level lower than it could be (what was I thinking). Now my DVD and PlayStation take advantage of component connections.
• Week’s vacation: I got the week from Dec. 24th to Jan 3rd off. Hu-ah! I have “nothing” plannedThat, of course means “whatever my wife wants done.”

So there .

The bigger news is that I finally figured out what was screwing me up with Monodevelop and gtksourceview-sharp. It was not using devel/gmake in the gtksourceview-sharp build and so the resulting port, while testing fine was in fact corrupted. Stupid.

Anyway, I’ve uploaded a whole suite of new Mono ports up on my ports page. Mono still sufferes from a problem with SysV semephores and garbage collection still doesn’t work… but we’re getting there.

Over the last two weeks I’ve fooled with the following (in no particular order):

• Updated x11/gtk+2 and it’s prerequisits to the 2.8 series, not committed yet, see here. This will necessitate a number of other updates to the ports tree (WANTLIB chnges), a lot of which are also in my tree.
• Committed diffs for the gdk_property_get warnings introduced by later gtk+ versions.
• Added static arch support to net/net-snmp
• Updated devel/gstreamer and devel/gstreamer-plugins to 0.8.11, not committed yet, see here (waiting on gtk+ 2.8).
• Updated Mono to 1.1.8.3 (yes, I know 1.1.9 is out), see here
• Removed Evolution 1 and committed what I had for Evolution 2 and marked it BROKEN. Good riddence, that POS has NEVER played nice with OpenBSD.
• Helped to add gpsmap support to net/kismet
• Unjacked the conf file handling for devel/gconf2
• Built and committed misc/gpsd

None too shabby…

They’re still based around version 1.1.7 of Mono due to a conversion to SysV Semephores in 1.1.8. This bites. It causes several issues, the first being the ports systrace support won’t allow semget so everything would need NO_SYSTRACE. Then the default semephore limits seem to be too low, so they run out. I’m not sure what magic setting raises them either. Then there are other issues as well.

So 1.1.7 for now.

Other than that, really it amounts to dependency cleanups, especially in the gtk-sharp* ports.