The BFC Computing Weblog

They don't make flash drives like they used to. I've seen several pfSense installs fail recently due to drives flaking out and the wear-leveling not working as advertised.

Of course, you make regular backups of your config file, but in case you forgot, we can probably rescue your config file off of a disk image. Re-installing pfSense doesn't take too long, but rebuilding a working config file can take many hours, so rescuing is preferential.

This script has worked for me with dozens of file versions, but one can imagine scenarios with fragmented files where it would fail. There's nothing fancy going on here (this was hacked up with a client standing in my office with a broken pfSense box), but it might prove useful in a pinch.

          #!/usr/bin/perl -w
          use strict;
          use warnings FATAL=>'all';
          
          =comment
          pfsense_extract.pl - extract pfSense configs from an input stream
          (c) 2010 BFC Computing, LLC.  Licensed under the same terms as pfSense.
          
          This is useful for taking an image file of a damaged pfSense install
          and pulling out config files.  If you can mount the image normally, you
          should do that first.
          
          Due to the nature of the filesystem, there are often many copies of a 
          config file in a disk image, from each time it was saved.  You will 
          find a bunch of output files named: pfsense-config-1.xml, pfsense-config-2.xml,
          etc.  You can then use tools like diff to find out which the right one was.
          
          Example: 
             dd if=/dev/sdg of=broken_pfsense_image.dd bs=2M conv=sync,noerror
             strings broken_pfsense_image.dd | perl pfsense_extract.pl
          
          Processing a 1GB image as per the example takes about 20 seconds on a standard 
          2GHz desktop machine.
          =cut
          
          my $BASENAME='pfsense-config-X.xml';
          my $counter = 0;
          my ($outfile);
          my $do_output = 0;
          
          while (<>) {
          
              chomp;
          
              if ($_ eq '<pfsense>') {
          	$counter++;
          	my $filename = $BASENAME;
          	$filename =~ s/X/$counter/;
          	open($outfile,">$filename");
          	$do_output = 1;
              }
          
              if ($do_output) {
          	print $outfile $_ . "\n";
              }
          
              if ($_ eq '</pfsense>') {
          	close $outfile;
          	$do_output = 0;
              }
              
          }
          

There's a problem when running two X-displays with MythTV - some events on the non-Myth screen will steal focus and then the MythTV controls will no longer respond. This thread describes the problem well, but is now closed for comments.

Since then, mouse-switchscreen has been written, and solves the problem correctly. It's possible to bind the program to a hotkey.

In the end, I found it better to just run one display at a time since I couldn't prevent the focus stealing.

Redhat recently released a set of virtualized I/O devices for KVM, the kernel virtual machine. This short post will outline a method of converting a Windows Vista install (on KVM) to the new drivers using Virt-Manager. It has been tested on Fedora 11.

1. Make sure Vista VM is up to date on patches and the disk is error free.

2. Download drivers from Redhat network or here.

3. Mount the .iso file as a CD-ROM device.

Now you might think you can use the ‘Add Hardware Wizard’ here and add the drivers, add the hardware, and be good. I did. I wound up with an unbootable disk. Apparently Vista’s autodetection is required in this process. So…

1. Add a new network device of type ‘virtio’. Vista will do its “you’ve got hardware” routine and run you through all of its wizards. When it asks you for drivers, point it at the i386/2008 directory on the driver disc image. Yes, Yes, OK, Yes, Really, Continue, etc.

2. Shutdown the VM and remove the old ethernet controller. Boot up Vista and make sure the network works. You can conceivably skip this step for now if you want to make troubleshooting harder.

3. Add a new Storage controller. Leave the existing one as-is for now. You’ll have to pick a disk image you’re not using right now, or make a new one. Anything is fine, we’re not going to ever use it inside Vista. Do the driver dance again.

4. Shutdown Windows. Remove the storage controllers, and add a new one, type ‘virtio’, with your normal hard drive image. Take care of the old ethernet controller here too, if you ignored my previous advice.

5. Boot Windows normally. It should now be coming up on VirtIO disk and network drivers. If you get a bluescreen or a plea to use the RepairCD, something went wrong. Use the repair CD to restore to a previous restore-point and try again.

If anybody knows where to find a sound driver, please leave a comment!

For folks who are running the current development, or soon-to-be-just-released Fedora 11, you might find Firefox to be very crashy. It's not because it's the semi-controversial 3.5b4 version (which is excellent), it's because of a buggy library.

I'm running it with the Tree Style Tab and NoScript extensions, and can get a crash half the time when Session Restore is running, and almost all the time when I allow a site in NoScript.

If you run firefox from the console, so you get the debug messages, you'll see:

cairo-ft-font.c:554: _cairo_ft_unscaled_font_lock_face: Assertion `!unscaled->from_face' failed 

when the crash happens. I tracked this down through the Mozilla and Freedesktop bug systems to a problem with the Cairo graphics engine improperly disposing of fonts which it didn't own, for which a fix was incorporated last December. However, the version of Cairo shipping in Fedora 11 is older than that.

So, I applied the simple patch, fixed up the .spec, and put up some new RPM's for i386 and an SRPM for hackers and x86_64 users to build (rpmbuild --rebuild cairo-1.8.6-3.fc11.src.rpm).

I haven't tried cross-compiling from i386 to x86_64 before, and --target=x86_64 doesn't work, so if anybody can tell me how to do that short of learning mock, please leave a comment and I'll put up RPM's for that too.

The Redhat bug is here. Hopefully it gets accepted soon.

This is a neat enhancement to postfix for reducing spam by attacking its economics: making sure it speaks SMTP properly.

A spammer gets paid by the message delivered. So, it's in his interest to flood them out as quickly as possible. Because of this, they rarely implement mailers which negotiate the SMTP connection politely - they simply open the TCP connection and start sending.

When an SMTP client doesn't respect the proper-back-and forth postfix expects, it'll flag it as unauthorized 'pipelining' - for example when multiple messages are sent in succession, but which would otherwise be OK.

We can take advantage of this by forcing the issue, and increasing the odds a spammer will make this mistake by waiting just a second between establishing the TCP connection and telling the spammer we're ready to take mail. A loaded mail server may behave this way anyway, so it's not outside the norm and the resource consumption is minimal, but it attacks the economics of spamming.

In your main.cf file, you would add to smtpd_client_restrictions something like this:

          smtpd_client_restrictions =
                  permit_sasl_authenticated,
                  permit_mynetworks,
                  check_client_access hash:/etc/postfix/access-client,
                  sleep 1,
                  reject_unauth_pipelining
          
          

We accept all of our own users' connections (interactive ones, perhaps) right away, and if the sender is totally unknown to us, we wait for just a second. Then we reject any unauthorized pipelining. The log will show something like this:

May 6 10:49:00 mailhub postfix/smtpd[8965]: NOQUEUE: reject: RCPT from unknown[10.1.2.3]: 403 4.5.0 spamtarget@example.com: Recipient address rejected: Improper use of SMTP command pipelining

when the spammer attempts to just send.

It's worth noting that this method may not scale to very large installations, as those one second delays may be too much. But for the average-sized postfix install, it can make yet another dent in the spam deluge. Where it does consume 'too many' resources, one must weight the cost of computing resources vs. the time cost of dealing with yet another spam.

A Mac user might want to export his Keychain passwords and notes for several reasons - using a third-party password manager on Mac OS X, creating a time-resistant backup of passwords, printouts of passwords for the safe-deposit box or attorney, or switching to another operating system.

There's no easy way to do this. Keychain Access only allows you to export certificates, and Apple recommends backing up the Keychain database files, which accomplishes none of the above goals and promotes lock-in.

The keychain code is itself open source, but I couldn't find it compiled for another platform anywhere. I assume that enough of the OSX toolchain is required to make this infeasible, though likely not impossible. Still, it's not there.

Fortunately, I ran across an Applescript that uses Keychain Scripting to create a text file from a user's login Keychain. Unfortunately, it didn't do a bunch of things I thought were required for moving my passwords to a Linux machine, so here's the delta:

version 2009030201:

• handle all keychains
• handle all key types
• handle comments and descriptions
• handle errors
• trim dangling whitespace
• write to tab delimited format
• unlock all keychains first, so the mad tapping won't hit 'cancel'
• add username to filename
• replace carriage returns/newlines in text fields with spaces
• use unix line endings in output file

and some general code cleanup. I'm assuming the sample code is in the public domain and releasing this version under GPLv2+. Please improve this and comment here when you do or send changes back. If you own the original code and feel this is improperly licensed, let me know ASAP.

I've run this out of Script Editor - the advantage there is it's easy; the disadvantage is double-confirming every keychain access, one for Script Editor, one for Keychain Scripting. Terribly time consuming. I suspect if you compile this it'll eliminate the first half.

I've set this to open all the keychains first. Otherwise when hitting "allow, allow, allow" you might hit 'cancel' if it asks to unlock a keychain. If your keychain is big enough you might not get through the whole thing before the keychain unlock times out, so be careful.

Your minutes of tapping on the mouse button like a human waiting for a treat will be rewarded with a ~/Desktop/Passwords-yourusername file. It'll be easy to then process with other scripts, importable into databases or spreadsheets for further manipulation. I'll leave it up to you to be smart and not leave this password file sitting around in some unencrypted/unprotected location for any longer than absolutely necessary. If it gets stolen you're probably up a creek, right? So, be careful, only aim at what you intend to kill.

Download KeychainExport.

Here's a technology idea: combine a solid-state flash drive, a synchronization engine, advanced virtual memory techniques, and a portable hardware abstraction layer to create a portable computer state device.

The idea would be like this: you have a small hardware device that you bring with your anywhere. When you plug it into one of your computers, it would synchronize the filesystem states, restore memory images, and resume your computing environment the way you left it at the last location.

It's roughly equivalent to the idea of network computers, except you don't need the ubiquitous ultra-high-speed Internet that doesn't really exist (when wireless gigabit is pervasive, this would become passe).

Current reasons this can't work, using linux as the obvious OS to start with, include the lack of an abstract HAL (root drive, home drive, etc) and the lack of virtual-memory restore on a per-process basis. Lots of the other parts exist already.

Initial limitations would probably be a restriction to the same hardware architecture (x86, AMD64, ARM, etc), inability to deal with filesystem changes greater than the capacity of the SSD, and an inability to restore stateful network connections (an IP proxy might work around the last one).

One company has made an approach at this experience by running the environment directly on the portable device, but this forfeits local resources and demands power draws unachievable on an external bus (for simple connectivity). That approach may gain viability over time, though, but not yet.

Would you, gentle reader, use such a device?

Intel thoughtfully has some ISO images of their BIOS flash upgrades, so you don't need to worry about finding the right flash software for your operating system and then timidly hoping that all works OK. You burn the image to a CD and reboot, then it flashes for you (using a FreeDOS/ISOLINUX system).

However, if you have a SATA CD-ROM drive, the device driver in FreeDOS doesn't support that. There is a SATA-compatible FreeDOS driver, but rather than rebuild Intel's ISO, there's an easier solution - make the BIOS emulate an IDE drive.

Go into BIOS Setup (F2 at boot), then Advanced ... Drive Configuration, and set 'Configure SATA as' to 'IDE' (mine was AHCI) and ATA/IDE Mode to 'Legacy'.

Reboot, allow the flash to succeed, then switch your BIOS settings back.

There's nothing wrong with this method, but Intel should highlight it on their download page.

KDE 4.2 looks like it's finally the right version to get me to use Linux as my daily desktop. 4.5 has more goodness baked in, 4.1 was insufficient, but 4.2 looks 'just right'. I used to be a GNOME user, but with GNOME's track towards Microsoft API's (mono) for its centerpiece applications I've gone over to KDE, and with its recent switch to LGPL I couldn't be more optimistic about its future.

For those who like to run official '-stable' versions of everything in Fedora, stop here. It'll be in Fedora 11 in a few months. Go read the warnings at the kde-redhat and the tracking bug if you want to know all the theoretical risks involved.

But for those eager to get on with things, I'll distil down what I think is the minimal command set to install the '-testing' release of KDE 4.2:

cd /etc/yum.repos.d

sudo wget http://blog.bfccomputing.com/files/kde.repo

sudo rpm -Uhv http://download1.rpmfusion.org/free/fedora/releases/10/Everything/i386/os/rpmfusion-free-release-10-1.noarch.rpm

sudo yum -y groupupdate kde-desktop

sudo yum -y update

(answering Y to importing GPG keys)

log out, log back in. You should be good to go.

I started with a working KDE 4.1 install, which wasn't easy either. If you haven't gotten that far first, be sure to do so. I have this in my notes from trial and error getting all the correct packages installed:

yum -y install kdebase kdegames kdegraphics kdemultimedia kdenetwork kdepim kdeplasma-addons kdeutils kipi-plugins PyKDE4 digikam-libs ebook-tools-libs kdebase-libs kdegames-libs kdegraphics-libs kdemultimedia-libs kdenetwork-libs kdepim-libs libgadu system-config-printer kdeaccessibility kdeartwork kdebase-workspace system-switch-displaymanager

but it may not be comprehensive (leave notes, please). Run 'system-switch-displaymanager KDM' to get the correct display manager selected. If your logins never succeed there are more packages to install. Unfortunately anaconda doesn't give a working KDE install, even if you select it at install-time.

Apple's Snow Leopard (10.6) operating system is due out in the next quarter according to slides shown recently at the LISA conference. It adds a small handful of features but it's mainly an architecture, performance, and bugfix release. Leopard (10.5) is pretty buggy and Apple readily admits it's not what an OS should be. So they're coming out with an update less than a year and a half since the last one, which is by most counts what Leopard should have been. This isn't really disputed, even Apple's name isn't for a new cat, this is the one with all the 'marks cleaned off'.

OK, so it's great that Apple's getting everything squared away so quickly, right? Yeah, it is if you've got recent hardware.

But what if you have a computer that was purchased in, say, the first half of 2006? It's going to have a PowerPC processor in it, and Snow Leopard doesn't support PowerPC. OK, so then you can run Leopard, which does support PowerPC. But, wait, Leopard is buggy, that's why they're fixing it.

OK, so you can run Tiger (10.4). Well, no, if you're going to be connected to a network you'd be foolish to do that; Apple only issues security updates for the current and previous versions of its OS, and with 10.6, 10.4 will go by the wayside. Within months there will be public exploits for your 10.4 machine available and the time to your machine being compromised is just a roll of the dice.

"Wait," you may be saying, "my machine is less than three years old and it's now unsupported?" "It's still under AppleCare warranty and I can't even get security updates?"

Yep, and there we see the tactical brilliance behind splitting the Leopard and Snow Leopard releases - Apple gets to book its revenue early on a not-ready OS, beat Microsoft to the market, and save a ton of money really only supporting one majoor version of its operating system. So, this doesn't really work out well for you? Just buy a new Mac, they're probably not going to do this again in three more years. Right?

This may be a dangerous gamble for Apple in a recessionary economic period, so perhaps they'll do the right thing and simultaneously keep their customer base. If not, Ubuntu 8/PPC isn't eligible for a commercial support contract but it'll run on your Mac and its security updates will be current for another two years. At that point your machine will be five years old and you can keep it around with debian or netbsd or if we're coming out of the downturn get yourself a brand new machine. By then you'll be so used to Ubuntu you'll have broad purchase options.