The BFC Computing Weblog

A note I wrote when asked about the future of ZFS and current best-practices for storage:

"My current line of thinking is that ZFS has the required reliability, OpenSolaris is where it actually works now.

FreeBSD 9 should have a good version of it (they can't get the zpool up to v23 in FreeBSD 8 based on FreeBSD's major-version compatibility requirements), and Linux is also going to get it. Either of those two are good destinations. Both are currently flakey though, and OpenSolaris is solid.

The ZFS storage layer has been ported to linux by one of the National Labs - they're still working on the POSIX layer. I suspect that'll be ready in a year or so.

I suspect Nexenta will wind up switching kernels itself and still calling itself Nexenta in a year or two. ZFS is endian- and platform agnostic, so moving a pool from one OS to another is a 'zpool export storage' on the source OS, re-install the base OS, and 'zpool import storage' and you're good to go.

Linux's btrfs might even be a decent in 2-3 years, but it's really really early still.

There's also the Illumos project which is making an Oracle-free OpenSolaris which stands a chance. Nexenta is funding those guys - at least they'll finally be able to do a complete Open Source build. That's probably transitional, though - why compete with FreeBSD on drivers? "Because OpenSolaris is much faster on disk access" is a fair answer, so they could also wind up supporting a more narrow set of hardware on an ongoing basis, until FreeBSD is that fast (they can figure out how OpenSolaris beats them pretty easily, especially with lots of people wanting it to be where they wind up.)

FreeBSD is license-compatible with CDDL, otherwise Linux would be the presumptive destination. Oracle could still chose to dual-license and settle the matter that quickly, but that's seeming less and less likely. Bill Moore, co-lead on ZFS has already left Oracle and joined the Nexenta board, so I suspect that's where ZFS will evolve, not at Oracle. If Jeff Bonwick does the same, it's pretty much settled."

Just a quick note for the search engines to find - peth0 can go missing from ifconfig if there is a GATEWAY= entry in ifcfg-eth0 (anaconda puts it here) and presumably -eth1, etc.. Put the default gateway in /etc/syconfig/network instead and use route-eth1 files instead to specify gateways.

Reboot for xend to do its setup correctly (please commment if there's a way to do this without reboot that works...) and instead of 'no peth0' you'll find one now exists. Also, your xenbr0 will be set up properly.

If you're seeing this, this article isn't quite done yet. Still setting things up right.

Fedora client, Nexenta Server.

On the linux side, start rpcidmapd and set it to start on boot.

service rpcidmapd start
          chkconfig --levels 345 rpcidmapd on
          

Wherever your DNS is, make sure your forward and reverse are set up correctly. No, really, make sure.

$ host my.linux.host.fqdn
          my.linux.host.fqdn has address 1.2.3.4
          $ host 1.2.3.4
          4.3.2.1.in-addr.arpa domain name pointer my.linux.host.fqdn.
          

Make sure dnsdomainname returns correctly on the linux host. You need to have my.linux.host.fqdn first on the line with 127.0.0.1 in /etc/hosts. This sets the NFSv4 domain name. Restart rpcidmapd if you needed to fix this. If this is wrong your files will all show as nobody:nobody on the mount (at this point everybody in the mailing lsit archives gives up and goes back to the crummy NFSv3). Make sure linux's dnsdomainname matches the output of Solaris's:

cat /var/run/nfs4_domain
          

Now, share under zfs:

zfs set sharenfs=rw=my.linux.host.fqdn,root=my.linux.host.fqdn pool/vol/subvol
          

mount under linux:

mount -t nfs4 solarismachine:/vol/subvol /mnt/localmount/ -o rw,intr,hard,proto=tcp,port=2049
          

Then set up a root nfs mount by:

blah, blah, blah, todo, todo, todo
          

I've got Jolicloud on the wife's netbook, and it's a nice easy-to-use distro.

Trouble is, it's based on Jaunty, which has old mythtv packages. These won't connect to our MythTV 0.23 backend in the TV room.

There is hope, though, the Avendard repo has newer packages compiled for Jaunty, but they're a bit tricky to install.

The process roughly:

Create a file:

/etc/apt/sources.list.d/avenard.list
          

with the lines:

deb http://www.avenard.org/files/ubuntu-repos jaunty release
          deb http://www.avenard.org/files/ubuntu-repos jaunty testing
          

and run the commands:

wget http://www.avenard.org/files/ubuntu-repos/ubuntu-repos.key && sudo apt-key add ubuntu-repos.key && rm ubuntu-repos.key
          apt-get update
          

to pull in the new repo. Now, remember this is dpkg/apt, so we can't just go installing mythtv first as the dependency resolution needs a bit of help.

First do:

sudo apt-get update
          sudo apt-get install nvidia-glx-185 nvidia-185-libvdpau nvidia-185-kernel-source
          sudo apt-get install libvdpau1
          

Now do:

sudo apt-get dist-upgrade mythtv-frontend mythvideo
          

and whichever other modules you need.

Then run:

sudo dpkg-reconfigure mythtv-common
          

to set your backend password.

Finally, install nfs and autofs to be able to mount your storage directory:

sudo apt-get install autofs nfs-common
          

and then edit:

/etc/auto.master
          

uncomment the /net entry, save, and run:

/etc/init.d/autofs restart
          

Then symlink to however you have your backend storage configured, e.g.:

ln -s /net/192.168.1.10/storage/ /storage
          

Now, launch mythfrontend from the Jolicloud Sound & Video group, where it will ask you for sudo access to add your user to the mythtv group and logout. Do it.

Log back in again, launch MythTV again, and go into 'Setup' and configure the storage directories for your media and/or recordings. Set parental controls as needed, they're front-end specific. Change the theme if needed, and set your painter to OpenGL if appropriate.

Those being done, you should be good to go to exit and start MythFrontend from the menu and just use it normally. SD MPEG-2 DVD video streams over 802.11g seems to work fine here in an ASUS 1000HE netbook. And now you have the world's most complex second(third,fourth) television.

Update: Jolicloud support writes via Twitter: "Adding third-party repositories could compromise your configuration. We won't be able to provide you support. ^CD" I suspect if you're reading this you can handle your own support, but be forewarned if you count on Jolicloud support. Personally, I'd rather see them engaging and supporting their community, but I understand about resource constraints.

They don't make flash drives like they used to. I've seen several pfSense installs fail recently due to drives flaking out and the wear-leveling not working as advertised.

Of course, you make regular backups of your config file, but in case you forgot, we can probably rescue your config file off of a disk image. Re-installing pfSense doesn't take too long, but rebuilding a working config file can take many hours, so rescuing is preferential.

This script has worked for me with dozens of file versions, but one can imagine scenarios with fragmented files where it would fail. There's nothing fancy going on here (this was hacked up with a client standing in my office with a broken pfSense box), but it might prove useful in a pinch.

          #!/usr/bin/perl -w
          use strict;
          use warnings FATAL=>'all';
          
          =comment
          pfsense_extract.pl - extract pfSense configs from an input stream
          (c) 2010 BFC Computing, LLC.  Licensed under the same terms as pfSense.
          
          This is useful for taking an image file of a damaged pfSense install
          and pulling out config files.  If you can mount the image normally, you
          should do that first.
          
          Due to the nature of the filesystem, there are often many copies of a 
          config file in a disk image, from each time it was saved.  You will 
          find a bunch of output files named: pfsense-config-1.xml, pfsense-config-2.xml,
          etc.  You can then use tools like diff to find out which the right one was.
          
          Example: 
             dd if=/dev/sdg of=broken_pfsense_image.dd bs=2M conv=sync,noerror
             strings broken_pfsense_image.dd | perl pfsense_extract.pl
          
          Processing a 1GB image as per the example takes about 20 seconds on a standard 
          2GHz desktop machine.
          =cut
          
          my $BASENAME='pfsense-config-X.xml';
          my $counter = 0;
          my ($outfile);
          my $do_output = 0;
          
          while (<>) {
          
              chomp;
          
              if ($_ eq '<pfsense>') {
          	$counter++;
          	my $filename = $BASENAME;
          	$filename =~ s/X/$counter/;
          	open($outfile,">$filename");
          	$do_output = 1;
              }
          
              if ($do_output) {
          	print $outfile $_ . "\n";
              }
          
              if ($_ eq '</pfsense>') {
          	close $outfile;
          	$do_output = 0;
              }
              
          }
          

There's a problem when running two X-displays with MythTV - some events on the non-Myth screen will steal focus and then the MythTV controls will no longer respond. This thread describes the problem well, but is now closed for comments.

Since then, mouse-switchscreen has been written, and solves the problem correctly. It's possible to bind the program to a hotkey.

In the end, I found it better to just run one display at a time since I couldn't prevent the focus stealing.

Redhat recently released a set of virtualized I/O devices for KVM, the kernel virtual machine. This short post will outline a method of converting a Windows Vista install (on KVM) to the new drivers using Virt-Manager. It has been tested on Fedora 11.

1. Make sure Vista VM is up to date on patches and the disk is error free.

2. Download drivers from Redhat network or here.

3. Mount the .iso file as a CD-ROM device.

Now you might think you can use the ‘Add Hardware Wizard’ here and add the drivers, add the hardware, and be good. I did. I wound up with an unbootable disk. Apparently Vista’s autodetection is required in this process. So…

1. Add a new network device of type ‘virtio’. Vista will do its “you’ve got hardware” routine and run you through all of its wizards. When it asks you for drivers, point it at the i386/2008 directory on the driver disc image. Yes, Yes, OK, Yes, Really, Continue, etc.

2. Shutdown the VM and remove the old ethernet controller. Boot up Vista and make sure the network works. You can conceivably skip this step for now if you want to make troubleshooting harder.

3. Add a new Storage controller. Leave the existing one as-is for now. You’ll have to pick a disk image you’re not using right now, or make a new one. Anything is fine, we’re not going to ever use it inside Vista. Do the driver dance again.

4. Shutdown Windows. Remove the storage controllers, and add a new one, type ‘virtio’, with your normal hard drive image. Take care of the old ethernet controller here too, if you ignored my previous advice.

5. Boot Windows normally. It should now be coming up on VirtIO disk and network drivers. If you get a bluescreen or a plea to use the RepairCD, something went wrong. Use the repair CD to restore to a previous restore-point and try again.

If anybody knows where to find a sound driver, please leave a comment!

For folks who are running the current development, or soon-to-be-just-released Fedora 11, you might find Firefox to be very crashy. It's not because it's the semi-controversial 3.5b4 version (which is excellent), it's because of a buggy library.

I'm running it with the Tree Style Tab and NoScript extensions, and can get a crash half the time when Session Restore is running, and almost all the time when I allow a site in NoScript.

If you run firefox from the console, so you get the debug messages, you'll see:

cairo-ft-font.c:554: _cairo_ft_unscaled_font_lock_face: Assertion `!unscaled->from_face' failed 

when the crash happens. I tracked this down through the Mozilla and Freedesktop bug systems to a problem with the Cairo graphics engine improperly disposing of fonts which it didn't own, for which a fix was incorporated last December. However, the version of Cairo shipping in Fedora 11 is older than that.

So, I applied the simple patch, fixed up the .spec, and put up some new RPM's for i386 and an SRPM for hackers and x86_64 users to build (rpmbuild --rebuild cairo-1.8.6-3.fc11.src.rpm).

I haven't tried cross-compiling from i386 to x86_64 before, and --target=x86_64 doesn't work, so if anybody can tell me how to do that short of learning mock, please leave a comment and I'll put up RPM's for that too.

The Redhat bug is here. Hopefully it gets accepted soon.

This is a neat enhancement to postfix for reducing spam by attacking its economics: making sure it speaks SMTP properly.

A spammer gets paid by the message delivered. So, it's in his interest to flood them out as quickly as possible. Because of this, they rarely implement mailers which negotiate the SMTP connection politely - they simply open the TCP connection and start sending.

When an SMTP client doesn't respect the proper-back-and forth postfix expects, it'll flag it as unauthorized 'pipelining' - for example when multiple messages are sent in succession, but which would otherwise be OK.

We can take advantage of this by forcing the issue, and increasing the odds a spammer will make this mistake by waiting just a second between establishing the TCP connection and telling the spammer we're ready to take mail. A loaded mail server may behave this way anyway, so it's not outside the norm and the resource consumption is minimal, but it attacks the economics of spamming.

In your main.cf file, you would add to smtpd_client_restrictions something like this:

          smtpd_client_restrictions =
                  permit_sasl_authenticated,
                  permit_mynetworks,
                  check_client_access hash:/etc/postfix/access-client,
                  sleep 1,
                  reject_unauth_pipelining
          
          

We accept all of our own users' connections (interactive ones, perhaps) right away, and if the sender is totally unknown to us, we wait for just a second. Then we reject any unauthorized pipelining. The log will show something like this:

May 6 10:49:00 mailhub postfix/smtpd[8965]: NOQUEUE: reject: RCPT from unknown[10.1.2.3]: 403 4.5.0 spamtarget@example.com: Recipient address rejected: Improper use of SMTP command pipelining

when the spammer attempts to just send.

It's worth noting that this method may not scale to very large installations, as those one second delays may be too much. But for the average-sized postfix install, it can make yet another dent in the spam deluge. Where it does consume 'too many' resources, one must weight the cost of computing resources vs. the time cost of dealing with yet another spam.

A Mac user might want to export his Keychain passwords and notes for several reasons - using a third-party password manager on Mac OS X, creating a time-resistant backup of passwords, printouts of passwords for the safe-deposit box or attorney, or switching to another operating system.

There's no easy way to do this. Keychain Access only allows you to export certificates, and Apple recommends backing up the Keychain database files, which accomplishes none of the above goals and promotes lock-in.

The keychain code is itself open source, but I couldn't find it compiled for another platform anywhere. I assume that enough of the OSX toolchain is required to make this infeasible, though likely not impossible. Still, it's not there.

Fortunately, I ran across an Applescript that uses Keychain Scripting to create a text file from a user's login Keychain. Unfortunately, it didn't do a bunch of things I thought were required for moving my passwords to a Linux machine, so here's the delta:

version 2009030201:

• handle all keychains
• handle all key types
• handle comments and descriptions
• handle errors
• trim dangling whitespace
• write to tab delimited format
• unlock all keychains first, so the mad tapping won't hit 'cancel'
• add username to filename
• replace carriage returns/newlines in text fields with spaces
• use unix line endings in output file

and some general code cleanup. I'm assuming the sample code is in the public domain and releasing this version under GPLv2+. Please improve this and comment here when you do or send changes back. If you own the original code and feel this is improperly licensed, let me know ASAP.

I've run this out of Script Editor - the advantage there is it's easy; the disadvantage is double-confirming every keychain access, one for Script Editor, one for Keychain Scripting. Terribly time consuming. I suspect if you compile this it'll eliminate the first half.

I've set this to open all the keychains first. Otherwise when hitting "allow, allow, allow" you might hit 'cancel' if it asks to unlock a keychain. If your keychain is big enough you might not get through the whole thing before the keychain unlock times out, so be careful.

Your minutes of tapping on the mouse button like a human waiting for a treat will be rewarded with a ~/Desktop/Passwords-yourusername file. It'll be easy to then process with other scripts, importable into databases or spreadsheets for further manipulation. I'll leave it up to you to be smart and not leave this password file sitting around in some unencrypted/unprotected location for any longer than absolutely necessary. If it gets stolen you're probably up a creek, right? So, be careful, only aim at what you intend to kill.

Download KeychainExport.