The BFC Computing Weblog: Category Open Source

peth0 missing from Xen Dom0 (RHEL, CentOS)

Bill McGonigle — Mon, 26 Jul 2010 15:20:00 -0400

Just a quick note for the search engines to find - peth0 can go missing from ifconfig if there is a GATEWAY= entry in ifcfg-eth0 (anaconda puts it here) and presumably -eth1, etc.. Put the default gateway in /etc/syconfig/network instead and use route-eth1 files instead to specify gateways.

Reboot for xend to do its setup correctly (please commment if there's a way to do this without reboot that works...) and instead of 'no peth0' you'll find one now exists. Also, your xenbr0 will be set up properly.

NFSv4 from Linux to ZFS under Solaris

Bill McGonigle — Tue, 22 Jun 2010 11:24:00 -0400

If you're seeing this, this article isn't quite done yet. Still setting things up right.

Fedora client, Nexenta Server.

On the linux side, start rpcidmapd and set it to start on boot.

service rpcidmapd start
chkconfig --levels 345 rpcidmapd on

Wherever your DNS is, make sure your forward and reverse are set up correctly. No, really, make sure.

$ host my.linux.host.fqdn
my.linux.host.fqdn has address 1.2.3.4
$ host 1.2.3.4
4.3.2.1.in-addr.arpa domain name pointer my.linux.host.fqdn.

Make sure dnsdomainname returns correctly on the linux host. You need to have my.linux.host.fqdn first on the line with 127.0.0.1 in /etc/hosts. This sets the NFSv4 domain name. Restart rpcidmapd if you needed to fix this. If this is wrong your files will all show as nobody:nobody on the mount (at this point everybody in the mailing lsit archives gives up and goes back to the crummy NFSv3). Make sure linux's dnsdomainname matches the output of Solaris's:

cat /var/run/nfs4_domain

Now, share under zfs:

zfs set sharenfs=rw=my.linux.host.fqdn,root=my.linux.host.fqdn pool/vol/subvol

mount under linux:

mount -t nfs4 solarismachine:/vol/subvol /mnt/localmount/ -o rw,intr,hard,proto=tcp,port=2049

Then set up a root nfs mount by:

blah, blah, blah, todo, todo, todo

Installing MythTV 0.23 on Jolicloud

Bill McGonigle — Wed, 02 Jun 2010 11:02:00 -0400

I've got Jolicloud on the wife's netbook, and it's a nice easy-to-use distro.

Trouble is, it's based on Jaunty, which has old mythtv packages. These won't connect to our MythTV 0.23 backend in the TV room.

There is hope, though, the Avendard repo has newer packages compiled for Jaunty, but they're a bit tricky to install.

The process roughly:

Create a file:

/etc/apt/sources.list.d/avenard.list

with the lines:

deb http://www.avenard.org/files/ubuntu-repos jaunty release
deb http://www.avenard.org/files/ubuntu-repos jaunty testing

and run the commands:

wget http://www.avenard.org/files/ubuntu-repos/ubuntu-repos.key && sudo apt-key add ubuntu-repos.key && rm ubuntu-repos.key
apt-get update

to pull in the new repo. Now, remember this is dpkg/apt, so we can't just go installing mythtv first as the dependency resolution needs a bit of help.

First do:

sudo apt-get update
sudo apt-get install nvidia-glx-185 nvidia-185-libvdpau nvidia-185-kernel-source
sudo apt-get install libvdpau1

Now do:

sudo apt-get dist-upgrade mythtv-frontend mythvideo

and whichever other modules you need.

Then run:

sudo dpkg-reconfigure mythtv-common

to set your backend password.

Finally, install nfs and autofs to be able to mount your storage directory:

sudo apt-get install autofs nfs-common

and then edit:

/etc/auto.master

uncomment the /net entry, save, and run:

/etc/init.d/autofs restart

Then symlink to however you have your backend storage configured, e.g.:

ln -s /net/192.168.1.10/storage/ /storage

Now, launch mythfrontend from the Jolicloud Sound & Video group, where it will ask you for sudo access to add your user to the mythtv group and logout. Do it.

Log back in again, launch MythTV again, and go into 'Setup' and configure the storage directories for your media and/or recordings. Set parental controls as needed, they're front-end specific. Change the theme if needed, and set your painter to OpenGL if appropriate.

Those being done, you should be good to go to exit and start MythFrontend from the menu and just use it normally. SD MPEG-2 DVD video streams over 802.11g seems to work fine here in an ASUS 1000HE netbook. And now you have the world's most complex second(third,fourth) television.

Update: Jolicloud support writes via Twitter: "Adding third-party repositories could compromise your configuration. We won't be able to provide you support. ^CD" I suspect if you're reading this you can handle your own support, but be forewarned if you count on Jolicloud support. Personally, I'd rather see them engaging and supporting their community, but I understand about resource constraints.

Rescuing a Broken pfSense Install

Bill McGonigle — Wed, 03 Mar 2010 13:54:00 -0500

They don't make flash drives like they used to. I've seen several pfSense installs fail recently due to drives flaking out and the wear-leveling not working as advertised.

Of course, you make regular backups of your config file, but in case you forgot, we can probably rescue your config file off of a disk image. Re-installing pfSense doesn't take too long, but rebuilding a working config file can take many hours, so rescuing is preferential.

This script has worked for me with dozens of file versions, but one can imagine scenarios with fragmented files where it would fail. There's nothing fancy going on here (this was hacked up with a client standing in my office with a broken pfSense box), but it might prove useful in a pinch.

#!/usr/bin/perl -w
use strict;
use warnings FATAL=>'all';

=comment
pfsense_extract.pl - extract pfSense configs from an input stream
(c) 2010 BFC Computing, LLC.  Licensed under the same terms as pfSense.

This is useful for taking an image file of a damaged pfSense install
and pulling out config files.  If you can mount the image normally, you
should do that first.

Due to the nature of the filesystem, there are often many copies of a 
config file in a disk image, from each time it was saved.  You will 
find a bunch of output files named: pfsense-config-1.xml, pfsense-config-2.xml,
etc.  You can then use tools like diff to find out which the right one was.

Example: 
   dd if=/dev/sdg of=broken_pfsense_image.dd bs=2M conv=sync,noerror
   strings broken_pfsense_image.dd | perl pfsense_extract.pl

Processing a 1GB image as per the example takes about 20 seconds on a standard 
2GHz desktop machine.
=cut

my $BASENAME='pfsense-config-X.xml';
my $counter = 0;
my ($outfile);
my $do_output = 0;

while (<>) {

    chomp;

    if ($_ eq '<pfsense>') {
	$counter++;
	my $filename = $BASENAME;
	$filename =~ s/X/$counter/;
	open($outfile,">$filename");
	$do_output = 1;
    }

    if ($do_output) {
	print $outfile $_ . "\n";
    }

    if ($_ eq '</pfsense>') {
	close $outfile;
	$do_output = 0;
    }
    
}

Dual Screen vs. MythTV vs. Mouse Focus

Bill McGonigle — Thu, 24 Sep 2009 00:35:00 -0400

There's a problem when running two X-displays with MythTV - some events on the non-Myth screen will steal focus and then the MythTV controls will no longer respond. This thread describes the problem well, but is now closed for comments.

Since then, mouse-switchscreen has been written, and solves the problem correctly. It's possible to bind the program to a hotkey.

In the end, I found it better to just run one display at a time since I couldn't prevent the focus stealing.

Converting a Windows Vista KVM Virtual Machine to Redhat VirtIO Drivers

Bill McGonigle — Mon, 14 Sep 2009 16:23:00 -0400

Redhat recently released a set of virtualized I/O devices for KVM, the kernel virtual machine. This short post will outline a method of converting a Windows Vista install (on KVM) to the new drivers using Virt-Manager. It has been tested on Fedora 11.

Make sure Vista VM is up to date on patches and the disk is error free.
Download drivers from Redhat network or here.
Mount the .iso file as a CD-ROM device.

Now you might think you can use the ‘Add Hardware Wizard’ here and add the drivers, add the hardware, and be good. I did. I wound up with an unbootable disk. Apparently Vista’s autodetection is required in this process. So…

Add a new network device of type ‘virtio’. Vista will do its “you’ve got hardware” routine and run you through all of its wizards. When it asks you for drivers, point it at the i386/2008 directory on the driver disc image. Yes, Yes, OK, Yes, Really, Continue, etc.
Shutdown the VM and remove the old ethernet controller. Boot up Vista and make sure the network works. You can conceivably skip this step for now if you want to make troubleshooting harder.
Add a new Storage controller. Leave the existing one as-is for now. You’ll have to pick a disk image you’re not using right now, or make a new one. Anything is fine, we’re not going to ever use it inside Vista. Do the driver dance again.
Shutdown Windows. Remove the storage controllers, and add a new one, type ‘virtio’, with your normal hard drive image. Take care of the old ethernet controller here too, if you ignored my previous advice.
Boot Windows normally. It should now be coming up on VirtIO disk and network drivers. If you get a bluescreen or a plea to use the RepairCD, something went wrong. Use the repair CD to restore to a previous restore-point and try again.

If anybody knows where to find a sound driver, please leave a comment!

Firefox Crashes on Fedora 11

Bill McGonigle — Thu, 28 May 2009 21:38:00 -0400

For folks who are running the current development, or soon-to-be-just-released Fedora 11, you might find Firefox to be very crashy. It's not because it's the semi-controversial 3.5b4 version (which is excellent), it's because of a buggy library.

I'm running it with the Tree Style Tab and NoScript extensions, and can get a crash half the time when Session Restore is running, and almost all the time when I allow a site in NoScript.

If you run firefox from the console, so you get the debug messages, you'll see:

cairo-ft-font.c:554: _cairo_ft_unscaled_font_lock_face: Assertion `!unscaled->from_face' failed

when the crash happens. I tracked this down through the Mozilla and Freedesktop bug systems to a problem with the Cairo graphics engine improperly disposing of fonts which it didn't own, for which a fix was incorporated last December. However, the version of Cairo shipping in Fedora 11 is older than that.

So, I applied the simple patch, fixed up the .spec, and put up some new RPM's for i386 and an SRPM for hackers and x86_64 users to build (rpmbuild --rebuild cairo-1.8.6-3.fc11.src.rpm).

I haven't tried cross-compiling from i386 to x86_64 before, and --target=x86_64 doesn't work, so if anybody can tell me how to do that short of learning mock, please leave a comment and I'll put up RPM's for that too.

The Redhat bug is here. Hopefully it gets accepted soon.

Reducing Spam with SMTP Validation on Postfix

Bill McGonigle — Wed, 06 May 2009 10:50:00 -0400

This is a neat enhancement to postfix for reducing spam by attacking its economics: making sure it speaks SMTP properly.

A spammer gets paid by the message delivered. So, it's in his interest to flood them out as quickly as possible. Because of this, they rarely implement mailers which negotiate the SMTP connection politely - they simply open the TCP connection and start sending.

When an SMTP client doesn't respect the proper-back-and forth postfix expects, it'll flag it as unauthorized 'pipelining' - for example when multiple messages are sent in succession, but which would otherwise be OK.

We can take advantage of this by forcing the issue, and increasing the odds a spammer will make this mistake by waiting just a second between establishing the TCP connection and telling the spammer we're ready to take mail. A loaded mail server may behave this way anyway, so it's not outside the norm and the resource consumption is minimal, but it attacks the economics of spamming.

In your main.cf file, you would add to smtpd_client_restrictions something like this:

smtpd_client_restrictions =
        permit_sasl_authenticated,
        permit_mynetworks,
        check_client_access hash:/etc/postfix/access-client,
        sleep 1,
        reject_unauth_pipelining

We accept all of our own users' connections (interactive ones, perhaps) right away, and if the sender is totally unknown to us, we wait for just a second. Then we reject any unauthorized pipelining. The log will show something like this:

May 6 10:49:00 mailhub postfix/smtpd[8965]: NOQUEUE: reject: RCPT from unknown[10.1.2.3]: 403 4.5.0 spamtarget@example.com: Recipient address rejected: Improper use of SMTP command pipelining

when the spammer attempts to just send.

It's worth noting that this method may not scale to very large installations, as those one second delays may be too much. But for the average-sized postfix install, it can make yet another dent in the spam deluge. Where it does consume 'too many' resources, one must weight the cost of computing resources vs. the time cost of dealing with yet another spam.

Mac OS X Keychain Export Tool

Bill McGonigle — Mon, 02 Mar 2009 23:34:00 -0500

A Mac user might want to export his Keychain passwords and notes for several reasons - using a third-party password manager on Mac OS X, creating a time-resistant backup of passwords, printouts of passwords for the safe-deposit box or attorney, or switching to another operating system.

There's no easy way to do this. Keychain Access only allows you to export certificates, and Apple recommends backing up the Keychain database files, which accomplishes none of the above goals and promotes lock-in.

The keychain code is itself open source, but I couldn't find it compiled for another platform anywhere. I assume that enough of the OSX toolchain is required to make this infeasible, though likely not impossible. Still, it's not there.

Fortunately, I ran across an Applescript that uses Keychain Scripting to create a text file from a user's login Keychain. Unfortunately, it didn't do a bunch of things I thought were required for moving my passwords to a Linux machine, so here's the delta:

version 2009030201:

handle all keychains
handle all key types
handle comments and descriptions
handle errors
trim dangling whitespace
write to tab delimited format
unlock all keychains first, so the mad tapping won't hit 'cancel'
add username to filename
replace carriage returns/newlines in text fields with spaces
use unix line endings in output file

and some general code cleanup. I'm assuming the sample code is in the public domain and releasing this version under GPLv2+. Please improve this and comment here when you do or send changes back. If you own the original code and feel this is improperly licensed, let me know ASAP.

I've run this out of Script Editor - the advantage there is it's easy; the disadvantage is double-confirming every keychain access, one for Script Editor, one for Keychain Scripting. Terribly time consuming. I suspect if you compile this it'll eliminate the first half.

I've set this to open all the keychains first. Otherwise when hitting "allow, allow, allow" you might hit 'cancel' if it asks to unlock a keychain. If your keychain is big enough you might not get through the whole thing before the keychain unlock times out, so be careful.

Your minutes of tapping on the mouse button like a human waiting for a treat will be rewarded with a ~/Desktop/Passwords-yourusername file. It'll be easy to then process with other scripts, importable into databases or spreadsheets for further manipulation. I'll leave it up to you to be smart and not leave this password file sitting around in some unencrypted/unprotected location for any longer than absolutely necessary. If it gets stolen you're probably up a creek, right? So, be careful, only aim at what you intend to kill.

Download KeychainExport.

Portable Computer States

Bill McGonigle — Tue, 17 Feb 2009 12:15:00 -0500

Here's a technology idea: combine a solid-state flash drive, a synchronization engine, advanced virtual memory techniques, and a portable hardware abstraction layer to create a portable computer state device.

The idea would be like this: you have a small hardware device that you bring with your anywhere. When you plug it into one of your computers, it would synchronize the filesystem states, restore memory images, and resume your computing environment the way you left it at the last location.

It's roughly equivalent to the idea of network computers, except you don't need the ubiquitous ultra-high-speed Internet that doesn't really exist (when wireless gigabit is pervasive, this would become passe).

Current reasons this can't work, using linux as the obvious OS to start with, include the lack of an abstract HAL (root drive, home drive, etc) and the lack of virtual-memory restore on a per-process basis. Lots of the other parts exist already.

Initial limitations would probably be a restriction to the same hardware architecture (x86, AMD64, ARM, etc), inability to deal with filesystem changes greater than the capacity of the SSD, and an inability to restore stateful network connections (an IP proxy might work around the last one).

One company has made an approach at this experience by running the environment directly on the portable device, but this forfeits local resources and demands power draws unachievable on an external bus (for simple connectivity). That approach may gain viability over time, though, but not yet.

Would you, gentle reader, use such a device?