The BFC Computing Weblog

They don't make flash drives like they used to. I've seen several pfSense installs fail recently due to drives flaking out and the wear-leveling not working as advertised.

Of course, you make regular backups of your config file, but in case you forgot, we can probably rescue your config file off of a disk image. Re-installing pfSense doesn't take too long, but rebuilding a working config file can take many hours, so rescuing is preferential.

This script has worked for me with dozens of file versions, but one can imagine scenarios with fragmented files where it would fail. There's nothing fancy going on here (this was hacked up with a client standing in my office with a broken pfSense box), but it might prove useful in a pinch.

          #!/usr/bin/perl -w
          use strict;
          use warnings FATAL=>'all';
          
          =comment
          pfsense_extract.pl - extract pfSense configs from an input stream
          (c) 2010 BFC Computing, LLC.  Licensed under the same terms as pfSense.
          
          This is useful for taking an image file of a damaged pfSense install
          and pulling out config files.  If you can mount the image normally, you
          should do that first.
          
          Due to the nature of the filesystem, there are often many copies of a 
          config file in a disk image, from each time it was saved.  You will 
          find a bunch of output files named: pfsense-config-1.xml, pfsense-config-2.xml,
          etc.  You can then use tools like diff to find out which the right one was.
          
          Example: 
             dd if=/dev/sdg of=broken_pfsense_image.dd bs=2M conv=sync,noerror
             strings broken_pfsense_image.dd | perl pfsense_extract.pl
          
          Processing a 1GB image as per the example takes about 20 seconds on a standard 
          2GHz desktop machine.
          =cut
          
          my $BASENAME='pfsense-config-X.xml';
          my $counter = 0;
          my ($outfile);
          my $do_output = 0;
          
          while (<>) {
          
              chomp;
          
              if ($_ eq '<pfsense>') {
          	$counter++;
          	my $filename = $BASENAME;
          	$filename =~ s/X/$counter/;
          	open($outfile,">$filename");
          	$do_output = 1;
              }
          
              if ($do_output) {
          	print $outfile $_ . "\n";
              }
          
              if ($_ eq '</pfsense>') {
          	close $outfile;
          	$do_output = 0;
              }
              
          }
          

I recently had the power supply fail on my SOHO server, which was a mongrel of old parts, far too many USB cables, and was pretty darn slow. It was also very expensive to run, having a Pentium IV in it, the worst of Intel's line.

My goals for a new server were:

• quiet
• energy efficient
• virtualization support
• lots of storage
• easy to take backups offsite
• rackmount
• budget-friendly.

After poking around NewEgg for a while (I think I enjoy shopping there a bit too much) I came up with a list of parts (after reading many of the helpful reviews), and I have to say I couldn't be happier with the system.

It's almost inaudible, runs at about 105W under normal load, has seven hard drives in it, of various capacities, fits in my rack, has a hot-swap drive for off-site backups, and runs Fedora 10 like a charm. The case is especially nice to work inside, and is of higher quality than you'd expect for the price.

I'm acually using the 2.66GHz version of the Core2Duo, but they don't seem to make that anymore - 3.0GHz seems to be the low-end. It's worth noting here that most of the commercial server builders try to force you into the Xeon line with a rackmount server and those are both more expensive and more power hungry than the Core2Duo and Core2Quad lines. Get what you really need, keeping in mind that virtualizing multiple systems onto one is a huge energy win.

Additionally, I got a cooler from BestBuy (surprisingly their in-stock cooler is the nicest I've found) and used Arctic Silver 5 thermal compound to bond the CPU. Plus a bunch of SATA cables I have in a box (they seem to spontaneously generate in there). The whole package comes in under $1200 even if you have to buy every part. Compare at fifty percent more to purchase pre-assembled.

Here's the parts list:

• 1 x ARK 4U-500-CA Black 4U Rackmount Case - Retail
• 4 x Athena Power 6" SATA II Y cable Model CABLE-YSATA290 - Retail
• 1 x ASUS P5N7A-VM LGA 775 NVIDIA GeForce 9300/nForce 730i HDMI Micro ATX Intel Motherboard - Retail
• 1 x Rosewill RG430-2 430W 80Plus Certified,ATX12V v2.3/EPS12V v2.91, Active-PFC Power Supply, UL,FCC,CE,TUV,ROHS - Retail
• 1 x ICY DOCK MB671SK-BB Tray-less 3.5" SATA I & II Mobile Rack Removable Hard Drive Kit - Retail
• 1 x  Intel Core 2 Duo E8400 Wolfdale 3.0GHz 6MB L2 Cache LGA 775 65W Dual-Core Processor - Retail
• 2 x Kingston 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory Model KVR800D2N5K2/4G - Retail
• 4 x Seagate Barracuda 7200.11 ST31500341AS 1.5TB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive (bare drive) - OEM
• 2 x MASSCOOL FD08025B1M3/4 80mm Case Fan - Retail
• 1 x Antec 761345-75120-9 120mm Case Fan - Retail
• 1 x Rosewill RCR-IC001 40-in-1 USB 2.0 3.5" Internal Card Reader w/ USB port / Extra silver face plate - Retail
• 1 x LG 22X DVD±R DVD Burner with LightScribe Black SATA Model GH22LS30 - OEM
• 1 x SYBA SD-SA2PEX-2IR PCI Express SATA II Controller Card - Retail

The secondary SATA controller is only needed if you're going over the number of drives the motherboard supports, and likewise the power splitters. If you were buying all new 1.5TB drives you'd likely not need this. Obviously the memory card reader is only if you need it. But who wants a floppy drive anymore?

Happy building!

A Mac user might want to export his Keychain passwords and notes for several reasons - using a third-party password manager on Mac OS X, creating a time-resistant backup of passwords, printouts of passwords for the safe-deposit box or attorney, or switching to another operating system.

There's no easy way to do this. Keychain Access only allows you to export certificates, and Apple recommends backing up the Keychain database files, which accomplishes none of the above goals and promotes lock-in.

The keychain code is itself open source, but I couldn't find it compiled for another platform anywhere. I assume that enough of the OSX toolchain is required to make this infeasible, though likely not impossible. Still, it's not there.

Fortunately, I ran across an Applescript that uses Keychain Scripting to create a text file from a user's login Keychain. Unfortunately, it didn't do a bunch of things I thought were required for moving my passwords to a Linux machine, so here's the delta:

version 2009030201:

• handle all keychains
• handle all key types
• handle comments and descriptions
• handle errors
• trim dangling whitespace
• write to tab delimited format
• unlock all keychains first, so the mad tapping won't hit 'cancel'
• add username to filename
• replace carriage returns/newlines in text fields with spaces
• use unix line endings in output file

and some general code cleanup. I'm assuming the sample code is in the public domain and releasing this version under GPLv2+. Please improve this and comment here when you do or send changes back. If you own the original code and feel this is improperly licensed, let me know ASAP.

I've run this out of Script Editor - the advantage there is it's easy; the disadvantage is double-confirming every keychain access, one for Script Editor, one for Keychain Scripting. Terribly time consuming. I suspect if you compile this it'll eliminate the first half.

I've set this to open all the keychains first. Otherwise when hitting "allow, allow, allow" you might hit 'cancel' if it asks to unlock a keychain. If your keychain is big enough you might not get through the whole thing before the keychain unlock times out, so be careful.

Your minutes of tapping on the mouse button like a human waiting for a treat will be rewarded with a ~/Desktop/Passwords-yourusername file. It'll be easy to then process with other scripts, importable into databases or spreadsheets for further manipulation. I'll leave it up to you to be smart and not leave this password file sitting around in some unencrypted/unprotected location for any longer than absolutely necessary. If it gets stolen you're probably up a creek, right? So, be careful, only aim at what you intend to kill.

Download KeychainExport.

This blog is on a shiny (ok, flat black, really) new server, with much faster everything, and it’s a half-U low-power para-virtualized beta product I’m working on (more on that to come, now that I have room for pictures!). Just typing this it’s immediately apparent that there’s way more snappy (live preview with round-trips to the server) but there’s bound to be a bug or two somewhere. Please let me know if you see anything amiss.

As mentioned before, all BFC Computing servers are named for real-life heroes, and this one is no exception. stevens.bfccomputing.com is named for Brenda Stevens, a grandmother from Deerfield, N.H., who was killed when a tornado struck her home on July 24th and the building collapsed onto her. As the tornado destroyed her home she held onto her stepson’s baby boy, whom she was babysitting, long enough to keep him from sustaining any more than minor injuries. Mrs. Stevens didn’t survive, but her grandson lives due to her ultimate sacrifice.

From today’s new Macbook announcement:

11:01AM Q: Concern about the glossy screens. Are you going to offer another option?
          A: Steve: We're going all glass -- we won't offer another version. 
          Phil: You offset the reflection by the brightness, and consumers love it. One of the great things about a notebook is you can turn it however you want!

I’ve used a Mac laptop since 1992 as my primary machine and often find myself using it in situations where I can’t actually rearrange the furniture or move the windows (Phil apparently lives in an opaque bubble). So I’ve always ordered a Macbook Pro with a matte screen, because my brain simply can’t see through the glare. Some people can, my eyes don’t work that way.

Yeah, their marketing images actually
show the reflected keyboard

So, today marks the end of availability of new Macs I can use. Since OSX doesn’t run on other hardware (securely) this means I can’t plan on using OSX into the future. I’ll keep a machine around for media work in the short term, but it’s obvious I need to get as much of my work moved over to Linux as possible if I’m going to have hardware that’s current technology.

With Apple’s primary focus on the iPod/Phone market, its draconian tactics there, and its inability to deliver a stable next OS release this is merely the last straw (if it were the only problem I’d consider investing in custom coatings, etc.) Thanks, Apple, it’s been a fun 16 years.

Update: Not just me.

I’ve been writing a short note here after each minor release of Mac OS X 10.5, noting the major problems with it, and 10.5.5 is unfortunately no different. Today I applied it to my main machine’s Leopard install and tried two fairly simple operations:

• delete a partition with Disk Utility
• install Software Updates

The first totally messed up my drive’s partition table, resizing a supposedly untouched partition from 96 to 26 GB, rendering it unusable. The second, applying a half dozen software updates failed on the first attempt, and on the second attempt rendered the system unusable (LoginWindow would keep crashing and re-loading in an endless cycle).

So, I’m restoring my machine from backup now, and will stick with 10.4 (Tiger) until Leopard is as stable as Tiger.

Maybe 10.5.6 will be better, but as of now I’m still recommending clients stay on 10.4.11. A year into Leopard now, and it still has fundamental problems - that Apple has 10.6 (Snow Leopard) planned as a no-new-features release specifically to address architectural problems is a sure sign the issues run deep.

I just ordered a new hard drive from PC Connection and when I went to check out I got quite a surprise.

I realize that for a while companies have been charging more for shipping to residential addresses than business, but PC Connection has taken this to a whole new level. My home address was first on the account as the account pre-dates my office, so when I went to buy the drive it was pre-selected and shipping was charged thusly:

Whoa. I switched it to my business address and:

got free shipping instead. Much better.

On an RPM-based system, yum-utils provides a utility called ‘package-cleanup’. It has two useful options:

–orphans shows RPM packages that do not belong to any currently-configured repositories, and:

–leaves shows RPM packages for which there are no dependencies; that is removing them won’t trigger the removal of other packages. By default it’s concerned with libraries, but –all removes that restriction.

So, ideally you’d like to run:

package-cleanup –orphans –leaves –all

to get a list of all the packages you might want to consider for cleanup, say before or after an upgrade. But package-cleanup doesn’t support that.

So, here’s a little perl script, called leavesorphans.pl on my system that will run package-cleanup twice and print for you the intersection of the two sets:

#!/usr/bin/perl -w
          use strict;
          use warnings FATAL=>'all';
          
          use Data::Dumper;
          
          my @orphans = `package-cleanup --orphans`;
          my @leaves = `package-cleanup --leaves --all`;
          
          my (%orphans,%leaves);
          foreach my $orphan (@orphans) {
              $orphans{$orphan} = 1;
          }
          foreach my $leaf (@leaves) {
              $leaves{$leaf} = 1;
          }
          
          my (@matches);
          foreach my $orphan (keys %orphans) {
              foreach my $leaf (keys %leaves) {
                  if ($orphan eq $leaf) {
                      push (@matches,$orphan);
                      delete $leaves{$leaf};
                  }
              }
          }
          
          
          foreach my $match (@matches) {
              if ($match !~  m/Setting up yum/) {
                  print $match;
              }
          }
          
          

I recently ran it and found a few packages that were lingering on my system since Fedora Core 4, just wasting system resources. If all of your proper packages belong to a repository you can simply pipe the output of the command to xargs rpm -e. I’m not quite that slick, so I manually reviewed the list and kept the packages I had installed by hand.

LEBANON, NH, September 1, 2008 – BFC Computing, a computer consulting firm based in Lebanon, NH, has launched McCainPalin2008.us, a website dedicated to news about the McCain-Palin Presidential Campaign. The site is updated with the latest headlines and videos gathered from thousands of news sources around the globe. McCainPalin2008.us provides a single location for news readers to find updates on the presidential campaign. Rather than having to sift through other news sites to find information that may or may not be relevant, McCainPalin2008.us gathers all of that information into one place.

Bill McGonigle, owner of BFC Computing, says, “McCainPalin2008.us is based on a technology we’ve been developing called NewsMaker, a tool for rapid deployment of specialized news websites. We recognized the likelihood of this ticket in July and began work on the site then. We realize many folks are political news junkies who want to focus on specific topics, and for them we hope this site is helpful and enjoyable. Expect improvements through Inauguration Day.”

For up-to-the-minute coverage of the McCain-Palin campaign, visit http://McCainPalin2008.us . BFC Computing works with clients to develop computing solutions that exceed their expectations. Open Source allows BFC Computing to deliver solutions that are ideally customized to clients needs, protects against obsolescence, and delivers top-notch security. For more information on BFC Computing and its services, visit http://bfccomputing.com or call (603) 448-4440.

I was cleaning up my hard drive today and found some screenshots I took of websites on 9/11, in Apple PICT format. Less than 7 years later, those PICT’s aren’t viewable on OSX in the Preview application (the standard image viewer). Seeing as this OS came out in 2005, it was likely abandoned then. At the time I was running the latest version of Mac OS 9, judging by the screenshots.

So, less than 4 years of support for that presumably very common file format.

I’ve converted the pictures to PNG (Using Photoshop 7, which can parse them), which as an industry standard open format ought to be recoverable for some time to come.

This has been reason #687 to avoid proprietary file formats.