Recently in General Computer Topics Category

The Great eBay Laptop Experiment - Part 2

| No Comments | No TrackBacks

The laptop experiment continues.

I realize that earlier I didn't explain why this laptop is going to be running Linux. I looked for a system capable of running Mac OS X 10.3 and there weren't any on eBay, even some that needed work, in the under-$100 range I was aiming for. Otherwise, I would have considered picking up a Mac so that I could continue to find Mac-related topics to write about here, since I'm doing less Mac work at the office now. (However, if you have such a laptop lying around that you'd like to give me, by all means feel free to so. My email address appears in the "Contact" section on this site. Let me know you want to send me a Mac laptop and I'll give you the address to send it to. If it's one of the Intel models, I'll even refund your shipping via PayPal.)

I could have considered running the "Hackintosh" (i.e., hacked OS X) environment on a non-Apple laptop, but that would have meant not only breaking the law (which I wasn't about to do) but also acquiring a system with at least a Pentium 4 CPU (to have SSE2 and/or SSE3 instruction sets needed by OS X). The only laptops in that performance range on eBay when I looked were well outside my $100 budget constraint.

In fact, the only laptops within my budget constraint were Pentium III and below. In the Pentium III category I found several in my price range, including the one I eventually purchased. It had 128MB of RAM, a 10GB hard drive, no battery, no CD-ROM or DVD drive, no AC adapter, no carrying case, and no operating system or operating system sticker on the bottom. Fortunately, I had a couple of used batteries that were compatible in my basement from when I used to repair Dell laptops on the side. I also had a "universal" laptop power adapter that could power the unit. I even found a 64MB SODIMM to boost it to 192MB of RAM. I also had a PCMCIA wireless card I picked up a couple of years ago, made by Hawking. Amazingly, it all worked smoothly together and I was able to cobble together a dual-battery wireless system with only $46 out of pocket.

The only thing missing at this point was an OS. I think I still had a Windows 98 license around somewhere, but I didn't have a legal license to Windows 2000 or Windows XP, so I couldn't load those on the machine. Windows 98 is no longer supported by Microsoft and isn't such a great OS for random wireless browsing anyway. That left options like ReactOS (which is a Windows clone that isn't quite yet ready for primetime, but getting there), BeOS, Linux, and the like. Linux has the best hardware and software support of all those, so I opted for Linux. The next question was which Linux "distro" (distribution) to get. I already had CDs/DVDs for Red Hat Fedora Core, Ubuntu, Kubuntu, Linspire, Debian, Gentoo, and 1-2 others I'd made for past experiments around the house. I ended up trying several of them. Ubuntu would boot into Live CD mode file, but the CD-ROM drive just ground and ground on the copies I had. Eventually I gave up because I would click an install option and have to wait an hour to see the result. For whatever reason (and I admit that it makes little or no sense to me), Kubuntu 6 worked and Ubuntu 7.04 didn't. So I chose Kubuntu because my copy of that distro's disk happened to be the one that worked.

Now that my eBay Special was up and running with the latest Kubuntu release, I was ready to start actually trying to use it. I was able to connect it to a nearby WiFi network successfully after loading it with Kubuntu, a Ubuntu Linux variant. While connected to the WiFi network, I upgraded it to the latest Kubuntu release (7.04 - though a newer one is about to come out) using instructions I found online. At that point, I was ready to begin loading it with applications, and I have to tell you it was easier than than either OS X or Windows would have been... by far!

To get the apps I knew I wanted loaded on the machine, I launched the Adept Installer, selected those applications from a list, and clicked "Apply Updates". Kubuntu obediently downloaded the applications and their dependencies, and installed them for me, all from that single click. Within minutes, I had FireFox, Scribus, Inkscape, and all the other applications I planned to use on the machine running for me. It was very slick and very impressive. And the apps all seem to work.

I loaded FireFox with some of my favorite extensions, including NoScript, AdBlock, Extended Copy Menu, DownThemAll and a few others. I populated it with bookmarks to all my sites (like this one) and some of my personal favorites like woot.com.

To make it completely usable for all my web work, however, I would need to find a Linux replacement for a tool I use to monitor game manufacturers' web sites for my game news site, gamerhotsheet.com. The tool I had been using up to this point was written for Windows using one of the Microsoft visual development tools, so it would not run as such on Linux (at least without WINE, which I would consider if I couldn't find a good alternative).

I've also decided that the 128MB of memory (plus another 64MB SODIMM I had lying around) wasn't going to be enough to keep this system running at peak efficiency. In fact, it seemed to be a bit sluggish already. I found some 256MB SODIMMs on eBay going at below $20 each and bid on those. If I get them, I'll be able to max the system at 512MB. That should help performance.

I've also decided that since this machine might be banging around in the car a lot, it might be good to ruggedize the data storage. Toward that end, I found a CompactFlash to 2.5" IDE adapter online and bought one. I'm hoping later to pick up a good-sized, fast CF card to place in it, then replace the old internal IDE drive with the adapter and CF card. That should minimize the potential for shock damage, at least to the data. To protect the screen, I ought to be able to find a decent-sized sleeve to put it in when I'm not using it, maybe something made out of thick neoprene.

To this point, I've spent $46 for the laptop, including shipping. If I can get the SODIMMs cheaply, I should be able to go to 512MB for $40-50, keeping inside my $100 budget. However, to implement the CF-to-IDE idea, I'm going to have to go over budget. The adapter was only $16. The CF card, though, could be a lot more - especially if I choose something like a 16GB Ultra III, which is perhaps the right thing to do. First, I want to test the adapter with one of my existing CF cards to see what the speed is like. If the adapter with an Ultra II flash card performs reasonably well, then it's worth investing in the bigger card. If it is too slow to be usable, then I can abandon the adapter and look at other options (like a decent backup or a spare drive).

I should point out that I am still waiting for word back from Dell on the master BIOS password to the unit so that I can fix some of the errant settings in the machine, such as date/time information and boot order.


Is There a USB Optical Mouse Problem?

| No Comments | No TrackBacks

Earlier this year, in my work as a system administrator, I was assigned to investigate why approximately 40 co-workers (out of 1700 using a PC or Mac) were having USB optical mouse issues. The symptoms were generally the same. While using the computer, the user's USB optical mouse would simply stop working. If they flipped the mouse over, it was as though the mouse was powered off. The laser would appear to have shut off. If they unplugged and re-plugged the mouse, it would nearly always come back to life. Only once in a while was it necessary to reboot the machine. Once in a great while, the USB keyboard would stop working as well. Again, unplugging and re-plugging seemed to fix it.

When I went through my first round of problems early in the year, a lengthy investigation proved that all of my 40 cases could be solved by performing the following steps:

1. Update the system BIOS from the manufacturer's web site. Sometimes this corrected the issue and no more work was needed.
2. Update the motherboard chipset drivers from the manufacturer's web site. Sometimes this corrected the issue and no more work was needed.
3. Update the mouse drivers from the mouse manufacturer's web site. Again, sometimes this corrected the issue and no more work was needed.
4. Replace the mouse with another mouse from stock.
5. Replace the keyboard with another keyboard from stock.
6. Install a self-powered USB hub between the computer and the mouse/keyboard.

The above 6 remedies seemed to take care of all 40 users.

Then, about 3 weeks ago, the problem reared its ugly head again. This time, the above 6 steps didn't work. People who were run through all 6 of them still had the problem. Our help desk bumped the problem back to me to resolve, since I'd handled the one earlier in the year.
This time I went through a pretty thorough search on the web, Microsoft's knowledgebase, Dell's knowledgebase, etc. What I learned was interesting. When I searched for mouse failures as a whole, probably 99% of the time it was an optical mouse that failed. Moreover, it was almost always a USB optical mouse that failed. This was true on Mac OS X, Linux, Windows, and one or two other lesser-known operating systems. The brand of mouse didn't seem to matter, either. I read about the problem existing with every major brand and even some generic types. It made me wonder about the reliability of USB optical mice.

As a troubleshooting step, I suggested that we identify 1-2 of our users who were having the most "mouse failures". We swapped them with some new mechanical (i.e., old "ball style") mice from stock. The problems vanished for those users. As a follow-up step, I attacked the users' former optical mice to my system. Sure enough, I began seeing the problem myself.

An interesting observation occurred when we had one user who wanted to attach both an older mechanical mouse and an optical mouse to his Windows XP Pro system at the same time. He found that his optical mouse would fail, while the mechanical one kept working normally. (And again, when I swapped his optical mouse for mine, I began to see the problem myself.)

Another interesting observation was made after we installed Microsoft Windows XP Hotfix 914015 and 918365 on the affected systems. Occasionally the mouse would stop working as before, but within a couple of seconds it tended to come back to life on its own without any effort. (Please do not ask me for these hotfixes. If Microsoft won't provide them to you, I can't help as it would be illegal to do so.)

So it appears to me at the moment that we may have two separate issues here. The first issue is that there is a bug in the Windows XP Pro USB stack that causes a mouse to go offline if it asks to have the USB controller reset due to an error. The Microsoft hotfix takes care of that. The other issue is that we seem to have quite a few defective USB mice on-hand. In fact, I pulled one randomly out of stock and tested it, and it turned out to be defective. I confirmed this by attaching it to a Mac OS X system (in addition to a Windows XP system) and finding the same result, the mouse quit working.

I have a suspicion that the reliability of USB optical mice is not as high as we all might think. Just gauging from the numbers I'm seeing at our site, as many as 3% of the optical mice in use are in fact defective. Whether this is just the failure rate we should expect or whether it's symptomatic of a widespread defect in USB optical mice, I don't know.

In any case, it's caused me to add a few items to the above list of 6 steps:

7. Apply Microsoft Hotfix 914015 (if it's a Windows XP system).
8. Apply Microsoft Hotfix 918365 (if it's a Windows XP system).
9. Test the user's mouse on a system that isn't currently experiencing the mouse issue and give the user a mouse that hasn't been shown so far to have the problem.
10. Swap the optical mouse with a mechanical (roller ball type) mouse.
11. If the system and mouse support it, try installing the mouse on a PS/2 port instead of USB. We didn't see these issues with PS/2 mice on any platform.

Those steps, so far, seem to eliminate the problem in all cases. Perhaps they'll help you if you're experiencing a problem with your USB mouse.

The Great eBay Laptop Experiment - Part 1

| No Comments | No TrackBacks
Given that WiFi seems to be proliferating throughout the areas where I live, work, and vacation, I thought it might be nice to have a laptop I could keep in the car with me and pull out whenever I happened to be stuck somewhere bored and do some blogging, writing, or whatever. My requirements for the system would be the following:

Total expenditure: $100 or less. That way, if someone breaks in the car and steals it or I smash it in some way, I'm not going to be too devastated since I will hopefully have gotten my $100 worth out of it.

Technology: Needs to be WiFi capable. Needs to be able to run something "reasonably current" in terms of operating system (e.g., Windows 2000, Mac OS X 10.3 or later, Linux 6.x kernel). Needs enough storage to hold a basic set of applications, including but not necessarily limited to the OS, an Office Suite (probably OpenOffice.org), web browser(s), GIMP, and other apps useful to me in blogging and writing.

I began searching eBay for used laptops. The ones listed as being in actual working order were all close to $200 when they sold. That was way out of the budget for this little experiment. Eventually, I settled on a Fujitsu LifeBook S-4546 that was listed without a hard drive or power adapter for about $40. I managed to win the auction and receive the laptop. Unfortunately, it's either dead or I haven't found the right power supply for it. I put it aside when I found that it also was missing a hard drive interface cable.

I searched eBay again, and managed to get a Dell Inspiron 3800 for around $50 shipped. Like the Fujitsu, it was missing an optical drive, battery, and AC adapter.. However, unlike the Fujitsu, it happened to use parts I had on hand from an old Dell Latitude CPxJ 750GT that I once used. I slapped a battery and DVD-ROM drive into it. Unfortunately, some idiot left an admin password in the BIOS and I couldn't convince it to boot from an operating system CD.

Since I work with Dells a lot, I know that they have a "master BIOS password" that can be used to unlock a system if you forget the password you (or someone else) has set. To get that password, however, you have to be able to prove ownership and possession of the laptop. I started a chat session with a Dell tech and managed to convince them to have someone look the password up for the system. Unfortunately, the techs who did that particular task were gone for the day.

The next day, I chatted with another Dell tech. They went to get the password for me, but found that their internal communication system was down and they couldn't reach the appropriate person(s).

Later in the same day, I chatted with another tech, who told me that they were having internal communication issues and she couldn't get the password either.

In the chat log she sent me, there was a reference to an address I could email to get help if the chat system wasn't doing it for me. I compiled all the information Dell's technicians had previously asked me into simple paragraph and emailed it to that address last Friday. It's late on Monday and no response yet.

In the meantime, I borrowed a Dell Latitude CP laptop (which is very similar to the Inspiron) and swapped hard drives with the Inspiron. I planned to load Ubuntu 7.04. Unfortunately, something about Ubuntu 7.04 just didn't get along with that laptop. It literally took 5-10 minutes to boot (from multiple copies of the CD). I then tried Linux Desktop XP 2006, which worked fine but I decided I didn't like. I then tried the new Vixta.org distribution, which loaded fine and looked great, but wouldn't install for some reason (even though the system met the specs). I pulled out an old Ubuntu 6 CD and booted from that. It installed without a hitch. I put the drive back in the Inspiron and it was once again functional as a laptop.

Ubuntu had appropriate video drivers, keyboard drivers, mouse drivers, sound drivers, etc. Everything seemed to be working properly when I went to bed last night.

The next step will be to plug a Dell wireless card I have into the machine and see if I can make it "speak WiFi". If so, I'll move on to working with the software configuration on the machine.

Is American Industry Shooting Itself in the Foot?

| No Comments | No TrackBacks

It should come as no secret or surprise to you that the Chinese are not known for having great respect for intellectual property laws. For years, they've flooded the world with bootleg music CDs, video games, DVDs, etc. They've also cloned popular pieces of high-tech gadgetry, such as the iPod. Korea and Japan were known for doing the same thing in prior decades. However, Korea and Japan were chased down by American (and other) intellectual property attorneys and eventually stopped (or at least radically decreasde) their efforts to pirate high tech inventions. In fact, they soon developed the expertise to develop and improve on those inventions, doing our own inventors one better in many cases.

There is a great deal of effort being poured into stopping the flood of illegal Chinese copies into the rest of the world. That pressure will inevitably give the pirates pause, and cause them to consider producing truly innovative goods as the Japanese and Koreans did before them. With China's abundant human and natural resources, they'll have the ability to crank out their goods more cheaply and in more quantity than perhaps any nation on Earth.

That's what I mean when I say that American industry could be shooting itself in the foot. By chasing down the Chinese outfits that are reverse engineering and cloning products like the Apple iPod and iPhone, they are in fact pushing the Chinese to develop their own technological expertise and design know-how. There is evidence, in fact, that this may already be happening.

Consider the Meizu miniOne, a phone patterned after the Apple iPhone. It's not a knock-off, per se. It doesn't run OS X, but Linux (or Windows CE according to some reports). It doesn't pretend to be an Apple product. Its specifications are impressive. The screen is 3.32 inches versus Apple's 3.5. Its resolution is 720x480 (standard DVD resolution) versus Apple's 320x480 (VHS resolution). The miniOne will be offered in 4, 8, and 16GB sizes. It will feature a 3 megapixel camera versus Apple's 2 megapixel camera. It will be the same thickness as the iPhone, but shorter and narrower. It doesn't use the potentially problematic MultiTouch display but a standard touchscreen.

Why is the miniOne significant? Consider several key points about it. First, it isn't just a feature-by-feature clone of Apple's iPhone. It's actually a different technology, utilizing design concepts inspired by the iPhone. But it also goes the iPhone one better. It will include a user-removable battery and a variety of functions the iPhone doesn't offer, such as the ability to work on any wireless carrier's network. Since I haven't gotten to play with an iPhone or a miniOne, I can't offer more of a comparison than the links I've provided above, but I can tell you that given the choice between a miniOne and an iPhone, I'd rather have the miniOne hands down. Regardless of whether it's based on Linux or Windows Mobile, I'll have much better odds finding and installing any apps I want on the miniOne.

If the Chinese can continue to innovate in this way, they won't need to be "cloning" our products in the near future. We'll want them because they're actually better...

A Peer-to-Peer Scam

| No Comments | No TrackBacks

I've seen this little "scam" a few times now and I've decided it's time to say something about them so that hopefully a few of you won't fall victim to them. The scam starts with an innocent-looking advertisement that promises to give you access to lots of music, movies, and other downloads cheaply, or even free. I've even seen such advertisements in banner ads and even in Google AdSense blocks like the one on this page.

You're usually directed to a page like the one I'm linking to here. This page tells you that you'll "find and download movies, music, and your favorite TV series" and that you will "get instant access to unlimited DVD quality movies (including new releases". It shows pictures of movies like "Mr. and Mrs. Smith", TV shows like "Desperate Housewives", and more. The implication from the ad is that by purchasing some software they are selling, you'll have access to some great network of "759,989 users" with "over 95,000,000 media files" that you can transfer to your computer or iPod.

What they're not telling you, unless you click on the link marked "Legal Disclaimer" is that they are selling you Peer-to-Peer (P2P) software. They're also not telling you that P2P software is generally available free of charge. The "service" they're selling you is most likely access to a list of links to P2P networks where you can download copyrighted movies, music, software, etc. While such materials ARE freely available on those networks, downloading such items from P2P networks is most definitely ILLEGAL and will open you up to prosecution. This is explained in their legal disclaimer, and is pretty much the opposite of what the rest of the site indicates. They imply that you'll be downloading the latest movies (for example) for free, but in the disclaimer they explain that doing this would be illegal.

So what we have here is someone attempting to rip off those who aren't familiar with P2P networks and the law. They charge the people $39.99 for "unlimited access" to P2P networks which cost nothing to access in the first place. They give people the impression that they can download anything they find on such networks safely and legally. Only in the disclaimer do they tell you that you'd actually be breaking the law if you do what they suggest. And believe me, the way the authorities are watching P2P networks these days, you'll very likely be caught.

Recover a Lost Lexar JumpDrive Secure 1.0 Password

| No Comments | No TrackBacks

A while ago, my wife gave me a Lexar JumpDrive Secure 1.0 512MB USB Flash Memory Drive.  This drive had been given to her by someone else, who had apparently configured it so that about half the drive's storage was in the "secure" zone.

I didn't realize this until the drive had become full.  I noticed that Windows XP saw it as a 256MB FAT32 drive instead of a 512MB drive.  I had wiped the drive when I received it, so I lost the original software.  A visit to Lexar's site got me the "SafeGuard" software Lexar originally shipped with the drive.  I tried to use this to reset the drive but it did not work without the original password used to set up the drive.

Below are pictures of the front and back of the JumpDrive Secure 1.0:

The device in question has a part number of JDS512-04-500C in the 512MB model. Theoretically other models in the same line should unlock this same way.

I did some searches online and found articles from 2004 indicating that it was POSSIBLE to read the password in cleartext from the SafeGuard software while it was running in memory, so I decided to give it a shot.  I figured I had little to lose.  Unfortunately, all of the articles online were very vague as to the precise details.  I would need to discover those myself.  After some time, I figured out what to do.

First, I found the program WinHex 12.9 SR-12 by Stefan Fleischmann on the web.  This program is a free download for evaluation purposes.  I installed it.  Then I went to the Tools menu and selected "Open RAM'.  I then chose the "SafeGuard" process.  I went back to SafeGuard and entered a password that was unlikely to appear in the program's memory (e.g., "mes123").  I got an error indicating that this was the wrong password.

I clicked the "find" icon (binoculars) on the toolbar.  I asked the program to search SafeGuard's memory for the "mes123" password I'd just typed in.  It found that password near the phrase "The password reminder is" in memory.  A few bytes after that, I saw the word "test" sitting all by itself among a bunch of blanks. 

I went back to SafeGuard and entered "test" as the password.  The drive unlocked immediately.  I had found it.


Advanced Searching with Google

| No Comments | No TrackBacks

Based on some of the questions I received in my email related to the Da Vinci Code Quest on Google, it sounds like some of you might find this article of interest.  When you're looking for something specific on Google, there are some very simple tips that can greatly increase your chances of finding it.

Use the Least-Common, Most-Likely Words to Match Your Result 

Let's say you're looking for a review of the Mel Brooks movie "High Anxiety".  You could try a generic search like "movie review sites" and have to wade through hundreds of results before you found a site that had a review of this older Mel Brooks film.  You could also search for "Mel Brooks" but given the man's very prolific career, you will probably have to wade through a lot of his other information to get to a review site.  So, in this case, the best thing to ask Google to find might be "review Mel Brooks High Anxiety" (without the quote marks).  This uses the less-common words "high anxiety" and the most likely word "review" to find your result.  In this example, Google gives us a review of the movie on the first page.

You might have to try two or three searches to decide what the best words are for the topic you're seeking.  For example, if you were looking for lyrics to a song you heard on the radio that included the line "you know I couldn't last" because you wanted to know who sang it, I'd recommend including the word "lyrics" in your search.  The reason for this is that this phrase contains a lot of commonly used words that would appear on many web pages.  Adding "lyrics" to the search term tells Google to narrow it down to pages with the word "lyrics" associated with them.  Since this is a song lyric you're thinking of, that will help.  Narrow down the results.  For example "couldn't last lyrics" will give you two main results, the song "You know I couldn't last" by Morrissey and "Couldn't last a moment" by Colin Raye. 

Careful use of Quotation Marks

It's not too widely known, but Google attaches a special meaning to quotation marks when you type them into a search box.  For example, looking back at that song lyric search above, you could enter the exact line of the song you heard, place it in quotes, and add "lyrics" to it.  This would take you to the best possible result for the search, which happens to be the Morrissey song.

Quotation marks can really help if you're looking for something very specific, too.  For instance, maybe you once read about a review of the movie "High Anxiety" that you saw years ago.  You can't remember much about it, but you'd like to find that review again because the phrase "your driver and sidekick" sticks out in your mind for some reason and you want to know why.  The best way to find it again might be the following search query (this time, typed exactly as shown below, with the quotes):

 "High Anxiety" review "your driver and sidekick"


This tells Google you're only interested in pages where the exact phrases "High Anxiety" and "your driver and sidekick" appear.  You'll find that this search results in a single article, a review of the movie from the New York Times.

This can be helpful also when searching for help for computer error messages, information about specific people by name, etc.

Use the Right Google Search

The "plain old" Google search that you get by going directly to http://www.google.com/ will find about anything you're looking for.  But if what you're looking for is something more specific, like a picture of a 1957 Buick, using the Google Image Search would be a better option.  Instead of having to wade through lots of written material about 1957 Buicks and having to find a decent picture of one, Google Image Search will limit the results you get to images only.  Instead of a page of search results listing lots of articles, you'll get a page full of thumbnail pictures of 1957 Buick automobiles, which is what you were looking for.

Similarly, if you're looking for information about a book, Google Book Search can be of more help than a "normal" Google search.


There are other Google searches as well, for things like Scholarly papers, maps, shopping, etc.  The extra couple of clicks it can take to get to these resources (most of which are available through the "More>>" link on the Google homepage) can save you a lot of time in sifting through the results you don't want to find the ones you do.

Got a Tip Not Listed Here? Share!

If you have had particularly good success using a technique I've not covered here, please feel free to email me (see the "Feedback" link on the home page for my email address) and I'll include it here when I get a chance.

Cracking the Zodiac Killer's "340 Cipher" Part 3

| No Comments | No TrackBacks

In the last installment, I told you something about the shortcuts and general flow of execution of the custom Visual Basic program I've written to try to crack The Zodiac Killer's thus-far-unbroken cipher known colloquially as "The 340 Cipher".  This time around I'll tell you a bit more about the program itself.

The program makes all of its "deciphering" decisions based on what I'm calling a "gated scoring algorithm" intended to expend the least possible effort determining that a potential "decode" of the message really does look like a decode.  The algorithm works something like this:

  • First, the program looks and counts the frequency of letters in the "decoded" message.  If the letters which appear most commonly in normal English writings appear in approximately the right frequency in the message, it generates a base score.  If that base score is too low (i.e., there are not enough of the most-common letters in the message), scoring stips here before too many processor cycles are used.
  • If the program finds "approximately" the right frequency of letters in the message, it then gets more granular about the letters it's looking for.  It makes sure that in the decoded message it finds the approximately-correct frequency of As, Bs, Cs, etc.  Each letter that is occurring in approximately the right frequency (+ or - 20% of normal English text) gets a higher score than those which don't.  Any letter appearing "too often" deducts from the total score (e.g., lots of "Zs" would drop the overall score).  If letters appear in the "decoded" message in approximately the right frequency, scoring continues. Otherwise, it stops here before more cycles are wasted.
  • If the frequency of individual letters looks good, the program looks at the most common bigraphs in the English language and compares these to the message.  If it finds approximately the right percentage, scoring moves on. Otherwise, it stops.
  • If the frequency of bigraphs looks about right, it then looks at a more granular list of bigraphs, trigraphs, and quadgraphs and scores the message based on whether these seem to be appearing in about the right amounts for a normal English text.  If so, the score is increased. If not, it isn't.  If the score isn't sufficiently high enough, no more scoring effort is performed.
  • Assuming the breakdown of bigraphs, trigraphs, and quadgraphs is within a reasonable tolerance from "normal" English text, the program then pores through a 20,000-word English dictionary, going from the longest to the shortest words. This dictionary provides a score for each word, with added weight given to those words the killer used often that don't occur normally in English writing (like the killer's tendency to misspell "having" as "haveing").  This part of the scoring process can take several seconds of elapsed time to complete, so it is only done by the program when there is a very good chance of finding lots of English words in the text.

I call this a "gated scoring algorithm" because the potential "decode" of the message must achieve a certain predefined score before it can get through the "gate" into a more time-consuming scoring method.  This method allows the program to "fly" past potential decodes that are worthless (like something that generates nothing but "QZCZQ" type text) and spends the most time on "decodes" that statistically look like English text.


Cracking the Zodiac Killer's "340 Cipher" Part 2

| No Comments | No TrackBacks

As I mentioned in yesterday's article, I am working to crack the "340 cipher" sent to police by the Zodiak Killer, who operated in the 1960s and 1970s in California. I also mentioned the assumptions I've made about the message (which could well be wrong) and the staggering size of the potential solution space. Clearly I needed to shortcut that 100-year process as much as possible since it's very unlikely I'll live to be 140 to see the end of it.

Some of the shortcuts I can take include:

  • I know that all of the symbols can't translate to the exact same letter, though it's highly likely that several of them do represent specific letters. Thus, I can (probably) discard any potential message key that has "too many" of the same letter. That reduces the size of the solution space a good bit.

  • I know that when the message is cracked, it's highly likely that there will be a pretty standard breakdown of the letters as seen in typical English texts. By spending a minimal amount of time on any key that generates a "possible solution" of the message whose character breakdown is too much "off" from that breakdown, I can speed up my trip through the solution space.

  • I know that when the message is cracked, it should contain a certain percentage of the most popular bigrams (2-letter combinations) and trigrams (3-letter combinations) found in English texts. By checking a possible solution against those percentages, I can avoid wasting time on "solutions" which are filled with unlikely bigrams (such as "QZ") and trigrams ("QZQ").

  • I know that the message is most likely written in English, so I can build a dictionary of the English language from online sources and compare any possible solution which has the right breakdown of characters, bigrams, and trigrams against that dictionary to see how many real words are in the message. The more words we find in the possible solution we're looking at, the more likely I'll have found the "right" solution.

  • Based on my analysis of the enciphered message, there are some symbols that occur too frequently to be likely to be letters like Z, Q, or X. When trying potential keys, I can discard those which are attempting to replace those symbols with characters they're unlikely to be.

Not being a mathemetician (I blame the lousy Calculus teachers I had at Syracuse University for squelching my confidence in my ability to do math), I have no idea just how much the above will cut down my solution space. Still, I expect it slices the space down pretty handily overall. To implement the above rules, I developed a scoring system that requires a potential solution to pass through a number of "gates" before going on to an analysis that is more thorough or computer time consuming. The scoring method works something like this:

  • I generate a possible message key.
  • I attempt to decrypt the message using that key.
  • I run the potential solution through a character counter to verify that it has approximately the right number of the "most common" characters. If not, I move on to the next key.
  • I compare the number of individual characters found against their typical frequencies in English. If there are too many or too few of a given character than expected, I move on to the next key.
  • I compare the message against the most common bigrams and trigrams used in English. If those don't occur in approximately the right proportions, I move on to the next key.
  • I compare the message to a dictionary of 20,000 English words. I weight the scoring in favor of larger words, and heavily in favor of words the killer used most frequently (especially those he liked to misspell). The more words I find and the larger the words are, the higher the "score" I get for the message.
  • I analyze the percentage of the characters in the solution that are "swallowed up" by the words I found in the message. The higher the percentage of "words to overall characters" the more likely this key is to have broken the message, so the higher the score will be.

In the next installment, I'll talk more about the program's logic to try and accomplish the above.

Cracking the Zodiac Killer's "340 Cipher"

| No Comments | No TrackBacks


I've not been posting a lot of new articles into the blog lately, and I thought it was about time I explained why. Aside from the fact that things have gotten busier at the office, and at home, I've also been channeling what little energy I do have into a few projects. First is to publicize my spam-inspired cartoon site, next is to publicize my site to help bloggers find ideas, and finally (which is the point of this little missive) to try and crack a very old cipher written by a serial killer about 30 years ago.

The serial killer in question is the Zodiac Killer, who operated in the San Francisco area in the 1960s and 1970s. He killed an unknown number of people, but took credit for double-digit numbers. In spite of the fact that he taunted police by writing letters to them and to the news media, he was never caught.   The last communication known to be received from him was a 340-character cipher which has never been cracked (at least it isn't publicly believed to have been cracked). I decided to take a crack at it.

I should begin by stating that I am not a cryptographer or any kind of an expert in the subject. I'm a computer geek, to be sure, but have no special training or background in such things. Regardless, I do have a morbid curiosity to know what this cipher says and what it might reveal about the killer. I'm also very curious to see if I can design and write a program which will crack this cipher.

Having read a bit about cryptography, I know that there is a pretty consistent frequency with which letters appear in English texts. I know that there are also certain pairs of letters which tend to appear together ("bigrams") and certain 3-letter combinations which tend to appear more frequently together ("trigrams"). Cryptanalysts uses this information to help them find the key used to decrypt messages. I've found and made use of this same data in the work I've done thus far.

I began by analyzing the known writings of the Zodiac Killer, verifying that his letter frequencies match typical English letter frequencies (they do), that the bigrams and trigrams in his writing occur approximately the same as in normal English texts (they do), and building a list of his "vocabulary" used in previous messages. Armed this this information, I was fairly confident that if in the future I do crack this cipher, any computer program I write should be able to use standard cryptanalysis tactics to identify a break.

The encoded message in question is referred to by analysts of the Zodiac Killer as "the 340 cipher" because it is written as 20 rows of 17 symbols long (20 x 17 = 340). There appear to be 62 individual symbols and/or letters used in the message. It is likely that the Zodiac used the "extra" symbols to make it harder to identify the most commonly used letters in English (e.g., he may have used 4-5 symbols to represent the letter "E" and the letter "T").

Before I could begin instructing a computer to attack this cipher, I had to make some assumptions about it, which I fully recognize could be completely wrong. Still, I had to start somewhere if  I was going to break the thing. My working assumptions at this point are the following:

    • The killer's previous ciphers were all simple substitution ciphers (e.g., the killer substituted one letter or symbol for another, and any time he used the same symbol it meant the same letter).
    • The killer's previous ciphers are all written in English, and thus this cipher is also in English.
    • The cipher contains an actual message and isn't just random scribbling that the killer sent to annoy the police and cryptanalysts.
    • When properly deciphered, the message will yield a string of words with no punctuation in them, just like the killer's prior ciphers.

In the next article, I'll discuss the method I used to build a program to try to crack this cipher.

Internet Domain Name Registration Basics

| No Comments | No TrackBacks

What is a Domain Name?

If you look at your web browser's address or location bar, you'll usually see in it a URL like "http://www.mikesalsbury.com" (the URL for this site). The "domain name" for this site is therefore "mikesalsbury.com". Since that domain name is already registered to me (and will be for some time), you can't register that one for your own site, even if your name happens to match mine. But you could register some other domain name that you like, such as "mike-s.com" if that's available. All that domain names really do is make it easier for human beings to remember the address of your web site.   Without domain names, we'd have to give people URLs like "199.205.42.113" to find our sites, which wouldn't be as easy to remember as "mikesalsbury.com" or "gamerhotsheet.com".

How Do You Get a Domain Name?

Getting a domain name is actually pretty easy. You find a company that has the authority to register domain names with one of the Internet authorities, pay them a small fee, and they'll register the name for you. This assumes that the name you want is not already registered to someone else.

Once you've registered a domain name, it's yours for at least one year. Some registrars allow you to register the domain name for several years in advance. Pricing can vary greatly. Some registrars will allow you to register a ".com" domain name for as little as $2.99. For example, Yahoo Small Business is currently allowing new customers to register domain names for $2.99 for the first year. GoDaddy.com offers domain registration for $1.99 if you purchase some other product, such as their web hosting services. A quick Internet search should reveal any number of registrars and prices.

M4 Message Breaking Project Posts Second Success!

| No Comments | No TrackBacks
Earlier I mentioned that I've been donating CPU time to the M4 Message Breaking Project, which is a distributed computing effort intended to crack some previously undecoded German military messages from World War II.  The project decoded one message pretty early on.  It failed to decode the second message after several tries, so it moved on to the third (planning to come back to the second again later to do a more thorough run against it).  The third message cracked some time in the last several hours.  It's a bit garbled, but the experts are working on it to complete the translation and determine exactly who sent it.

It appears to say:

On escort course 55 degrees nothing found, following given grid (square). Position AJ3995. [wind] South East 4, seedrem(?), 10/10 overcast, [barometer] 28mb risen, visibility in fog 1 nautical mile

And it appears to be from a captain Schreeder, Schroeder, or Schreiber.  (There's some debate as to exactly what the name is due to some of the garbling, potential mis-keying, mis-spelling, etc.)

Regardless, it feels very cool to be a part of a project that's uncovering a tiny piece of history.

The final message is currently being attacked.  If you're interested in taking part in the project or just monitoring its progress, the official site is http://www.bytereef.org/m4_project.html

 

Sad But True Tech Support Stories

| No Comments | No TrackBacks

I work in the IT Division of a major Columbus, Ohio, area non-profit organization.  We have about 1,200 employees and, due to the nature of our business, most of them are computer users.  Since we handle the "front lines" of tech support (i.e., taking the phone calls, desk-side repairs, new system deliveries, and the like), we end up working with almost everyone eventually.  Some of the things we see are pretty amusing.  Some are a bit sad. During a recent meeting we were (I guess the right word is) "inspired" to recall a few recent ones.  Here they are...

The New AOpen Pandora

| No Comments | No TrackBacks

AOpen has announced its new "Pandora" mini PC.  This ultra-small-form-factor PC should look very familiar to anyone who recognizes Apple's Mac Mini:

On the left above is the AOpen Pandora.  On the right is the Apple Mac Mini.  While there's a definite "copycat" factor here, it does show that Apple's not the only one who can produce a small, stylish little box.

 
AOpen's
Pandora    Apple's Mac Mini
Separated at Birth? AOpen's Pandora (left) and Apple's Mac Mini (right)

My Experiences with Virtual Machine Software

| No Comments | No TrackBacks

As part of the research I've been doing into how easy it could be for a Mac OS X user to switch to Linux, I decided to load one of the more popular UNIX distributions and familiarize myself with it.  I didn't want to dedicate an entire PC to Linux, so setting up a virtual machine in which to test Linux seemed the right approach. 

What is virtual machine software?  That's hard to explain, but I'll give it a shot.  Virtual machine software "pretends" to be a complete computer, only it's software rather than hardware.  A good virtual machine program will allow an entire operating system to run inside it, with that operating system completely unaware that it isn't running inside its own PC.  There are many uses for virtual machine software, including testing and debugging operating systems, performing security testing ("honeypots" to attract viruses/worms, for example), and testing software with multiple operating systems without having to devote an entire PC to one OS.  In this case, having virtual machine software will allow me to run Linux on a Windows XP Pro system without disturbing my Windows installation.  Linux will be there when I want it or need it.  All I have to do is launch the virtual machine.