Of course, no one is trying to win a design contest here. What people care about is, are the results any good.
A blind monkey with a hammer and a learning disorder can get you to IBM.com when you search for IBM. What matters is can Live Search give you good results when the answer isn’t obvious. We devised a set of Search Engine Results Evaluation Criteria which you can read in detail. Or, here is the short, short, version:
To test, we ran a series of searches. The topic is Comodo Antivirus. Comodo is hardly a secret, but it is not a household name either. The term “antivirus” gives us a good search engine test because it can be spelled multiple ways, it is a generic term that is used without Comodo, and there are plenty of ad filled junk sites out there that target the word antivirus. Finally, Comodo is better known for its firewall software than its antivirus product.
What grade does Windows Live Search get?

click to enlarge
The search for comodo antivirus (no quotes) yields 795,000 results. The top search result is the official product page for Comodo Antivirus. That’s good.
The next six results are on the official Comodo website, with the exception of the Wikipedia entry. We aren’t going to quibble with what page should be 4 and which page should be 5. For this first glance, we only care if the results are useful.
The first site that is an unknown is the 9th result. The result is from a page at trustix.com. The only result on the page that even has the word Comodo in it says “Comodo Product”. A bit of research turns up that Trustix is long out of business, having been bought by Comodo and then shuttered in December 2007. We’ll have to call this one a “bad” result.
The 10th result is the download page for the Beta version of the software from Major Geeks. Major Geeks is a large site and therefore has some pull. One might argue that this isn’t a very good result since the only thing on the page is a copy and paste from Comodo’s copy and download links. But, a two word search can only give the search engine so much of an idea of what you actually want. In this case, Live Search is giving you a way to download what you searched for. That can’t be regarded as a wrong result.
On the first page we get 9/10 “good” results. So far, the first glance looks O.K. Let’s go a step deeper with Part 2.