Google and the evolution of search – human evaluators

For many years, Google, on its Explanation of Our Search Results page, claimed that “a site’s ranking in Google’s search results is automatically determined by computer algorithms using thousands of factors to calculate a page’s relevance to a given query.”

Then in May of 2007, that statement changed: “A site’s ranking in Google’s search results relies heavily on computer algorithms using thousands of factors to calculate a page’s relevance to a given query.”

A slight adjustment in wording, but an important comment on the supremacy of the algorithm that Google had touted for years. Google had finally acknowledged that its search results were no longer solely and automatically determined by the company’s vaunted algorithms. Now they simply “relied heavily” on them. Why the sudden change?

Google claims it was arbitrary, unrelated to any sudden philosophical shifts within the company. But it seems far too specific an adjustment to chalk up to a random brand-management edit. We are, after all, talking about the company’s official explanation of its search results. And indeed, sources say the language was changed to account for the continual calibration of the algorithm, which these days is done with a bit of human help.

Google, for example, employs a vast team of human search “Quality Raters” (You’ll find a copy of an old training manual here). Spread out around the world, these evaluators, mostly college students, review search returns against established criteria–testing different algorithms and see which works “best” in predicting the quality of a site (though not directly judging the quality of any individual site itself).

They’re aided by Google’s own registered users, who can now, when logged into their Google accounts, promote and delete sites from their own search returns according to their preferences. These data too are used to tweak and further optimize the algorithm. So Google’s objective evaluation and ranking of Web sites is to some extent defined by subjective reasoning of a collective human intelligence. And so it must be if Google is to continue returning search results that we perceive to be the “best” answers to our search queries.

In interviews serialized over the next three days, key Google engineers with central roles in managing the company’s search engine discuss resources and techniques they use to optimize the system for users world-wide. The series kicks off below with Engineering director Scott Huffman, who oversees the company’s search evaluation team. Senior Google software engineer Matt Cutts appears tomorrow. And Google Fellow Amit Singhal wraps up the series on Friday.

Report: Gmail about one-third as expensive as hosted e-mail

What does it cost to host an e-mail account? It seems like a simple question, but a remarkable number of enterprises surveyed by Forrester had no idea of how to answer that question. A new report by the research company has taken a look under the hood of both in-house and commercial e-mail services, and put some numbers on the per-user costs associated with a variety of options. The surprise result was not so much that Google’s corporate services come out ahead, but rather how large a lead it has on every other option.

The title of the report, “Should Your e-mail Live In The Cloud? A Comparative Cost Analysis,” is actually somewhat misleading. The cloud implies a diffuse network of servers that hold partially redundant copies of information. Some of the services examined by the report don’t necessarily offer that sort of setup, although the report frequently refers to any off-site service as “the cloud.”

Semantics aside, Forrester surveyed over 50 IT workers at major enterprise companies to see how they handle e-mail, contacts, and calendaring services. They also discussed options and costs with 21 vendors of these services, and created some rough estimates of how much each portion of the various services cost.

One of the things they discovered is that the business community is largely unaware of the costs of running an e-mail account. Many of those surveyed gave guesses from $2 to $11 per user, although a detailed accounting showed that the costs were often several times that (Forrester came up with $25.18 per month, compared with $8.47 for Gmail). Part of the problem is that costs are often split among several cost centers, with software licenses part of a different department’s budget from the salaries of the people that support it. In some cases, the e-mail system was running on older hardware that had initially been bought for a different purpose and had been depreciated.

Despite the confusion, a lot of companies realize that e-mail has become expensive for two simple reasons: spam and malware. Nearly half of those surveyed were evaluating off-site solutions because e-mail costs had risen, while another 30 percent were performing the evaluation as part of an upgrade or service consolidation process. Fully 85 percent of these companies were leaning towards moving some of the services off-site.

The biggest reason for doing this seemed to be so that someone else could deal with staying on top of spam and malware; over half of those surveyed were planning on implementing a hybrid system where an external service filtered mail on its way into and/or out of the company’s internal servers. Reasons cited include the challenges of staying on top of the threats and up-to-date with the software, as well as the resource-intensive nature of combating mal-mail. Another 30 percent were leaning towards a complete outsourcing of the service, presumably in part because of these costs.

The author of the report calculated the monthly costs for the components of various systems, such as storage and client software. The biggest cost was clearly archiving, which is often legally required for a lot of positions. Beyond that, the software and filtering costs all came in at roughly 10 to 15 percent of the costs when they’re needed—off-site services, for example, eliminate separate purchases of server and filtering software, and lower staff costs in exchange for a monthly subscription. The overall conclusion is that any company with an employee count of under 15,000 would probably benefit from using off-site services.

The two examples of actual cloud services, Microsoft’s Exchange Online and Google Apps for Business, came out significantly ahead. Exchange Online provided significantly lower costs until somewhere above 30,000 seats, while Google Apps’ monthly cost consistently came in at half the cost of others, in part because its subscription cost is so low, and in part because the “client software” is a free web browser.

The author of the report cautions that there are a lot of variables to consider, such as how often the company adds and removes users, the frequency of large attachments, and the archiving requirements. Still, the results make it clear that Microsoft is now undercutting most of other services available, including those that rely on Exchange itself. But Google has managed to significantly undercut Microsoft. Although its solution is nowhere near as integrated as Exchange, an increasing percentage of the workforce is getting comfortable with managing their life and e-mail through a web browser.

Google pushing users away from IE?

Google has taken the aggressive step of advising some of the people using its Gmail webmail to use Chrome or Firefox rather than Internet Explorer.

When users log into their Gmail using Internet Explorer a red text link appears at the top right of the page saying ‘get faster Google Mail.’

If you click on the link then you are taken through to a Google answers page that suggests that you should use a faster browser.

The suggestions it makes are Google’s own Chrome browser or Firefox 3.0.

We suggest you upgrade

“Browsers are getting faster and better at running web applications like Google Mail that use browser technology to its limits. In order to get the best Google Mail experience possible, we suggest that you upgrade your browser to one of the fastest Google Mail supported browsers that work on Windows,” reads the text.

There is a proviso that IE8 is being worked on ‘Note: A faster version of Internet Explorer, IE8, is in development and available in a beta release.’

Although not all users appear to be affected in our early investigation, at first glance it is a particularly aggressive approach from Google.

To actively push two browsers over the currently dominant Internet Explorer is far from the normal Google softly, softly approach – especially in a week where Internet Explorer has been beset by news of a major security problem.

It seems, however, that Google is only pushing users to the other browsers if they are currently using Internet Explorer 7. Those that are using Internet Explorer 6 are told to upgrade to either Chrome, Firefox or Internet Explorer 7 for a faster Google service.

Whether this is a silly overview on Google’s part or an active push away from Internet Explorer by the search kings remains to be seen.

What is for sure, is that Google is sending out mixed messages to its users, depending on what version of browser they are using.

Google Chrome Browser to support customization

Google has launched an effort to make it possible for developers to offer ad-blocking and other extensions for Chrome, a move that would give the Google Web browser the same level of customization as Mozilla Firefox.

The ability to install third-party applications that add capabilities chosen by users, but not provided by Mozilla, is a key reason for the open source browser’s popularity. Google is apparently borrowing from that playbook in proposing the extension system to Chromium, the open source project behind the development of Chrome.

Google’s proposal was introduced over the weekend in a blog post from Aaron Boodman, a Google programmer working on Chrome. The design document outlines areas that would have to be addressed, such as application programming interfaces to connect extensions to the Chrome engine.

Under the heading “use cases,” Google lists some types of extensions that the company would like to support in Chrome, such as ad and flash blockers. Google makes its money from selling Web advertising but has decided not to ignore two of the most popular Firefox extensions. Other third-party apps Google says it would support include bookmarking/navigation tools, download helpers, and privacy and parental controls.

Having an add-on system from Chrome tops users’ wish list. “If I can’t even add a third-party extension, this browser won’t stay long on my computer,” one person wrote on the Chromium forum.

Google did not set a timetable for releasing an extension system for Chrome, but the design documentation for Chromium developers indicates the search engine has already started to work on the technology.

Google designed Chrome to be lightweight and fast, to have a minimalist user interface, and to resist crashing under have JavaScript demands of Web applications. While a reviewer for InformationWeek believes Google has largely met its goal, not having an extension system gives rival Firefox the upper hand. Microsoft also doesn’t provide an open extension system for Internet Explorer.

Extensions give users more choices in customizing the browser to meet their needs, while relieving the browser maker from having to add a lot of features that can hinder performance. Internet Explorer accounts for more than 70% of the browser market, followed by Firefox with almost 20%. Chrome, which is in beta, has less than 1%.

Blackberry Storm proves worthy rival to iPhone

To its fiercest devotees, one of the best things about the BlackBerry is its carefully designed physical keyboard, which the skilled BlackBerry addict can play like a violin. These folks scorn Apple’s popular iPhone, whose keyboard is virtual and must be operated by tapping on the screen.

But, on Friday, Verizon Wireless and Research in Motion, the BlackBerry’s maker, will do the unthinkable: They will introduce a BlackBerry model without a physical keyboard, one where typing and navigating require tapping on glass, just as users do on the iPhone. This new model is called the BlackBerry Storm, and will sell for $250 with a two-year contract, though a $50 mail-in rebate can bring the price down close to the $199 that Apple charges for the base model of the iPhone.

Despite its lack of a keyboard, the Storm is a real BlackBerry in every other respect, with push email, corporate features and the familiar BlackBerry menus. In many respects, the Storm is a touch-based, large-screen version of the recently released BlackBerry Bold, which is the most polished version of a traditional BlackBerry. It is also the latest member of the new class of hand-held computers, the super-smart phone category kicked off by the iPhone last year and joined by the Google G1 earlier this year.

The Storm sports a large, high-resolution touch screen that fills most of its surface and automatically switches from portrait to landscape mode when the phone is turned. There’s also a forthcoming souped-up download store for third-party software, meant to be similar to the ones on the iPhone and the Google phone. And the Storm can even be used in European and other countries where most Verizon phones don’t work.

However, the biggest innovation in the Storm is a clever feature RIM hopes will give it a big advantage over the iPhone. When you strike a key or icon on the Storm’s screen, you feel a physical sensation, as if you were pressing down on a real key or button. That’s because you are, in fact, pressing a real button. The entire glass display is one large button, mounted on a mechanical substructure that allows it to be depressed when pressure is applied.

The idea behind this feature is to make typing on glass feel much more like typing on a real keyboard, and thus to make the virtual keyboard, and the touch interface, more acceptable to people used to physical keyboards and buttons. This push-down screen also replaces the side-mounted scroll wheel or track ball on other BlackBerrys for activating menu choices and icons.

But, in my tests, this physical feedback feature, which RIM calls SurePress, didn’t magically turn the Storm’s touch interface and virtual keyboard into their physical counterparts. The feature does provide a more reassuring confirmation that a key has been struck or an icon has been clicked than the mere visual feedback one receives from the iPhone. But neither I, nor any of the several BlackBerry addicts I asked to try it out, considered typing on the Storm’s keyboard to be very similar to using the keyboard of a traditional full-sized BlackBerry.

In my opinion, using the Storm’s keyboard is much more like using the iPhone’s keyboard than a traditional BlackBerry’s. I found that I could type quite well on the Storm after awhile, but that a greater adjustment, and more practice, were required than with a physical keyboard.

The Storm also has a keyboard oddity that I found annoying, and that may put off others. It presents you with a full virtual keyboard only when you are holding it horizontally. When you hold the Storm vertically, you get a mashed-up keyboard, like the one on the narrower BlackBerry Pearl, which has multiple letters on each key. This keyboard design relies on software to guess which letter you meant to press. You can also switch to a virtual cellphone-style keypad that requires you to hit each key multiple times.

This is a curious design decision. Once a company ditches a physical keyboard for a virtual one, it can create all kinds of keyboard variations. RIM could have offered a full, vertically oriented keyboard, even if it would have had smaller, more closely spaced keys.

RIM also failed to customize the Storm’s virtual keyboard for some common, specific tasks. For instance, on the iPhone, when you are typing in a Web address in the browser, the keyboard morphs to offer a convenient key that automatically enters “.com”. Not so on the Storm.

There’s another glaring deficit in the Storm: It lacks Wi-Fi capability. This means that, unlike on the Bold, the iPhone or the Google G1, if high-speed cellphone data service is absent or pokey, you can’t fall back on speedy Wi-Fi connections in public places. And, at home or in the office, you can’t take advantage of Wi-Fi connections that are often much faster than cellphone data networks.

The Storm has some important advantages over the iPhone. Its screen, while 7% smaller physically, offers about 13% higher resolution. Photos and videos look beautiful on it. It has much better battery life for phone calls than either the iPhone or the Google G1. While the latter two phones deliver just under their claimed five hours of talk time, in my tests, the Storm lasted a bit over six hours, which is actually half an hour more than its claimed 5.5 hours of talk time. And the Storm has a removable battery, unlike its Apple rival.

This new BlackBerry comes with more memory than the similarly priced base model of the iPhone — nine gigabytes versus eight gigabytes. And, unlike the iPhone’s memory, the Storm’s is expandable, via larger flash cards.

The Storm’s camera is much better than the iPhone’s, at 3.2 megapixels, versus just 2 megapixels for the Apple device. It also has zoom and flash, features the iPhone’s camera lacks. And, unlike the iPhone or the Google G1, the Storm can record videos. In my tests, all of these camera features worked well.

Also, the Storm has copy and paste functionality; MMS (a service for sending photos directly to other phones without using email); voice dialing; and the ability to act as a modem for your laptop. It also allows you to edit, and not just to view, Microsoft Office documents. All of these features are missing from the iPhone out of the box.

The Storm also has a better speaker than the iPhone, and a noise-canceling microphone system. Phone calls, even on speaker phone, were crisp, clear and plenty loud. Physically, the Storm is attractive but hardly svelte. While it’s about the same length and width as the iPhone, it is 15% thicker and 17% heavier — almost as heavy as the chunky G1.

The Verizon high-speed network on which the Storm runs is older and better-established than either the T-Mobile high-speed system the G1 uses or the AT&T 3G network used by the current iPhone. Where Verizon’s high-speed data coverage is strong, the Storm flies.

But, because it lacks Wi-Fi, the Storm can be much slower at Web access than its main competitors. I tested these Web speeds in two hotels in Silicon Valley. In the first, where Verizon reception was strong, the Storm trounced the iPhone on cellphone data speeds, averaging over 800 kilobits per second to the iPhone’s 621 kbps over AT&T. But, when I switched the iPhone to use the hotel’s Wi-Fi network, it beat the Storm by 100 kbps or so.

At the second hotel, barely a mile away, the Storm’s lack of Wi-Fi hurt much more. There, Verizon’s signal was poor, and data speeds on the Storm averaged a horrible 96 kbps. But the iPhone on AT&T averaged 459 kbps, and on Wi-Fi the iPhone averaged 785 kbps.

My test Storm, which was a near-final model missing only a few minor software tweaks, was also sluggish at some tasks. It took noticeably longer than the iPhone to flip the first photo from landscape to portrait orientation, or to start the process of flipping through a series of photos by swiping them with a finger. And some other tasks were also slow. It’s possible that production models will be quicker.

BlackBerry Storm’s touch screen switches from portrait to landscape mode when turned, and aims to make typing on glass feel more like typing on a real keyboard.

Rim has tweaked the familiar BlackBerry user interface for the touch screen, and in general these changes worked well. You select the menu item or icon you want with a light touch, then press down on the screen to activate or confirm your choice. There are even a couple of cool new touch features. For instance, in a list of emails, if you lightly touch and hold one entry, the Storm shows you all messages in that thread.

But this combination of a light touch followed by a hard press on the large screen took some practice, just like typing did. It befuddled several BlackBerry veterans at first.

And some common tasks took more steps than on the iPhone. For instance, emailing a link from a Web page required four steps on the Storm, versus two on the Apple device. The Storm’s email system will be familiar to every BlackBerry user. It has the same corporate email features as other BlackBerrys, and I was easily able as well to use a BlackBerry Internet email account and to set up several personal email accounts, including Gmail.

The Web browser is much improved over the one in older BlackBerry models, and offers multiple ways to view and navigate pages, including one in which a finger moves a cursor, just as on a PC. But I found that panning and zooming in the browser was a bit slower and more awkward than on the iPhone. And, to make some Web sites work properly, I had to dig through menus to change options.

Using the BlackBerry desktop software, I was easily able to synchronize my calendar and contact data over a cable from a Windows PC. (There’s also Mac software for the same task.) But, unlike the iPhone or the G1, the Storm doesn’t offer wireless synchronization from consumer services, only from corporate servers.

The Storm’s multimedia software isn’t as fancy as the iPhone’s, but it’s better than the G1’s, and worked very well in my tests.

Overall, the Storm is a very capable handheld computer that will appeal to BlackBerry users who have been pining for a touch-controlled device with a larger screen. And it offers yet another good option for anyone who is looking to buy one of the new, more powerful, pocket computers.