Facebook – Keeping Up

Recently posted on the Facebook blog:

Almost two million new users from around the world sign up for Facebook each week—and we couldn’t be happier. It’s tremendously rewarding to see so many people find what we work on useful and fun. As we continue to add new users and features, however, the load on our thousands of servers continues to increase at a pretty astounding rate. A few weeks ago we reached full capacity in our California datacenters. In the past we handled this problem by purchasing a few dozen servers, hooking them up, and getting on with our lives, but this time we didn’t have it so easy. We’d actually run out of space in our datacenters for new machines.

Fortunately we saw this problem coming a long time ago and started work on a new datacenter in Virginia. Now, we identify whether a user would be better off talking to the east coast datacenter or a west coast data center. For people in Europe and the eastern half of the US, it’s noticeably faster to talk to a server in Virginia than in California. For these users we direct them to Virginia whenever they’re browsing the site and not making any changes.

Whenever that person goes to change some data—uploading a photo album, or changing profile info for example—we send them off to California so that all our modifying operations happen in the same location. This decision was made to prevent two or more modifications from conflicting with each other and messing up our data. It might sound like we’re forcing our users to go to California a lot but only about 10% of our traffic causes a modifying operation. MySQL has a great replication feature that allows us to, in real time, stream all the modifications happening on a California MySQL server to another one in Virginia. Replication happens so fast, even across the country, that the Virginia servers are almost never more than one or two seconds behind the California servers.

Even though all of the modification happens in California and streams instantly to Virginia, we were faced with another problem. Although Facebook’s data is stored in MySQL database servers, we use a large number of memcached servers to store copies of the data. Memcached is much faster and able to keep up with requests quicker than the databases themselves can keep up. We had to figure out a way for memcached servers to replicate data concurrently with the MySQL databases. Because of various technical limitations of our architecture there was no easy way to do so.

Fortunately MySQL is open source software, meaning we can actually change the way it works by modifying the code. We did just that—embedding extra information in to the MySQL replication stream that allows us to properly update memcached in Virginia. This ensures that the cache and the database are always in sync. Over the last seven months a great team of Facebook employees has been building new software and setting up new servers like I described above. Over Thanksgiving we finally flipped the switch and since then almost 30% of our traffic has been served from Virginia.

The east coast datacenter is a great first step towards keeping Facebook fast and reliable as the site grows. Going forward we have lots of exciting plans to expand our infrastructure and improve performance so no user ever has to sit around waiting for a page to load.

Pentagon bans Google map-makers

The US defence department has banned the giant internet search engine Google from filming inside and making detailed studies of US military bases.

Close-up, ground-level imagery of US military sites posed a “potential threat” to security, it said.

The move follows the discovery of images of the Fort Sam Houston army base in Texas on Google Maps.

A Google spokesman said that where the US military had expressed concerns, images had been removed.

Google has now been barred from filming and conducting detailed studies of bases, following the discovery of detailed, three-dimensional panoramas online – and in particular, views of the Texan base.

“Images include 360-degree views of the covered area to include access control points, barriers, headquarters, facilities and community areas,” said the defence department in a statement quoted by AFP news agency.

It said such detailed mapping could pose a threat.

Google spokesman Larry Yu said the decision by a Google team to enter the Texas base, which is in San Antonio, and undertake a detailed survey, had been “a mistake”.

He told the BBC that it was “not our policy to request access to military installations, but in this instance the operator of the vehicle with the camera on top – which is how we go about capturing imagery for Street-View – requested permission to access a military installation, was given access, and after learning of the incident we quickly removed the imagery”.

Individuals and governments
Military officials are currently looking into exactly what imagery is available – though it may not be able to order its removal if images are taken from public streets.

Among the popular mapping services offered by Google are Street View, which allows web users to “drive” along virtual US landscapes with ground-level views, and Google Earth, which offers detailed satellite and 3D images of locations around the world.

In this case, it was imagery offered on Street View that caused the concern.

But both have provoked complaints – from individuals depicted in the images and from governments concerned that satellite images could compromise security.

Gary Ross, a spokesman for the US Northern Command, told AFP that although such services could be useful, “there has to be a balance”.

But Mr Yu said Google would listen to concerns about privacy and security.

“We try to have a compliant image removal policy – not only relative to the military but to consumers also,” said Mr Yu.

“If people have concerns, they should contact us.”

Microsoft still wants to be welcomed to the social

Microsoft showed off two social-networking projects at TechFest on Tuesday that show that the company wants to do more in this area than just invest in Facebook.

One project, known as Salsa, aims to use one’s corporate data to piece together their social network, or at least their network of co-workers. In its current form, the software is a plug-in to Outlook that shows social-networking information such as a photo and profile next to an incoming e-mail message. The program also pieces together a list of “friends” based on e-mail frequency and other data.

“When you start looking there is a surprising amount of information that gets locked in e-mail,” said Shane Williams, one of the Microsoft Research team that worked on Salsa.

Lili Cheng, the Microsoft veteran who heads the social-computing team at Microsoft Research, said that part of the power of Salsa is simply putting a human face on e-mail. She said her own use of the site has borne out the power of that, noting it is harder to argue with a colleague when she sees a picture of them with their cute kid or pet.

“E-mail can be very dehumanizing,” Cheng said.

Cheng said that in addition to deploying it inside Microsoft, she’d like to see how Salsa works within one or two other large companies to see if it is more broadly useful.

In another project from Cheng’s group, known as C2, Microsoft researchers have created a Windows application that pieces together contact data from a variety of social-networking sites. For the purposes of Tuesday’s demonstration, the researchers focused on Windows Live Spaces and Facebook. Researcher Steve Ickman said he chose those two because they represent among the most open (Spaces) and closed (Facebook) when it comes to data sharing.

Although Facebook is notoriously restrictive when it comes to members scraping their data, Ickman said that he believes he was able to stay within Facebook’s terms of service by grabbing only approved data from one’s own contacts and not caching the information long-term. “It’s totally legal, at least at this point.”

The project is more of a technology demonstration than anything geared toward a specific product, Ickman said, adding that he hoped it would demonstrate to the product teams that they can be more ambitious. “We tend to cancel things because they are too hard,” he said.

Goodbye Netscape Navigator, AOL says no longer supported

A web browser that gave many people their first experience of the web is set to disappear.

Netscape Navigator, now owned by AOL, will no longer be supported after 1 March 2008, the company has said.

In the mid-1990s, as the commercial web began to take off, the browser was used by more than 90% of people online.

Its market share has since slipped to just 0.6% as other browsers such as Microsoft’s Internet Explorer (IE) and Firefox have eroded its user base.

The company recommends that users upgrade their browser to either Firefox or Flock, which are both built on the same underlying technologies as Navigator.

“I think we represent the hope that was of Netscape,” Mitchell Baker, chair of the Mozilla Foundation which coordinates development of Firefox, told BBC News.

“We have picked up many of the things that Netscape launched but we’ve taken them further in terms of openness and public participation.”

Ms Baker was one of the first employees at Netscape in 1994.

Web window
Netscape was created by Marc Andreessen who as a student had co-authored Mosaic, the first popular web browser.

His company Netscape Communications Corporation released the first version in 1994.

According to Shawn Hardin, President and CEO of Flock, Netscape played an important role in making the internet “a relevant mass market phenomenon”.

“Netscape had a critical role in taking all of these zeros and ones – this very academic and technical environment – and giving it a graphical user interface where an average person could come online and consume information,” he told BBC News.

“During its halcyon days it really felt like the internet and Netscape were really the same thing,” he said.

Other companies capitalised on Netscape’s success, notably Microsoft, which began to bundle IE with its Windows operating systems.

Although this led to legal wrangles over anti-competitive behaviour, IE now dominates the browser landscape with an 80% market share.

As a result, Netscape became unviable.

“While internal groups within AOL have invested a great deal of time and energy in attempting to revive Netscape Navigator, these efforts have not been successful in gaining market share from Microsoft’s Internet Explorer,” said Tom Drapeau on the Netscape blog last year, when the demise of the browser was first announced.

Future return?
For the past week Netscape users have been shown a message alerting them to the end of support for the browser.

“Given AOL’s current business focus, support for Netscape browsers will be discontinued as of March 1st, 2008,” the message reads.

It then suggests users upgrade to either Flock or Firefox.

Firefox is the main competitor to IE, particularly in Europe where it has a 28% market share, according to some statistics.

The open source browser’s development is coordinated by the Mozilla foundation, set up by Netscape staff made redundant in 2003.

It has had more than 500 million downloads worldwide and in countries such as Finland it is the most popular browser.

“Competition is what brings quality,” said Ms Baker.

Flock describes itself as “the social web browser” and allows people to see feeds from community websites, such as Flickr and Facebook, and post to blogs without having to navigate to the page.

“There are lots of ways that people are engaging in having a conversation and Flock is very focused on making that as effortless and convenient as possible,” said Mr Hardin.

However, not all Netscape users are happy about having to change browser.

“I’m sad. Flock still needs improvement and I am not happy with Firefox’s interface. I’m [an] orphan!” read one post on the Netscape blog.

Others who posted comments on the blog predicted the browser will make a return.

“Netscape is a wonderful browser, and it will be so in the future,” read one.