Webware

On June 4, 2009, Google.cn blocked all searches for "Tiananmen Square," even ones not related to the massacre that took place on that date in 1989. It refuses to say why.

(Credit: Screenshot by Tom Krazit/CNET)

Google was going to help democratize data in China. Instead, about three years after entering the Middle Kingdom, the search company still finds itself in an uncomfortable working relationship with government censors.

For about eight days between June 3 and June 11, Google.cn blocked all results that might come from searches for Beijing's Tiananmen Square. Not just politically sensitive results, not just historical accounts of the hundreds of deaths on June 4, 1989, but every single result--including directions to the square--with an error message that read "Search results can not be displayed as they may contain contents that do not comply to related laws and policy."

As of Thursday, things had appeared to return to normal. A search for "Tiananmen Square" in either English or Chinese brought up links to shops in the area, historical documents about one of China's most storied places, and images of fun, happy times in downtown Beijing.

So how did Google know that it was supposed to drop the hammer on all results for Tiananmen Square for that brief period of time? And how did it know that it was once again safe to reapply the limited filter?

Google isn't saying, beyond pointing to previous interviews and statements it has given on its tricky balancing act in China. "Google.cn complies with Chinese laws. The differences in search results over time in China are the result of a variety of factors, including the content that is available on the Internet and the regulations we follow in China," the company said in a statement last week.

But it has confirmed that Google has dropped a previous method of determining how to self-censor its search results--pinging the so-called Great Firewall of China to see what sites are blocked--in favor of a new self-censorship method that the company refuses to disclose.

Difficult choices
Google's formal entry into China in 2006 with Google.cn forced the company to strike a difficult balance between its stated goal of making the world's information widely available and the requirement that all Internet companies doing business in China adhere to government regulations regarding censorship.

In some ways, Google has improved the flow of information in China. Upon entering the market, it made sure to include a disclaimer like the one above alongside search results for sensitive queries, something even Baidu does now. That decision allowed Chinese Internet searchers to know they weren't getting the full extent of what was available on the Internet for a given query.

In addition, a study published by the University of Toronto's Citizen Lab in June 2008 found that Google is actually the least censored search engine in China. Google is the second-most widely used search engine in China, behind Baidu.

In the past, company executives have justified Google's censored presence in China with a glass-three-quarters-full analogy: it's better to offer Chinese Internet users access to a wealth of information they might be otherwise unable to find at the expense of "pulling a few books out of the library," so to speak. They are also, of course, unwilling to miss out on perhaps the greatest Internet land rush of the 21st century as China's massive population continues to come online.

However, determining which books to leave and which books to pull is not an easy task. Google representatives over the past week pointed repeatedly to an article in "The New York Times" from 2006 that described Google's methodology for making those tough choices.

From the article:

Brin's team had one more challenge to confront: how to determine which sites to block? The Chinese government wouldn't give them a list. So Google's engineers hit on a high-tech solution. They set up a computer inside China and programmed it to try to access Web sites outside the country, one after another. If a site was blocked by the firewall, it meant the government regarded it as illicit -- so it became part of Google's blacklist.

That system is no longer in place, Google representatives confirmed. Despite repeated inquiries, no information was made available about the new system: whether it involves taking direct cues from the government, self-selection by Google engineers, or something else.

In a way, Google's reluctance to talk about censorship and China is understandable. The Chinese government's regulations seem to be written in a deliberately vague way as to encourage Internet companies to censor more than the government would actually like to see pulled from the Internet.

In 2006, CNET's Declan McCullagh noted that Google.cn censored far more search results than seemed necessary, which was proven when Google restored access to Web sites like Budweiser.com following the article, with no apparent repercussions from the Chinese government.

The Times article from 2006 also noted the existence of weekly meetings between government officials and Internet companies known as the "wind-blowing" meetings; as in, you don't need a weatherman to know which way the wind blows in China, you need a bureaucrat. During those meetings, government officials would discuss upcoming events and hint at the ones they'd prefer to go unnoticed, according to the article.

Relevance lost
Ahead of the 20th anniversary of the massacre in Tiananmen Square, it seems several such meetings took place. Web sites across China were forced to shut down for a brief period of time in the days surrounding June 4, which many of them sarcastically dubbed "Chinese Internet Maintenance Day."

Unlike Twitter, Google's YouTube, and Wordpress, Google.cn was not shut down during the days surrounding the anniversary. But it was certainly far more stingy with search results than it was before the first week of June, or at present.

Whatever filter Google is using is both flexible and imprecise. Searches for obvious terms like "Tiananmen Square" and "Tank Man" returned no results between approximately June 3 and June 10, but as of last Thursday once again returned generic results unrelated to the events of June 4, 1989.

However, during "Chinese Internet Maintenance Week," searches on Google.cn for "June 4 incident" (the Chinese term for the events of June 4, 1989), "Goddess of Democracy" and "Tiananmen Square massacre," all returned results that one might think would be frowned upon by the Chinese government, including images of the Goddess of Democracy--a Statue of Liberty-like figure constructed by student protesters--staring defiantly at a portrait of Chairman Mao above the Tiananmen Gate.

Google's new filtering method allows Google.cn searches in English to produce results the government might not like. The same search in Chinese does not lead to Wikipedia.

(Credit: Screenshot by Tom Krazit/CNET)

And during that week, a search for "June 4 incident" on Google.cn actually returned (and still does return) links pointing to Wikipedia's article on the subject as well as a YouTube video with bloody images of the government's crackdown on student protesters in the top two positions. A search for that term in Chinese returns what appears to be censored results with the "According to local laws and regulations and policies, some search results are not displayed" disclaimer.

Perhaps that's why the Chinese government has announced plans to require all PCs sold in the country to have filtering software preinstalled that would block Web sites and even monitor keystrokes in word-processing applications. Whatever new filtering method Google has chosen, it may not be enough to satisfy the government's desire to keep certain topics out of the public eye.

Google has justified its presence in China as part of its lofty mission; this is a company that really does think it's engaged in business to better the world. But doing business in China while maintaining the moral high ground could well be more difficult than digitizing all the world's information.

It slows down your browsing. It makes some Web sites inaccessible for no discernible reason. It doesn't even offer you any xiao long bao or pu'er tea for your troubles. But if you want to know what life behind the Great Firewall of China is like, then the Firefox plug-in China Channel is the cheapest and fastest way to experience using the Internet in China without actually being there.

Tibet.com as it renders under the China Channel...

(Credit: CNET Networks)

After installation, getting to experience Web surfing the way the Chinese do isn't hard at all. Users have three ways to activate China Channel: via the China Channel toolbar, a navigation bar button that you must drag and drop onto the bar to get access to, and a status bar button. The buttons function by opening a menu, from which you choose to switch from None to the China Channel. Much like the IE Tab extension, the page will then render as if your IP address is inside China.

The toolbar is interesting for a slightly different workflow that results in a Web page that informs you of your IP address and its country of origin. Choose the China Channel from the drop down, and then hit the big red Go button. With China Channel activated, the page will declare that the plug-in has been activated. Switch back to None and refresh the page, and it changes to reflect your proxy server-free surfing experience.

...and unblocked by the Great Firewall.

(Credit: CNET Networks)

The experience drives home the point of just how severe Internet censorship is in China, going beyond government hot-topic issues like Tibet and Tiannamen Square to that hotbed of revolutionaries known as Wikipedia. Even my own innocuous blog was blocked when I was there, although two years later it seems to be free. Or at least it was when I tested out China Channel: while sensitive material seems to be permanently blocked, the 30,000 employees of the Great Firewall appear to apply their censorship in a more arbitrary manner for less topical Web sites.

This is a great experiential plug-in that's worth grabbing just to see how citizens in countries with Internet censorship have to struggle with hamstrung browsing.

When blogger Robert Scoble began to post updates on Twitter about the China earthquake, the information spread quickly to other users. Statistically, Scoble is one of the most 'listened to' users of the service.

(Credit: Twitter)

At around 11:50 p.m. Pacific time Sunday night, uber-blogger Robert Scoble posted a short note to Twitter: "@dtan just reported an earthquake in Beijing. Wonder how large it is? Off to check out USGS site."

Of course, as the world knows by now, Scoble was referring to the devastating quake that is already believed to have killed at least 12,000 people.

Some may be skeptical about Scoble's subsequent claims that news of the disaster was flying around Twitter before the U.S. Geological Survey posted anything on it, but one thing seems clear. Because it was Scoble who picked up on the quake and soon began writing dozens of Twitter posts about it, news of the catastrophe and direct reports of what had occurred in China spread a lot more quickly than it might have otherwise.

Scoble, after all, is one of the most followed users of Twitter. His activity on the micro-blogging service is currently monitored by 23,264 people. That, according to the site Twitterholic--which tracks the 100 most followed users--makes him the fifth most followed user of the service.

But what exactly does that mean to Scoble and the thousands who follow his posts? Does that mean he's one of the people whose participation provides the most value? Some observers are looking for such answers in numbers that measure users' behavior: how many people they follow, how many follow them, and the total number of posts they've made.

Twitter allows any user to see the number of people other users are following, how many people are following them, and how many posts they've made.

(Credit: Twitter)

"The many thousands of people who use Twitter do so in wildly different ways," Louis Gray, author of LouisGray.com, wrote in a widely discussed blog post. "I feel there are different categories of Twitter users--from those who have a listening audience, measured by a high 'followers'-to-'updates' ratio; those who are engaging, seen with near equal 'followers' and 'updates;' and those who are more noisy, with a lot more 'updates' than actual 'followers.'"

As part of his post, Gray introduced what he called a "noise ratio," which looked solely at the ratio of someone's posts--known on Twitter as "updates."

For those who make less than one Twitter post per follower, Gray assigned the term "listener." Those in the "middle ground" had posted up to twice as many updates as they have followers. After that, Gray called users "conversationalists" and "megaphones."

"If you look at some of the most visible and vocal Twitter users, like Scoble and (Mahalo founder Jason) Calacanis, if you look at their total number of followers, they have tens of thousands of followers," Gray told me by phone Monday. "Those people must be following them for a reason."

Indeed, many Twitter users employ the updates/followers ratio when evaluating other Twitter users, something they have to do each time they get a new follower and must decide whether to follow that new person in return.

To be sure, making such a determination can be tricky. You might want to follow everyone who follows you, but that can be a time-intensive proposition, since you will subsequently have to wade through every single such person's updates. As anyone who follows more than a few dozen people knows, that can mean a flood of information.

Tracking the Twitterers
But to some people, measuring the value another Twitter user offers them comes from more than just looking at their noise ratio.

"What I (look) at," said Chris Heuer, the founder of Social Media Club, "is the idea that you could see very easily from the number of followers to the number of following, what someone's intention was."

Heuer calls his statistic the "Twitter intention barometer."

"The idea," Heuer said, is looking at "the one number relative to the other. If someone has a very high followers-to-following ratio, that just shows a more intentional use of the service where there might be a lot of people interested in you."

I pointed out to Heuer that well-known writer Seth Godin is followed by thousands of people but follows no one.

"That shows a clear intention to use (Twitter) as a broadcast medium," Heuer said, and not to take part in the larger Twitter water cooler conversation.

One site, TweetStats, allows anyone to get a glimpse of the Twitter activity of any other user. By entering any Twitter account ID, it is possible to see that person's Twitter usage, by month, as well as the times of day they most often post and the people they interact with most frequently.

TweetStats lets anyone see graphs of statistics about Twitter behaviors, including, as this chart shows, what time of day someone is most likely to post updates.

(Credit: TweetStats)

But while TweetStat's creator, Damon Cortesi, said he would like to come up with a way to definitively nail down Twitter behaviors, he's finding that very difficult.

"Everybody uses Twitter for different purposes," said Cortesi, "so the given value of one Twitter user is different depending on your point of view."

Still, Cortesi thinks there are ways to approach the question.

"What I think would be interesting to see is a statistic showing the frequency of updates combined with certain keywords for topics I might be interested in," he said, adding that he's also seeking a system to determine if someone is "chatty, (or) are they picky about who they follow."

Yet through all this, there is a great deal of useful dialogue going on on Twitter. To understand how to access it, it helps to recognize that the value of Twitter as a service comes from its mesh quality. Used properly, Twitter is a blend of conversations between individual users and the people they follow and those who follow them, and all the other people who expand out in an ever-growing ring from any individual.

"I think that's one of the great viral aspects of the system," Heuer said. "By having it open and architected this way, it allows us to find people who might have something interesting to say."

A 'real meritocracy'
More to the point, Heuer said, "It is a real meritocracy at the end of the day," referring to the fact that those who add the most value to the system are the ones who get the most followers.

Yet, some argue that it's not just about how many people follow you, no matter how much our egos want it to be.

"One thing Scoble says," Gray told me, "is that power comes in whom you follow, not who follows you...The more data you can take in, the stronger and smarter you are."

But one thing that's important to realize is that people likely are going to want to measure things if they can be quantified. That's probably why Twitter allows people to see everyone's statistics.

"I think that Twitter understood that if they didn't put (the stats) up there," Gray said, "people would make scripts to do it."

Heuer takes that thought even further: he thinks the peoples' attention to stats has a secondary effect.

"By looking too closely at the stats, it ends up modifying our behavior," Heuer said. "If we care more about the numbers than the engagement, it can impact our behavior."

On June 10, Geek Gestalt hits the highways for Road Trip 2008. I'll start in Orlando, Fla., and visit many of the South's most interesting destinations. Stay tuned, and be sure to keep up, both now and during the trip, with what I'm doing on Twitter.

U.S. presidential candidate Barack Obama appears in cartoon form on the logo of Chinese search engine Baidu.

(Credit: Baidu)

Chinese-language search engine Baidu has an unusual new mascot atop its home page: U.S. Democratic presidential candidate Barack Obama.

A cartoon version of Obama is depicted next to a donkey, the Democratic party emblem. He's holding a net as though casting it, and attached to the end of the net is a computer mouse--get it? It's the Internet.

This is part of a "person of the month" feature that Baidu has instituted since November, the blog Shanghaiist explains. Each month, Baidu selects a real-life or fictional personality who has ranked high in its search queries. As Shanghaiist explains, it's "a bit like Google Trends meets Time Person of the Year on a monthly basis." Barack Obama is the sixth installment in the series.

The series is hosted on the domain renwu.baidu.com; "renwu" means "historically important person."

While the biography of Obama on Baidu is largely celebratory, this is not a formal endorsement of the candidate. It is, however, an endorsement of his Web-savviness. Clicking on the Obama-adorned logo on Baidu redirects to a Chinese-language biography of the candidate and links to various media; the central talking point is Obama's status as a young politician who has successfully leveraged digital media and the Web to rise to fame. Of particular note, according to his Baidu page, is his speech about race in Philadelphia that soared to the YouTube stratosphere after appearing on television earlier.

But of more local relevance, the Baidu site about Obama also highlights the high volume of Chinese search queries for both Obama and his Democratic rival, Hillary Clinton. Charts and graphs detail politics-related searches both Chinese and international. There are also information resources pertaining to what the U.S. presidential election means to China, and what Chinese citizens think about it.

"State and world affairs have become the most popular topics of concern for Internet users," a translation of part of Baidu's page about Obama reads. It doesn't seem to mesh particularly well with the Chinese government's rigid stance on the spread of information, particularly political rhetoric, on the Web.

Nor was it clear whether the Obama campaign would react positively, considering the tense relationship between the U.S. and China. Calls to the campaign's press office for comment were not immediately returned.

That's right, Wikipedia now has 10 million articles. But participation in this global brain-share is restricted in China.

Wikipedia being blocked is news to no one in China, but there's a bit of a catch-22 even for those who use proxies to get around the restrictions: many proxy URLs and anonymizers are banned from editing Wikipedia to reduce vandalism.

When I want to see an article on Wikipedia, I pop it into the Anonymouse Web site, and the content comes right up. But if I see a mistake in an article, I'm unable to make my contribution.

Vandalism on Wikipedia is a serious issue. People turn entire pages into insults directed at their subject. Others insert more insidious misinformation that's hard to detect. The community is generally very good at catching these things, but banning open proxies was seen as a good way to reduce the number of people doing these things with impunity. If you don't want your own IP to get banned for vandalism, maybe you'd use a service that hid your identity.

Tor is perhaps the best known relatively robust anonymizing tool online. The Global Voices Online project promotes it in its guide to anonymous blogging. (It's in English, but not blocked in China.) But Tor nodes, too, are usually blocked for editing.

This means that people in China would have to display exceptional ingenuity to participate in the great compilation of information going on at Wikipedia. Some time ago, I wrote a review of now-Harvard Law School Professor Cass Sunstein's book Infotopia. Sunstein focuses his book on the great potential, and potentially great downfalls, of online information gathering by massive communities.

To his reservations, I add one. By no means am I the first to point this out, but when Wikipedia excludes most Internet users from the most populous country on Earth, it's got a long way to go before its relative robustness in English is matched in Chinese. Of course, the billions of individuals not online around the world are also missing their say.

Here's to 10 million nodes in this emerging body of knowledge, but idealists should be careful to note the limits of the project. I just hope the franchise extends more and more. If nothing else, I have a lot to learn from people who aren't yet participating.

This post was updated at 12:12 PM PT to provide comment from Facebook.

Facebook is denying a report that it is chasing after Chinese social network Zhanzuo.com.

According to a Monday report in The Times, Facebook offered $85 million for Zhanzuo, which has about 7 million active users. (Facebook has slightly more than 50 million, about 100,000 of which are in China, according to The Times.)

While the newspaper cited a Facebook spokeswoman in reporting that Zhanzuo CEO Jack Zhang is "acquainted" with Mark Zuckerberg and that "there could be more information by the end of the month," a Facebook representative on Monday afternoon denied to CNET News.com that any such offer exists.

"No offer has been made, and no acquisition of any company in China is being considered by Facebook," the representative wrote in an e-mail. "We do not know who the spokeperson is that they are referring to in The Times story and were never contacted by the paper to confirm the accuracy of this story."

So far, Facebook's only major acquisition has been Parakey, a start-up founded by the creators of the Firefox browser. But rumors have circulated for weeks that the Mark Zuckerberg-founded company has been looking to expand into languages other than English, and international acquisitions are one way that Facebook could tackle that strategy.

An acquisition, additionally, would give Facebook a foothold in a market that has been politically difficult for many U.S. companies to enter. Yahoo recently came under government scrutiny over its role in providing information that landed two Chinese journalists in jail.

A Chinese Facebook would not be without competition. Facebook's chief rival, MySpace.com, launched a Chinese-language version of its site in 2006. (MySpace is owned by News Corp., which also owns The Times.) And social-networking site Friendster, which has consistently lost ground to Facebook and MySpace in the United States, has made surprising inroads in Asia; earlier this fall, Friendster launched a Chinese version of its site.

UPDATE: 11:30 a.m. PT Adds background on proposed Smith law prohibiting U.S. companies from cooperating with governments to censor the Internet.

Yahoo listened.

A week after being excoriated by lawmakers over supplying information to the Chinese government that landed two journalists in jail, Yahoo has settled the lawsuit filed by the journalists.

Shi Tao and Wang Xiaoning, both serving 10-year prison sentences, sued Yahoo in April alleging that Yahoo Hong Kong willingly provided their e-mails, IP addresses, and physical addresses to the Chinese government. The men were arrested for allegedly leaking state secrets, code language that should have tipped Yahoo off to the fact that the government was attempting to repress free speech for political reasons, said Morton Sklar, executive director of the World Organization for Human Rights USA, which filed the lawsuit on behalf of the men.

Yahoo had denied responsibility, saying it was merely complying with Chinese law, and had asked the federal court in San Francisco to dismiss the lawsuit.

The jailed men and their families are happy with the settlement, Sklar said. A lawsuit could have taken as long as five years to conclude, he added. Terms of the settlement are sealed by the court.

"The settlement provides more immediate help for the detainees and their families, but also provides a precedent making clear that U.S. companies have to do much more than just follow the orders of their host governments; that they have to look to U.S. laws and U.S. human rights standards when they make their decisions abroad," Sklar said.

In addition to providing financial support to the families, Yahoo will provide a humanitarian relief fund to support other political dissidents and their families, the company said in a statement.

"After meeting with the families, it was clear to me what we had to do to make this right for them, for Yahoo, and for the future," Chief Executive Jerry Yang said in the statement. "Yahoo was founded on the idea that the free exchange of information can fundamentally change how people lead their lives, conduct their business, and interact with their governments. We are committed to making sure our actions match our values around the world."

In a hearing before the U.S. House of Representatives Foreign Affairs Committee last Tuesday, Rep. Chris Smith (R-N.J.), urged Yang and Yahoo General Counsel Michael Callahan to settle the lawsuit.

"Settle it, and I would say settle it generously in their favor," Smith said. "That would be one way you could convey to the committee, to shareholders, and especially (to) the victims that you recognize there are true victims because of this complicity."

Smith sponsored the Global Online Freedom Act of 2007, which would prohibit U.S. Internet companies from cooperating with repressive regimes on censoring the Internet and using Internet user account data to track down and punish pro-democracy activists.

Sklar speculated that Yahoo decided to settle the case because of the hearing, at which the Yahoo executives were called moral "pygmies" by a lawmaker, and because Yahoo would have been forced to disclose in court information about its involvement in other Chinese investigations.

"It's clear they could not tolerate any longer being on the wrong side of this case," he said. "It was causing too much embarrassment and too much public exposure."

There are more than 200 documented cases of Internet users being jailed in China as a result of disclosures of their user ID information, many of them Yahoo users, according to Sklar.

Sklar hopes those cases will be addressed by Congress, which has asked Yahoo to provide information on other Chinese investigations in which the company has cooperated.

Congress has held several hearings on the matter of tech companies cooperating with China, including one in February 2006 at which Callahan said Yahoo didn't know the nature of the investigation when it gave information to Chinese authorities.

Callahan said last week that he learned approximately six months later what the details of the investigation were and apologized for not providing that updated information to Congress.

The settlement was first reported by The Associated Press.

Ticked off that the United States gave the Dalai Lama the prestigious Congressional Gold Medal this week, China may be taking out its aggression by "hijacking" American search engines.

There's speculation that the Dalai Lama's recent award from President Bush (their meeting, pictured above) has prompted Net users in China to be rerouted from U.S. search sites to Baidu.

(Credit: White House)

Over at Search Engine Land, Danny Sullivan reports that numerous users trying to access Google, Yahoo, or Microsoft search engines from within China or using Chinese Internet service providers are being redirected to Chinese-owned search engine Baidu.

Sullivan says it's not exactly clear how that process is working, but he cites a news report from 2002 that indicates this sort of thing has happened in China before. At the time, a Baidu official denied having any part in the rerouting.

So is the Chinese government to blame? After all, its extensive attempts at censoring what its citizens view on the Internet have been well-documented.

It's worth noting, however, that the reported redirects may not have any direct link to the Dalai Lama events. The Associated Press reported earlier this week that Beijing has been ramping up its filtering of political sites in an attempt to stifle political dissent leading up to the Communist Party Congress, a meeting in which leaders are selected to serve under the president for the next five years.

The chairman of the U.S. House Foreign Affairs Committee is summoning Yahoo Chief Executive Jerry Yang to Washington to talk about "how the Internet company gave false information to Congress about its role in a human rights case in China that sent a journalist to jail for a decade," according to a release from the committee chairman's office.

Chairman Tom Lantos has asked Yang and Yahoo General Counsel Michael Callahan to appear at a hearing on November 6.

"Our committee has established that Yahoo provided false information to Congress in early 2006," Lantos said in the statement.

A Yahoo spokeswoman released a statement saying that Callahan's testimony was accurate. The company has said that the testimony was accurate at the time it was given in February 2006 and that executives only learned what the investigations were related to after that time.

"The House Foreign Affairs Committee's decision to single out Yahoo and accuse the company of making misstatements is grossly unfair and mischaracterizes the nature and intent of our past testimony," the statement said.

"As we have made clear to Chairman Lantos and the Committee on Foreign Affairs, Yahoo has treated these issues with the gravity and attention they demand," the statement continued. "We are engaged in a multi-stakeholder process with other companies and the human rights community to develop a global code of conduct for operating in countries around the world, including China. We are also actively engaged with the Department of State to assist and encourage the government's efforts to deal with these issues on a diplomatic level."

Yahoo has been sued by several Chinese political dissidents who complained that Yahoo provided information to the Chinese government that led to their imprisonment for allegedly distributing state secrets over the Internet. One man, Shi Tao, was arrested in 2004 by Chinese officials after Yahoo cooperated with a request to provide information about the Yahoo Mail customer.

Callahan testified last year that Yahoo did not know the nature of the Chinese investigation when it provided information about Shi.

after the Dui Hua Foundation, a human rights group that focuses on China, released a document that it said shows that the Beijing State Security Bureau had told Yahoo in writing that Shi was suspected of "illegal provision of state secrets to foreign entities."