Badware URL Analysis

One of the projects I am affiliated with in an advisory capacity is the Berkman Center's StopBadware.org project. Over the weekend (2007-03-25) I scraped and analyzed the 18328 badware URLs from StopBadware.org's Badware Website Clearinghouse, a "a collaborative effort to build a comprehensive list of websites that host, link to, or otherwise distribute badware". The results are available here.

The source of all of the URLs (100%) was Google, one of the corporate sponsors of StopBadware.org. Although there are 18328 URLs there were only 6856 distinct IP addresses and 0.4% of the URL's were given a decision of "Badware" -- "Sites that StopBadware has tested itself and determined to contain or link to badware" --, with the balance being listed as "Undetermined".

• The top TLD's were .com: 10710, .org: 1550, .info: 1352, .net: 1300 followed by the ccTLD's cn: 1216, .ru: 352, .uk: 275, .ua: 226, .it: 129 and .pl: 118.


• The top countries (based on IP allocation) in which badware URLs are hosted are US: 10037, CN: 4336, ?? (unknown): 1357, DE: 433, RU: 361, GB: 349, UA: 210, IT: 186, CA: 154 and NL: 81.


• The top AS number are AS30380: 3435, AS4134: 1819, AS17233: 1537, ASNA (unknown): 1315 and AS21844: 734.


• The top network names are IPOWER - iPowerWeb, Inc.: 3435, CHINANET-BACKBONE No.31,Jin-rong Street: 1819, ATT-CERFNET-BLOCK - AT&T Enhanced Network Services: 1537 NA (unknown: 1315 and THEPLANET-AS - THE PLANET: 734.

An interesting note is that Google appears as the 13th (GOOGLE - Google Inc.: 169) network name with 169 badware URLs all of which appear to be *.blogspot blogs.

30,000 Internet Police in China Myth, Please Not Again!

UPDATED (28 Mar 07)

BBC:

Hundreds of millions of dollars have been spent on building what is known as the Great Firewall Of China - a network of state-licensed internet access providers, and around 30,000 internet police censors who filter sites between China and the rest of the world.

Nearly two years after the "30,000" myth was exposed it is still being repeated by reputable news media.

I searched LexisNexis (and Googled) and collected articles that discussed Internet police in China and those that specifically stated the magic 30,000 number. The earliest reference I can find (if you have an earlier one, please send it to me) is an Ethan Gutmann article in The Weekly Standard 02/15/2002

Although it was widely rumored in Beijing that up to 30,000 state security employees were monitoring the Internet in that city alone, the monitoring was also laughed at.

Note that it states it is a rumor and that it was in Beijing, not all of China. Following that, on the 27th of February 2002, Amnesty International releases a report which states:

30,000 state security personnel are reportedly monitoring websites, chat rooms and private e-mail messages.

Rumored has turned in to reportedly, but at least there is a qualifier. But by the 25th of August 2002 the LA Times dropped the modifier:

More than 30,000 state security employees are currently conducting surveillance of Web sites, chat rooms and private e-mail messages--including those sent from home computers.

On November 7, 2002 the Washinton Post decides to leave out the "rumored" and "laughed at" part:

But Beijing, with 30,000 "Internet police," has acted swiftly to clamp down on dissent through the ethers.

And so it begins. Rumor turned into fact. Some publications have and continue to include references to "rumored" or "estimated" along with the 30,000 figure. (An interesting sidenote is that publications continue to reference 30,000 (no change since 2002) despite a huge rise in China's number of Internet users). However, the New York times has upped the number to 50,000 and dropped all qualifiers:

Stern instructions like those are in keeping with a trend aimed at assigning greater responsibility to Internet providers to assist the government and its army of as many as 50,000 Internet police, who enforce limits on what can be seen and said. -- New York Times, March 4, 2005

I guess there were some massive firing because in 2006 the number was back down to 30,000. This time however, USA Today claims that China Internet Network Information Center is the source of the number (I cannot find it on their website, email me if you can). But the part that gets me is the "shadowy force" -- so shadowy that they have their own websites? Why yes, try www.cyberpolice.cn or any of the numerous local sites.

Even with an estimated 30,000 internet police, he said it was difficult to monitor bulletin boards. "The technology hasn't reached a level that will allow us to control them..." -- The Guardian February 14, 2006

China has roughly 30,000 Internet police who use Internet Detective and other tools to monitor Web users, according to the China Internet Network Information Center, an arm of China's Ministry of Information Industry. Officials at the Ministry of Public Security declined interview requests for details of this shadowy force. -- USA Today 4/3/2006

This is an analysis from EastSouthWestNorth:

Here is a myth: there are 30,000 Internet police in China who sit around all day looking for harmful information. 30,000 is a big number, since it could fill a soccer stadium. But with respect to 100 million Internet users, 30,000 is woefully inadequate to patrol all possible Internet content material (that is, one Internet police officer has to keep an eye on 3,333 users at the same time, or less than 10 seconds per user per day).

In any case, one has to ask just how well trained these 30,000 Internet police are. A fair bet is that when they come across a website with pictures of naked people, they will take action. But if they come across a copy of Anti's blog post (see Comment 200601#029) or the scholarly article History Textbooks in China, they would not have a clue what to think.

In the scheme of things, I don't think these 30,000 Internet police form the front line. Rather, it is the Internet unit administrators (such as BBS forum masters) who do the active work because they have domain knowledge. The Internet police are only there to catch the periodic leak so as to hold the Internet unit adminstrators accountable for negligence and/or sabotage. If you are a vigilant forum master that has everything under control (e.g. wiping every mention of Beijing News immediately), then you will rarely come into contact with the Internet police; if you are a progressive forum master, you will get phone calls every day and eventually you will be dismissed and your website may even be disappeared like the Yannan forum. That is why it was no surprise that nothing was found on this day by the reporter who played Internet police.

Postscript: Oh, by the way, they obviously don't read overseas English-language blogs ...

EastSouthWestNorth touches on a very interesting point. Media reports seem to merge together the self-censorship practices by forums, portals, blog hosting companies and so forth with the Internet police.

An old article in The Guardian I missed earlier does discuss this issue:

Better still is scaring users into censoring themselves. No one I spoke to could tell me where the figure of 30,000 internet policemen originated. But researchers pointed out that it was in Beijing's interests to persuade its citizens that Big Brother lurks in every cafe. Similarly, arrest a few people and you frighten many more into compliance.

There are Internet police in China, they have websites, lots of them. They engage in law enforcement duties. They also investigate websites. It is also, in my view, safe to assume that they investigate and arrest dissidents. In fact the Beijing cyberpolice accept reports (appears to be via SMS) from the public against persons who want to split the nation, or attack the party and the government, and people with "wrong doctrines opinion"/ falungong. (babelfish). Seems quite clear to me that the cyber police's mandate is to investigate reports of people who criticize the government or belong to falun gong -- it is right in their incident report form.

However, its the manufactured number and the near godlike capabilities assigned to China's Internet police and filtering/monitoring technology in most news reports that infuriates me. There is definitely a lot going on in this area in China, there is a lot left to investigate. However, reporting rumor as fact is not the way to go about this. It doesn't help people better understand what is really going on in China, but it does re-enforce a climate of self-censorship in China.

Please, stop repeating the "30,000" myth.

Unblocked?

Recently Google changed the IP address of its blogspot blogging service. This caused the site to become accessible in all countries that blocked the site by its IP address --including Pakistan and China. China and Pakistan did not unblock blogspot, rather it became available because of actions taken on Google's part. Well, the new blogspot IP is now blocked in China, but it is still accessible in Pakistan.

Iran has, however, unblocked Baztab, a conservative news website. Hoder wrote about this case about a month ago:

The Iranian cultural ministry has now ordered all major ISPs to block Baztab, a news website close to moderate conservatives and linked to an influential former commander of the elite revolutionary guards. It has also demanded the website to stop its activities.

But the well-connected editors of Baztab have hit back. They have refused to stop publishing new articles, have called the order illegal and illegitimate, and have also said they are going to bring the case to court.

They have argued that it is only the judiciary has the constitutional authority to decide weather a website has violated laws. They have also disputed the legality of a set of regulations passed in the governemnt cabinet last month to be executed by the ministry of culture.

Hoder argued that "by pushing for the judiciary to take up the responsibility of internet filtering, Iranian internet users can slow down the process of filtering, hold the authorities accountable, and force them to make the behind-the-scenes process transparent.". When I linked to this article a while back, a comment was posted which stated that "[i]n Iran it is the judiciary that has spearheaded the arrest and torture of bloggers."

This is an interesting case. One of the recommendations of the HRW report on China is:

Create formal, well-documented and legally transparent processes by which content censorship requests are made to companies, formal written procedures by which companies can challenge or respond to censorship requests, and formal, transparent legal procedures by which members of the Chinese public can safely and fairly challenge the legality of any act of censorship without fear of reprisal.

HRW reccomends that Iran "should further seek to pass new laws that affirmatively protect the right to freely access or disseminate information or opinions and clarify the narrow circumstances in which government interference would be warranted according to international standards.".

The unblocking of one website -- one run by well connected people -- is a small victory but it could be very significant. If the procedures for determining content to block become transparent, if there is an appeals process and some level of accountability I believe it becomes increasingly difficult for governments to justify censorship. I believe, as recommended by HRW, that this process needs to be accompanied by movements to affirmatively protect freedom of expression as well.

FT Censorship Series

FT has recently run a series of articles on internet censorship. Each touches on an interesting theme.

• Dissidents find ingenious ways to hide digital traces

TOR can be used for both anonymity and censorship circumvention, but while "anonymous" proxies can be used for censorship circumvention they not really anonymous. A "proxy" may sheild your identity from the website you are visiting but it does not hide you or anything you are doing from the owner of the proxy. And if the proxy is not encrypted -- most of the "open" proxies are not -- then anyone monitoring Internet traffic can also see everything you do through the proxy. TOR, on the other hand, encrypts your traffic and hides what you are doing from the TOR network itself, it is hardly comparable to "open" proxies. I have not looked closely at GPass, but it appears to be an encrypted Socks proxy, and if so, is not anonymous -- all traffic through it can be viewed by the owners of GPass. (And you don't have to use Swedish Google, Google just redirects you to the localized version, you can always click the google.com link and use google.com).

• Repressive governments widen stifling techniques

It is not only "repressive" governments that are increasing their level of filtering and employing new techniques (new techniques for the country, not for filtering in general), countries such as India and Thailand are filtering as well. There is a tendency to analyze all regulations and restriction in particular countries, such as China and Iran, out of context. For example, there is a tendency to think of China's Internte cafe's as places teeming with cyberdissdents and therefore when China closed many and instituted restrictions after a deadly fire in an unlicensed cafe many interpretted it as a crackdown on free expression. I think that the Iranian bandwidth limitation story may prove to go this way as well -- it's more likely to do with porn than with politics. But, hey, I could be wrong.

Human rights groups and NGO's worldwide have long protested that they are often the victims of state surveillance, computer breakins and denial of service attacks. ONI has documented an attack on Kyrgyz opposition newspaper websites during that countries elections in 2005 and there have been reports of such Denial of Service attacks during elections in Belarus as well. What is new is not the technique but the correlation between the target -- important opposition website -- and the time period -- during an election. Denial of Service disrupts access to a website for everyone -- as opposed to filtering which would only block it from the affected location. It also provides deniability on the part of the government. In the Kyrgyz case, the attacks appear to have been conducted by a "botnet for hire" leaving the conection to the government circumstantial. This is a trend we will probably see more of especially in countries that don't have a national filtering system (or officially filter very little content).

• Web censorship spreading globally

A good article about the forthcoming ONI study, however, some instances listed as "new censorship techniques" are not really new at all. They may be new to certain countries, but they are standard filtering techniques. And there is not yet evidence that Zimbabwe is censoring the Internet, let alone using the same techniques as China. I have heard reports about this, but even if they are true, it has not been implemented.

Pakistan: Blogspot accessible, but block has not been lifted

Athough Blogspot is currently accessible in Pakistan the block has not been lifted.

In the past the blogs at *.blogspot.com were on one IP -- blocked in Pakistan -- while the interface to update your blog at blogger.com was on a different IP -- unblocked in Pakistan. So you could update you blog, but you could not access it at the *.blogspot.com address (or any other blogs on blogspot). Google recently upgraded Blogspot -- it is no longer "beta". This resulted in several changes including the ability to login to all your Google services including Blogspot from one account. Quite simply, the IP address of *.blogspot.com has been changed. the new IP address is accessible, the old IP address is still blocked.

Global Voices reports that Dr Awab Alvi from the co-founder of the Pakistani “Don’t Block The Blog” campaign suggests that he "would not be surprised to see some blocking to come again." The authorities have also stated that they are "are still filtering the websites".

I have been wanting to build something monitor this for quite some time, especially after reading about the availability issues of wikipedia in China.

Microsoft has censored me

Well, Microsoft's Chinese version of live.com is now censoring me (www.nartv.org).

Pakistan Overblocking

Not content to block Yahoo's hosting service causing tens of thousands of websites to be blocked, it is now being reported that the Pakistan's ITI is now blocking some Akamai servers. Because they block by IP address, all other content hosted on the server will also be blocked. This has resulted in massive overblocking.

The PTCL starting blocking Internet Protocols at its key gateway—ITI Karachi—(technically termed Router) which handles 95 per cent or more of Pakistan’s Internet traffic passes. The Internet-users have been facing problems intermittently in accessing websites of CNN, BBC, Yahoo, Microsoft, download.com, Symantecs, etc.

In other words, the PTCL has blocked some sites of www.akamai.net at their Karachi gateway exchange. Akamai.net has hundreds of servers, hosted across the globe, to provide content to a myriad of websites.

I've found that when a handful of sites are blocked, (the number of officially blocked websites is fairly low the orders sent to ISP's here, here and here) many people don't really care. What they don't realize is that inevitably sites that were never intended to be blocked will be blocked and once in place it is far too tempting for governments not to extend the use of filtering regardless of the original justification for implementing it.

Overblocking tends to create a significant backlash, especially from non-activist Internet users. While people can often tolerate the blocking of some sites (often extremist, or offensive etc...) it does not significantly impact the experience of everyday users. But when people's regular browsing and blogging is interrupted they quickly become aware of censorship's impact and campaign against it. An excellent example has been the "Don't Block the Blog" campaign in Pakistan which was started after Pakistan blocked access to Blogspot. pkblogs.com offers an alternate means of accessing Blogspot, bypassing the ITI's filtering.

However, in response the authorities will often seek to implement filtering techniques that better target the specific sites they want to blocking while minimizing the overblocking. We've seen the case in India, for example, where public outcry about overblocking led the authorities to send out instructions on DNS tampering so that IP blocking could be avoided. Hopefully, even when Pakistan refines its filtering techniques and does not block all of Blogspot, Yahoo's hosting, and Akamai the movement against Internet censorship will be strong enough to continue the struggle against it.