Research outfit Pear Analytics randomly sampled 2,000 tweets from the public timeline over a two-week period and categorized them as News, Spam, Self Promotion, Conversational, Pass-Along Value and Pointless Babble (click on image to enlarge).

Unsurprisingly, the largest portion, 40.55 percent, fell under Pointless Babble, which Pear Analytics loosely defines as those “I am eating a sandwich now” tweets. Conversational ranked second with 37.55 percent, and Pass-Along Value a distant third with 8.7 percent of the tweets captured. Oddly, Self Promotion and Spam ranked fairly low, with 5.85 percent and 3.75 percent of total tweets captured, respectively. Still, both were higher than News, which ranked at the very bottom, with 3.6 percent.

“We thought that both Spam and Self Promotion percentages would be much higher,” Pear noted in its study. “Also, we thought the News category would have more weight than dead last, since this seems to be contrary to Twitter’s new position of being the premier source of news and events.”

An interesting variation on the “membership drive” and one that’s gotten Tagged in hot water with New York Attorney General Andrew M. Cuomo, who intends to sue the company “for deceptive e-mail marketing practices and invasion of privacy.”

“This company stole the address books and identities of millions of people,” Cuomo said in a statement. “Consumers had their privacy invaded and were forced into the embarrassing position of having to apologize to all their e-mail contacts for Tagged’s unethical–and illegal–behavior. This very virulent form of spam is the online equivalent of breaking into a home, stealing address books and sending phony mail to all of an individual’s personal contacts. We would never accept this behavior in the real world, and we cannot accept it online.”

Tagged, for its part, claims this is all just a big misunderstanding. In a statement of its own, the company denied abusing its users’ personal address books, saying, essentially, it had their consent to do so.

“When our company tested a new registration process, we discovered that our ‘invite your friends’ language was confusing,” said Tagged CEO Greg Tseng. “…In no instance did Tagged access a person’s personal address book without their consent and no emails were sent without the person giving us permission. We realize that some were confused and accidentally agreed to invite their friends.”

Oh, and next time you register for a social network, be sure to read its Terms of Service–especially the portions that are presented in ALL CAPS. They might be important. From Tagged’s Terms of Service:

“E) Notice Regarding Commercial Email

MEMBERS CONSENT TO RECEIVE COMMERCIAL E-MAIL MESSAGES FROM TAGGED, AND ACKNOWLEDGE AND AGREE THAT THEIR EMAIL ADDRESSES AND OTHER PERSONAL INFORMATION MAY BE USED BY TAGGED FOR THE PURPOSE OF INITIATING COMMERCIAL E-MAIL MESSAGES.”

Google and the Evolution of Search

1. Human Evaluators — Google Engineering director Scott Huffman
2. Cheating the System — Google software engineer Matt Cutts
3. What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal

Part II: Matt Cutts

John Paczkowski: How do you maintain quality in search?

Matt Cutts: Well, broadly, we improve our algorithms and hopefully, every so often, develop some punctuated equilibrium where we create totally new ways to improve our relevance. My contribution… is ensuring that people who try to cheat the system don’t show up higher than they deserve to in our results. We want sites ranking high based on merit, not based on shortcuts.

JP: OK, so how do you do that?

MC: Essentially we look at a wide variety of input. We look at user complaints, for example. We also have a variety of internal metrics we use to track current trends. They help show us what people are using to spam right now. What’s getting past our defenses. And when we detect those things, we write some new algorithms or develop some tool that helps us detect and, hopefully, counteract them. So a large part of what we do is simply spotting trends in spam.

JP: Is there a human evaluation element here as well?

MC: Each team is responsible for general search-quality evaluations, but it’s not like they’re changing rankings or anything like that. That said, there are some policy violations that are pretty egregious. So, for example, if you type in your name and instead of getting All Things Digital, you got a porn site, you would get pretty angry about that. And you might complain to Google. And it would be frustrating if our reply was, “Yeah, well, we think we might have an algorithm that might fix that problem in five or six months, so we’re just going to leave that porn site as the top result for All Things D until we get an algorithm up to help you out.” Obviously, that’s a deeply dissatisfying answer.

So in spam, we are sometimes willing to take manual action on those sorts of policy violations. But Google’s philosophy is that wherever you can use machines and algorithms, it is much better, more robust, more scalable. And so, to the extent that we can, we always want to rely on the computers as our first line of defense.

JP: But you’re willing to remove spam manually until you can find an algorithm to counteract it. Do you think that will always be the case? Will we some day reach a point where human intervention of the sort you just described won’t be necessary or are we headed toward increasing human intervention?

MC: That’s a really fascinating question, but I don’t know the answer. What’s interesting to think about is that page rank, the raw page rank algorithm, actually improves as it ranks more pages. So the more pages you add to it, the easier it is to determine how reputable a particular page is without human intervention.

But as the Web grows in size we also encounter new and different policy violations–hidden text, cloaking. Those are the sorts of things that humans are very good at spotting. You can certainly identify some of them with a computer algorithm, but not all. And so our intent is always to try to make sure that we handle things efficiently with machines and algorithms. But I don’t know that we will ever get there completely.

Google and the Evolution of Search

1. Human Evaluators — Google Engineering director Scott Huffman
2. Cheating the System — Google software engineer Matt Cutts
3. What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal

Denied Internet access by Global Crossing and Hurricane Electric, bot-hosting network McColo is clearly having trouble spewing out spam and malware. There has been a 41 percent drop in spam volume since the Washington Post story broke.

Sadly, it’s certain to rise again, once McColo finds some new upstream providers.
(Thanks to reader Dave Barnes for the tip.)

There is damning evidence that this activity has been going on there for way too long, and plenty of people in the security community have gone out of their way to raise awareness about this network, but nobody seems to care.”

– Paul Ferguson, a threat researcher with computer security firm Trend Micro

According to security experts, Web-hosting outfit McColo is responsible for enabling the broadcast of more than 75 percent of all spam globally. Its client list is a rogues gallery of bad-guy syndicates involved in everything from botnets to counterfeit pharmaceuticals and kiddie porn. So how is it that MoColo’s ISPs, Hurricane Electric and Global Crossing, were unaware of that until notified by a Washington Post reporter?

I’m not sure there’s a good answer to that question, though it would certainly be interesting to hear one. Almost as interesting as hearing the two ISPs explain away their network traffic from known criminal botnets Mega-D, Srizbi, Pushdo, Rustock and Warezov, all of which have their master servers hosted at McColo.

“We shut them down,” Benny Ng, director of marketing for Hurricane Electric, told the Post. “We looked into it a bit, saw the size and scope of the problem you were reporting and said ‘Holy cow!’ Within the hour we had terminated all of our connections to them.”

“Holy cow?” More like, “Holy cow, someone finally noticed we’re the preferred ISP of a massive criminal syndicate! What do we do?!?”

“ISPs can’t take the ‘I see nothing, I hear nothing’ approach to this content,” said Mark Rasch, a former cyber crime prosecutor for the Justice Department. “It’s a little bit like a landlord who owns a building and sees people coming in and out of the apartment complex constantly at all hours and not suspecting their may be drug activity going on. There are certain things that raise red flags, such as the nature, volume, source and destination of the Internet traffic, that can and should raise red flags. And to have so many third parties looking at the volume and content from this Internet provider saying ‘This is outrageous,’ clearly the people doing the hosting should know that as well.”

Consider this: Using the Storm botnet and 75,869 of its zombie members, researchers at UC Berkeley and UC San Diego broadcast 350 million pieces of spam peddling male enhancement supplements. All but 28 were ignored. “After 26 days, and almost 350 million e-mail messages, only 28 sales resulted,” the researchers explained. “Taken together, these conversions would have resulted in revenues of $2,731.88–a bit over $100 a day for the measurement period.”

That’s a conversion rate of well under 0.00001 percent. Laughable, right?

Not really. The supplements were priced at $100. And botnet overhead, as you might imagine, is quite low. Which means it’s entirely possible to turn a nice profit by managing just one sale per 12.5 million emails sent. Said the researchers, “Under the assumption that our measurements are representative over time, we can extrapolate that…Storm-generated pharmaceutical spam would produce roughly $3.5 million dollars of revenue a year.”

Earlier today the blog offered nothing more than a placeholder page, but a few moments ago it went live with Icahn’s promised anecdotes on “the desultory state of corporate governance in America.”

And, specifically, Yahoo.

Which seems to figure in many of Icahn’s first few posts–though the company is never mentioned by name.

• Absurdity of Corporate Board Elections
• About CEOs–Anti Darwinian Metaphor–Survival of the Unfittest
• Absurdity of the Poison Pill
• Corporate Democracy is a Myth

Unfit CEOs. Absurd poison pills. Absurd board elections. It’s pretty clear to whom Icahn is referring, isn’t it? Presumably he’s had his site admins ban Yahoo’s IP address from the blog. I’m told comment spam can be a real nightmare.

The judgment is believed to be the largest anti-spam award to date. Not that it really matters, because MySpace (NWS) is unlikely to collect it. Wallace–who was by some estimates responsible for 80% of the spam on the Net back in his heyday–has rarely paid the judgments against him. Moreover, he has a bad habit of disappearing at the first sign of legal trouble.

And that appears to be exactly what he’s done here. Because the $230 million award given MySpace in this case is a default judgment meted out after Wallace failed to appear in court. “It is … a defendant’s responsibility to respond to discovery, obey court orders and avoid dilatory tactics,” the court wrote in its order. “Taking all of the above factors into account, a default is appropriate. The court finds that Wallace’s noncompliance is due to willfulness, fault or bad faith. … Wallace has had every opportunity to avoid the sanction of default. (He) has never provided any explanation for his behavior to the court.”

Gordon has testified that in 2006 he received no income that was not the result of a settlement of a dispute. He also testified that he has been filing lawsuits over commercial email since at least 2003. … He also admitted that his “clients”–apparently people to whom he provides email accounts–supply him with emails they deem are spam for him to use in his disputes, and that they get an unspecified percentage of the settlements.”

–Man, Oh Man, What a Racket, DIRECT, May 22, 2007

Spam litigation is not the stuff of which cottage industries are made. Just ask James Gordon who’s been ordered to pay attorneys’ fees and costs in the amount of $111,440 to Virtumundo, the email marketer he sued under the CAN-SPAM Act.

In his complaint, Gordon–whose only source of income for the past year was “settlements and disputes,” according to court documents–demanded damages of more than $20 million, claiming Virtumundo had buried him in some 13,800 unsolicited emails. Problem was, Gordon doesn’t operate a traditional ISP, and thus lacked legal standing to sue under CAN-SPAM. And so in May a U.S. district court in Washington state dismissed his claim, ruling Virtumundo was entitled to recover attorneys’ fees.

All $111,440 worth of them, according to the court decision released last week that adds insult to Gordon’s financial injury. “The court finds that the [plaintiff's] instant lawsuit is an excellent example of the ill-motivated, unreasonable and frivolous type of lawsuit that justifies an award of attorneys’ fees to defendants,” according to the decision. “The Court finds that the goal of deterrence is particularly relevant here. Plaintiffs should be deterred from further litigating their numerous other CAN-SPAM lawsuits now that they are aware of their lack of CAN-SPAM standing.”

Ironic, isn’t it? Especially given Virtumundo’s Spamhaus status.

“I believe this ruling represents the first time that a CAN-SPAM plaintiff has been ordered to pay attorneys’ fees and costs to a defendant,” Santa Clara University School of Law professor Eric Goldman wrote in a post to his blog. “As a result, it’s a leading example that courts can and do grow tired of bogus antimarketing lawsuits, and perhaps it will serve as an expensive warning to CAN-SPAM plaintiffs to ensure the merits of their lawsuit. As the court describes, Gordon runs a ’spam business’–basically, a for-profit plaintiff litigation shop to go after spammers (the court also calls it a ‘litigation factory’). The court correctly notes that Congress really wasn’t trying to enable lots of private lawsuits from CAN-SPAM, so the risk of chilled plaintiffs is appropriate in this context.”