Google and the Evolution of Search II: Cheating the System
by John Paczkowski
Posted on June 4, 2009 at 4:30 AM PT
This is the second of three interviews with members of the Google (GOOG) team responsible for overseeing search algorithms at the company. The introduction and Part I, an interview with Scott Huffman, appeared yesterday. In today’s installment Google software engineer Matt Cutts talks about search quality and spam. In Part III tomorrow, Google Fellow Amit Singhal will wrap up the series.
Google and the Evolution of Search
- Human Evaluators — Google Engineering director Scott Huffman
- Cheating the System — Google software engineer Matt Cutts
- What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal
Part II: Matt Cutts
John Paczkowski: How do you maintain quality in search?
Matt Cutts: Well, broadly, we improve our algorithms and hopefully, every so often, develop some punctuated equilibrium where we create totally new ways to improve our relevance. My contribution… is ensuring that people who try to cheat the system don’t show up higher than they deserve to in our results. We want sites ranking high based on merit, not based on shortcuts.
JP: OK, so how do you do that?
MC: Essentially we look at a wide variety of input. We look at user complaints, for example. We also have a variety of internal metrics we use to track current trends. They help show us what people are using to spam right now. What’s getting past our defenses. And when we detect those things, we write some new algorithms or develop some tool that helps us detect and, hopefully, counteract them. So a large part of what we do is simply spotting trends in spam.
JP: Is there a human evaluation element here as well?
MC: Each team is responsible for general search-quality evaluations, but it’s not like they’re changing rankings or anything like that. That said, there are some policy violations that are pretty egregious. So, for example, if you type in your name and instead of getting All Things Digital, you got a porn site, you would get pretty angry about that. And you might complain to Google. And it would be frustrating if our reply was, “Yeah, well, we think we might have an algorithm that might fix that problem in five or six months, so we’re just going to leave that porn site as the top result for All Things D until we get an algorithm up to help you out.” Obviously, that’s a deeply dissatisfying answer.
So in spam, we are sometimes willing to take manual action on those sorts of policy violations. But Google’s philosophy is that wherever you can use machines and algorithms, it is much better, more robust, more scalable. And so, to the extent that we can, we always want to rely on the computers as our first line of defense.
JP: But you’re willing to remove spam manually until you can find an algorithm to counteract it. Do you think that will always be the case? Will we some day reach a point where human intervention of the sort you just described won’t be necessary or are we headed toward increasing human intervention?
MC: That’s a really fascinating question, but I don’t know the answer. What’s interesting to think about is that page rank, the raw page rank algorithm, actually improves as it ranks more pages. So the more pages you add to it, the easier it is to determine how reputable a particular page is without human intervention.
But as the Web grows in size we also encounter new and different policy violations–hidden text, cloaking. Those are the sorts of things that humans are very good at spotting. You can certainly identify some of them with a computer algorithm, but not all. And so our intent is always to try to make sure that we handle things efficiently with machines and algorithms. But I don’t know that we will ever get there completely.
Google and the Evolution of Search
- Human Evaluators — Google Engineering director Scott Huffman
- Cheating the System — Google software engineer Matt Cutts
- What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal
Comments
These interviews show that it’s the quality of the page, not the SEO on the page, that counts. A website can be SEOed to the gills, but Google’s ten thousand Quality Raters evaluate it and if they don’t like it, Google engineers write new filters to block it.
Google uses humans, not software, to evaluate sites. The software does the heavy work: the indexing of billions of pages. But “bad” pages creep into the top results: either the filter was poor, the page is spam, or the page uses SEO tricks. So humans look at the top results, evaluate these, and the filters are adjusted. As for bad pages, these are pushed down (Matt Cutts admits that in today’s interview.)
This means that much of what passes for SEO (keyword density, page rank, back links, etc. has a limited value: it can get a page INTO the index and it can bring a page up in ranking, but the Google Quality Raters will look at the page and evaluate not on the keyword density, meta-tags, etc., but on the quality, which means navigational, informational, or transactional criteria.
yrs,
Posted by andreas ramos at June 4th, 2009 at 6:50 amandreas
andreas.com
Who cares about pagerank, when we get no response on questions like http://www.google.com/support/.....&hl=en
Posted by Marcis Gasuns at June 5th, 2009 at 10:08 amSincerely,
M.G.
These interviews (in particular the discussion regarding human intervention) may serve as a roadmap for future antitrust litigation. The relative subjectivity of the manual intervention compared to the “objective” algorithms serves as a salient reminder of how Google wields its dominant market power in a way that organizes the Web in the manner Google chooses.
Posted by Elbert McQuiller at June 6th, 2009 at 11:02 amIf I were Google’s legal counsel, I would encourage them to stop discussing issues like this in the public domain. However, I would encourage the continued use of human feedback loops as discussed in the first interview. Google may need to leverage diverse segments or sub-markets as opposed to merely broad geographic “locales.” This would have the dual purpose of improving intended search results and mitigating the subjectivity of the kind of manual intervention described in the second interview.
http://www.MyBlackNetworks.com
Posted by Elbert McQuiller at June 6th, 2009 at 11:19 am