The Guardian
topics
Close Box

News From Us:

Our latest report; our new video section; and jobs with paidContent.org and paidContent:UK


Inside Word
Inside Word: What The Netflix Prize Says About The Shortfalls Of Ad-Targeting Startups

The Inside Word is a weekly feature that looks at compelling industry debates and discussions unfolding on the blogs of employees at digital-media companies.

Blogger: Andrew Chen

Position: A self-described entrepreneur-out-of residence, Chen is a former entrepreneur-in-residence at early-stage VC firm Mohr Davidow Ventures and was a director of product marketing at behavioral ad targeting firm AudienceScience.

Blog name: Andrew Chen

Backstory: In October 2006, Netflix (NSDQ: NFLX) offered a $1 million prize to anybody who could build an algorithm that would improve on its system of predicting a user’s movie ratings by 10 percent. Last month, Netflix declared a winner, handing out the prize money to a team that had beat its algorithm by 10.5 percent.

Blog post: In a blog post, Chen writes that the story of the prize should put a damper on ad networks’ claims that they can distinguish themselves by offering better ad-targeting technology. “This means if you combine dozens of the best machine-learning people in the world, some of the cleanest datasets, you get a measly 10.5 percent increase,” he writes. “Compare this to starting a new ad network where you end up with noisy datasets, lots of crappy traffic, and a small team looking at the problem – that’s not an easy path to disruptive change.”

“In general, 10 percent is not a big enough number to counteract the other economic drivers in the ad market, which revolves around better deal terms, a larger selection of advertisers, better ad inventory, etc.”

Post-script: I asked Chen why—if there is, in fact, not much difference between the targeting technology of various ad networks—the ad networks nevertheless were making those claims. He responded: “First off, they are geeks and prefer to emphasize targeting. Second, they (and by extension, their investors) have Google (NSDQ: GOOG) envy, and they think data + algos can solve everything. Third, they look at the ad industry and see how backward it is from a technology standpoint, but don’t appreciate the tremendous amount of sales and marketing excellence in the industry. So as geeks, of course they think they can one-up the ad industry on technology, even though that’s not sufficient to win in the ad industry.

There are just a ton of variables that go into the performance of an actual campaign, of which targeting is merely one variable. And because most ad networks are black-boxes, you can’t really say, ‘we’ll run both of these campaigns side by side but use different targeting techniques and we’ll compare them.’ Furthermore, some of these variables clearly swamp the other ones - for example where the ads run has a huge effect, larger than the targeting technique in most cases.”

Know of an insightful employee blog? Please e-mail the URL to .(JavaScript must be enabled to view this email address), so that I can include it in a future edition of the Inside Word.

Related Stories
Oct 23, 2009 3:10 PM ET

Andrew Chen

Share

Posted In: Advertising, Features, Inside Word

  • Joseph Tartakoff

    Hi Magwa,

    What interesting blog posts that have been erased are you referring to?

    —Joe Tartakoff, paidContent.org

  • magwa

    I notice all the interesting blog posts have been erased.

    The netflix price simply proves that netflix algo was already state of the art.

    Any one person can do a subset of recommendations better than an algorithm, of course, but you cannot do it at scale.

    This is where people misunderstand the application of these algorithms and their utility.

  • Russell Glass

    Very interesting insight, particularly from someone that designed products at Audience Science.  Does this mean that Andrew believes that all behavioral targeting is pretty much bunk?

    The push back here is that not all targeting is created equal.  At Bizo we target specific business people based on their business demographic (bizographic) information, and the lifts can be significant (many cases well greater than 100%).  Granted, we're not looking for behavior—just the demos of people that are a fit for B2B products and services.  And to Andrew's point, we've found that the media is also important and the best lifts are created through a combination of data and media.

The Economics of Content | paidContent Newsletter

Know something we don’t?

Send Us a News Tip

All tips are anonymous and untraced.

Sponsors

Contributors