Many of the ad innovations have been a hit with brands and the trade press. But what they haven’t done is stanch a troubling decline in the Times‘ digital ad revenue. What’s the problem? To get a better idea of how the Idea Lab fits into the Times‘ operations and the company’s revenue strategy, I dropped by for a chat with Todd Haskell, the group’s VP of advertising.

Haskell began by explaining that the Idea Lab was spun off in 2010 from the Times’ R&D Lab, a blue-sky experimental forum, in order to create practical products for advertisers in a short time. He described recent examples:

• A Prudential campaign that invited readers to pull up the front page of the New York Times on the day they were born. The ad format, which drew on pre-existing Times’ archival tools, was a hit on social media. Here’s a screenshot:

• A Wisk campaign that was derived from a Times’ culture story about how Picasso painted over canvasses. The original story invited readers to wave their mouse over a Picasso photo to reveal what lay underneath. For the detergent ad, Wisk asked readers to use their mouse to undercover underlying dirt stains. According to Haskell, those who did spent an unusual amount of time fussing over the ad. Here’s a screenshot (you can touch up Picasso here and play with stains here) :

• The Times‘ city guide app, The Scoop, inserted a Citibank map for New York City’s new bike-sharing program

• A National Geographic campaign drew on the “Times Machine” (which Haskell says is normally used only by “history geeks”) to provide historical backdrop for its Killing Lincoln show

According to Haskell, the New York Times‘ digital story-telling machinery is appealing to companies as a way to convey heritage and complicated brand stories. He adds that clients like Prudential say they have had tremendous response to their campaigns, including huge lifts from social media.

But despite the promise of such ad tools — and clever platform tools like Ricochet and Sparking Stories – the Times’ overall ad performance is limping. Recent earnings results show that digital ad sales are not just flat but actually declining — a troubling development at a time when digital revenue is supposed to stabilize the company as it faces a permanent decline in its print business.

Haskell says the company has been unable to pre-sell all its inventory, and attributes the overall ad challenges to two factors — “an explosion of inventory from social channels” (read Facebook) and the rise of

automated or “programmatic” buying which lets advertisers purchase digital ads on real time exchanges.

New York Times Idea Lab, photo by Rani Molla

He predicts, however, that the new formats will help woo brands back from the programmatic trough; he adds that the Times‘ reader data and performance metrics will also persuade

companies to spend more money. Finally, Haskell also sees programmatic as an opportunity as well as a threat — the Times is offering customers who make premium buys a chance to participate in a private exchange of Times ad inventory (where prices will stay higher since the Times ad slots aren’t sloshing around with shady fly-by-night websites like Toothbrushing.net).

All of these questions will be sitting on the doorstep of the Times’ newly minted head of sales. The company announced the new position in April, but says its still having a good long look for candidates.

MLB is also planning online ad exchanges in which brands can bid in real time to show ads to a specialty audience — say affluent female car buyers — on sites they own such as the New York Yankees team page. According to a Thursday press release:

“Advertisers now can identify affluent audiences based on a range of demographic, behavioral and purchasing attributes and target them across all MLB properties.” The ad initiative reflects MLB’s role as the most tech-savvy sports league; to learn more, come see MLB Advanced Media CEO Bob Bowman join us at paidContent Live on April 17.

The release was put out by BlueKai, a company that collects and analyzes data from consumers as they move across different websites. In a phone interview, BlueKai Director of Business Development, Gina Kim, said the company is providing MLB with a data platform but that the league will not use it sell customer information to other publishers.

MLB’s plans to create more granular advertising segments also reflect a broader trend across major websites. Facebook, for instance, announced this week that it’s now using third party data companies to offer super-specific audience segments like “children’s cereals” or “full-size sedan buyers.”

The BlueKai executive said the data deal for now just covers desktop browsers and not mobile devices.

A new study by Forrester Research claims that so-called “programmatic” buying or “real time bidding” will account for nearly 25 percent of online video ad purchases by next year. This mirrors what is going on in the world of display advertising where more big advertisers are using ad tech tools to serve ads to diverse audiences in real time.

The report, which was commissioned by SpotXchange (an online video exchange that has skin in the ad game), also says that premium publishers have been slower to adopt programmatic bidding, in part because they fear it will undercut the value of their inventory. The report predicts, however, that many of these hold-out publishers will change their position as brands get accustomed to programmatic buying and begin to demand it.

The impact of programmatic on video ad prices is debatable. People in the ad tech industry point out that automated ad buying is simply a tool — not a reflection of ad quality. By this reasoning, publishers can hold their pricing line if they wish while also ensuring that their space is available in real time when there is a surge in demand. Conversely, as the report points out, publishers remain wary that brands will use the tools (as they did for display advertising) to drive down prices.

Overall, the future of video prices in the short term may be determined less by ad tech tools than by more basic principles of supply and demand. On this front, the good news for publishers can be seen in this chart which shows online ad spending rising quickly:

Another recent report is even more optimistic — pegging the 2013 number at $4.1 billion.

The bad news, though, is that the word is out about video’s promise and more and more people are showing up to grab a slice of the pie. Ad industry sources told the Wall Street Journal last month that there is ”not enough to feed everybody.” The Journal reported that, despite brands beginning to reallocate their TV budgets, prices are already under pressure; $15 to $20 per thousand views (CPM’s) last year versus a CPM of $17 to $25 in 2011.

The Forrester report also predicts that video ad inventory will be become increasingly divided between private and public exchanges.

This is about more than just some tinkering with the machinery that underlies the content we consume. As Felix Salmon notes in a post at Reuters, virtually every media entity — from behemoths like the New York Times to the smallest independent online player — is being forced to reinvent how they generate revenue because digital advertising is not paying the bills (Note: We will be discussing alternative monetization methods on a number of panels at our paidContent Live media conference on April 17 in New York).

Another battle between algorithms and humans

Like most of the other changes in the media industry, this shift didn’t just just happen overnight. Instead, it’s a wave or a series of waves that have been building steadily over the past several years. As with so many other elements of the disruption in digital content, Google arguably triggered this particular tsunami with the introduction of its search-keyword based auction process — an idea it borrowed from Bill Gross and Overture — but the ripple effects of that decision have continued to grow and expand in force.

In a nutshell, advertising has become yet another battleground for the fight between humans and algorithms: the human beings at ad-buying firms and ad networks who used to buy and sell banners and other traditional forms of advertising, and the algorithms that drive “programmatic” buying based on keywords, topics that are trending in social media and other factors.

As we described earlier, this is something the New York Times has been experimenting with via an in-house tool that tracks which stories are seeing the most activity on Twitter, and then offers advertisers the ability to insert their ads into those stories. It’s no longer about where that content appears in a physical product like a newspaper, or even what the story is actually about. Instead, it’s more about who is engaging with it, and where, and how.

Part of this shift is the transition from what some marketers call “outbound” marketing — which includes direct mail, banner ads and other methods that try to reach out and grab the attention of potential customers — to “inbound” marketing, which relies on search-engine optimization and other content-based strategies that make it easier for users to find a brand or advertiser without being bombarded by ads (Demand Media and other “content farms” have tried to apply this approach to content, with mixed results).

Hubspot, which specializes in inbound marketing, says it is more or less taking over the online world, and the consequences for traditional media companies are fairly obvious:

Automated buying vs. human-created content

So what does all this have to do with the Twitter advertising API? In a nutshell, the API is a way for companies to automate more of their ad spending. At the moment, the list of official partners isn’t that long, but it will presumably grow — and the API combined with Twitter’s self-serve ad platform will theoretically allow advertisers to promote tweets based on what is trending and where the activity is. Twitter is also likely providing a host of information around users and their interests.

This is essentially the same game Facebook is playing, and while Google doesn’t have an open API for Google+ yet, it is likely thinking along the same lines. For social networks, in which the content generated by users is almost indistinguishable from the advertising — and in the case of Facebook, actually becomes part of that advertising, through features like sponsored stories — offering tools that let advertisers automate their spending based on hard data is potentially far more lucrative than another generic banner ad.

The problem for many media companies is that this is a game they are ill-equipped to play: for the most part, they have little or no data about their users that can compete with the granularity that Facebook or Twitter can offer, and they have no APIs or other automated, self-serve features to offer even if they did have that kind of data. On top of that, brands are setting up newsrooms and becoming content publishers in their own right, and further disintermediating the media.

This is part of the reason why so many players like The Atlantic and even Gawker Media have been focusing on alternative methods such as sponsored content or “native” advertising or affiliate links. But as Salmon notes in his post, these kinds of approaches are often labor-intensive (if you want them to be effective), and therefore high cost. In some ways, the market seems to be bifurcating: on one side is a growing business driven by algorithms, and on the other is a human-driven business based on customized content. Is there room for both?

Post and thumbnail images courtesy of Shutterstock / Everett Collection and Borrell Associates