While content engines have been around for a while, their growing presence is influencing how readers explore the web. They are also taking on a new importance as vanguards of “native advertising,” a trend that many hope will reinvigorate the online ad economy.

Here’s an overview of the content engine industry and what it means for publishers, advertisers and readers. (To learn more about the latest publishing strategies, be sure to attend paidContent Live, April 17 in the New York).

Keeping readers close to home

New York magazine uses a variety of tactics to increase visits to its website and to induce readers to stay on the site. These include boxes showing its most-popular stories as well as cross-promotion arrangements with other news sites.  New York has also recently signed on with content engine nRelate, which uses algorithms to display other stories a reader may find interesting; the stories come from New York’s own site or one of its affiliates.

nRelate promises publishers more pageviews but also a new revenue stream as a result of “sponsored stories” it inserts in the list of recommended content – if a reader clicks on the advertiser’s “story” (typically a promotional article written like a news story) the publisher and nRelate share the money.

Michael Silberman, the GM of digital media at New York, says he welcomes the chance for extra revenue but that the main purpose of the nRelate experiment is to get readers to stay on the site. And, like so much in publishing these days, the outcome will be data driven. “We’ll do A/B testing to see if there’s an overall lift. If all it does is redirect traffic we would have had anyways, it’s not worth it. If it leads to more pageviews per visit, then yes,” said Silberman, adding that publishers have to take care that tools like nRelate don’t clutter or slow down the site.

Tools like nRelate promise more money or traffic but they also represent a growing strategic importance for publishers. The reason is that, in age of the social media, fewer and fewer readers arrive at a website directly through a publisher’s home page; this means that publishers are doing more than ever to persuade them to stick around.

“We find [Outbrain] quite useful as a tool to drive traffic to our sites and circulate traffic through our sites.  The numbers suggest that our users find it useful as well,” said John Lerner, CEO of Breaking Media, which owns sites like Above the Law. (see screenshot at right). He did not provide specific numbers.

Meanwhile, the content engines are also a source of money. Joshua Albertson, the President of Curbed Media, says in a good year, they can bring in six figures of revenue through its sites Curbed, Eater and Racked.

Rivals collide on a new advertising frontier

While tools like nRelate hold promise for publishers, they’re also being pitched as a way for advertisers to break through to readers who tune out conventional online ads. The idea is that, by packaging the ad as a story to “read next,” a reader is far more likely to click on it and digest the marketing message.

This ad format – call it “native advertising” or “content marketing” or whatever you wish – has produced a growing list of companies that want to serve up sponsored stories. The biggest of these is Outbrain, which has already received $64 million in funding and claims to be on more than 90,000 blogs and websites like Slate, whose editor, David Plotz, says he has been pleased with the three-year partnership.

“We’ve been really happy with it. We needed to provide related links and Outbrain combines it with a tool that makes quite a lot of money for us.”

Outbrain’s hold on the market is far from firm, however, as competitors have been peeling away some of its clients. According to Curbed’s Albertson, the company decided to switch after it found Outbrain’s performance declining and rival Taboola offered it a bigger share of advertising revenue. Here is a screenshot of Taboola, which serves both stories and video and recently received $15 million in new funding:

As the rivals move in, Outbrain has begun slapping its logo on its boxes of recommended stories in an effort to be seen as a brand. It also recently fired a quarter of its advertising clients because of the poor quality of their story content. The company explained the decision this way:

“If companies coming into this space are not respectful of the audience who are clicking on these links, then the user trust will decline over time. That means this space — which is currently so valuable to the audience and publisher — will be just like the the display ad market. People will become ‘content link blind,’ just like they became ‘banner blind.’”

Meanwhile, other companies are pivoting from their core business to get into the content engine game too. These include dictionary company Reverb, which believes its experience with syntax will make it good at finding content, and the commenting platform, Disqus.

In an interview, Disqus CEO Daniel Ha said the company has an advantage due to its existing relationship with thousands of publishers, and the ability to offer brands a way to reach hyper-engaged comment communities. Ha added that the Disqus strategy is based on the “independent web” and “the middle-tail of websites,” pointing to examples like a Toronto Maple Leafs fan forum and coupon site Southern Savers.

A boom or a bubble?

Content engines may be a hot topic but is there enough money and web real estate to go around? Neil Mody, the CEO of nRelate, says there is not. “There’s a classic network effect. You’ll get one big winner. Numbers two through five will do OK. Numbers 6 thru 20 will go niche or fail.”

Mody says there is already evidence of a coming shakeout. He claims some content engines are paying publishers to get on their sites even though they have no ad revenue. He also says the industry is over-capitalized and that there is risk of a backlash as some content engines flood the space with junky “lose your belly” type of articles.

Another challenge for content engines is the low cost of switching for publishers. The engines are built so that it’s easy for publishers to put them up, but also take them down — a situation that doesn’t favor long-term relationships. The result is that publishers can simply sign up with whomever offers the best content or most money. This means the winner is likely to be content engine with the best algorithms and the highest quality advertisements.

paidContent caught up with CEO Yaron Galai in Outbrain’s office near New York’s Union Square this month to hear about his plans to stay on top. In the (likely) event you’re not sure what Outbrain does, here’s a screenshot from Time that shows how and why people and publishers use it:

The business model is pretty simple.  Outbrain helps publishers keep readers on their websites by giving them a tool to surface related content that they’ve published in the past. The company’s “From Around the Web” tool also provides a way for publishers to buy and sell traffic — in the example above, if someone clicks on one of the related stories links, Time might get a referral fee from the outside publication (of which Outbrain would take a cut). Small publishers can use the service for free to surface their own content but once they reach a certain volume of traffic, they’re obliged to add paid links. Outbrain claims its tools are installed on more than 90,000 blogs and websites, including big names like CNN, the New York Post and Slate.

So far, Outbrain has done pretty well. The company is backed by $64 million in venture funding and claims its recommendations are clicked on billions of times a year.

But now competitors are arriving, including comment site Disqus , which has relationships with millions of websites, and Wordnik, a dictionary site that says its semantic knowledge lets it offer better recommendations. The would-be rivals could undercut Outbrain by copying its service and offering it for a lower price or even for free. Meanwhile, Outbrain must also contend with the enormous referral power of little companies with names like Google, Facebook and LinkedIn.

Yaron, however, has had to defend his turf  before (most famously while closing a funding deal as an officer in a combat zone). In doing so, he’s made some very bold decision such as firing legions of his own customers for furnishing spammy content. The decision may eliminate up to 25 percent of the company’s revenue but Yaron thinks it will pay off as a long-term strategy. ”A lot of the clones will show good results for the short term [but will soon fade],” he says.

Yaron believes Outbrain has deeper relationships with major publishers and advertisers like GE that make higher-quality ad content– and that that will allow Outbrain to become the go-to place for industry leaderes to do business with each other. To boost the effort, Outbrain is also starting to brand the recommendation with its logo in the hopes readers begin to notice it.

Yaron compares Outbrain’s situation to the early days of search advertising when dozens of competitors scrambled for a piece of the action until Google AdWords finally blew them all out of the water. It’s an impressive vision — but, at this point, it will be at least a year before we know if it can come true.

(Image by ollyy via Shutterstock)

Reverb, which officially launched today, draws on the semantic knowledge of its parent site, Wordnik, an online dictionary that assigns meaning to words based on both standard definitions and real-life context. Reverb believes Wordnik’s technology will allow it to do a better job finding relevant stories than other content suggestion engines.

“We want to go beyond the lowest common denominator of celebrity posts,” said CEO Tony Tam in a phone interview, and claimed that Reverb’s ability to surface relevant content means publishers will get more life from their old stories. Tam says more than 1,500 publishers and blogs are so far using the technology, including sites like Laughing Squid and Dressaday.com

For now, Reverb’s service is free to publishers. The no-charge model appears to be an attempt to get a foothold in the market for story suggestion engines. Right now, that market is dominated by Outbrain, a New York company that helps publishers surface their own stories but also place links to content from around the web. If a reader clicks on one of the outside stories or ads, the publisher gets a commission that it shares with Outbrain. Meanwhile, commenting service Disqus has also launched an ambitious plan to make money from proposing suggested stories and ads in the reader comment section of websites.

At one end of the debate are skeptics who think comment sections are cesspools of trolls and cretins. At the other end are publishers like Gawker’s Nick Denton and New York magazine who believe reader contributions should be a core part of their editorial strategy.

Are readers who look at comments worth more than regular readers?

It’s no surprise that Disqus is in the “reader comments are great” camp — after all, the company not only sells commenting tools but is also building up a second line of business dedicated to turning comments into a forum for advertisers . The service, called “Promoted Discovery,” does this by perching links to paid-for content right next to reader comments. It’s already appearing in the comment sections of sites like Mens Health and the Observer and it looks like this:

As you can see above, the tool lets a website surface its own content on the left side while selling space in the right hand “recommended for you” box. In trying to tap this market, Disqus is competing with market leader Outbrain which also helps publishers and advertisers buy and sell web traffic.

Disqus’s big pitch to advertisers is that someone who takes the time to read the reader comments of a story is likely to be passionate about the topic — and more likely to click on, buy or otherwise engage with the proposed content. In its December “report card,” Disqus claimed readers who dwell in comments are more likely to visit other pages and spend more than twice as much time on the site. Meanwhile, reader comments overall are being treated with a new seriousness due to recent studies that suggest they shape perceptions more than we thought.

Big money from “native advertising?”

Disqus is also betting its promoted content will gain traction as a form of “native advertising.” The concept, hailed by some as a magic bullet to solve falling online ad prices, involves selling ads that mimic the content around it — for instance, sponsored tweets on Twitter. In the case of Disqus, the American Express story in the above graphic is a “native” ad. The company is so confident of the concept that it’s betting that the advertising scheme will soon become the bulk of its business.

“We expect advertising revenue to grow from less than 5% in Q4 of 2012 to over 60% of our revenue by the end of this year,” said the company’s CEO, Daniel Ha, by email.

It’s an ambitious goal but can Disqus pull this off? It has a number of factors in its favor, including the presence of veteran VC and native advertising champion Fred Wilson as one of its investors. The company is also in a good position to sign up clients due to the fact that it already has access to millions of websites.

On the other hand, the company’s quest to turn reader comments into gold faces some obstacles. One is Disqus’s capacity to find relevant content. When I visited several sites using “Promoted Discovery,” the suggested stories it proposed often had nothing to do with the article I was reading. A Disqus spokesman responded by saying the product is brand new and that its capacity to propose content will improve rapidly with use and as inventory grows. This seems a fair response — it’s likely Disqus will improve with scale.

The longer term challenge to making money from content is likely to hinge on publishers’ willingness to offer a service that will take readers away from their own website. For now, though, the websites are likely to simply welcome the extra money they receive from Promoted Discovery; Disqus says the first batch of checks is going out this quarter. It will be interesting to see what type of revenue-share arrangement Disqus and other middlemen will ask going forward — a Disqus spokesman wouldn’t disclose any specifics, saying the firm’s take is “industry average.”

And, finally, Disqus’s bet that comments will be an advertising gold mine could also be affected by competition from much larger players such as Tumblr; its founder, David Karp (who is speaking at our paidContent 2013 media conference in New York on April 17), recently said the site downplayed comments in favor of “a beautiful design and no jerks.”

(Image by ollyy via Shutterstock)