Ken Doctor analyzes the news business on his blog Content Bridges and for Outsell, an information-market analytics firm. He worked at Knight Ridder for 21 years and was an executive in content services, strategy and editorial for the company’s digital unit.

You can see the familiar battle lines drawn, in this week’s script: At the Newspaper Association of America’s annual conference in San Diego this week, the semi-mad newspaper owners open the web network window to scream, “I’m mad as hell and won’t take it anymore” while web denizens below merrily prepare to dance on those owners’ graves, deriding how out-of-touch with internet reality those owners are. AP fights gamely on behalf of the Old Guard as New Guard vanguard Eric Schmidt tells it like it is, even urging publishers “not to piss off their customers.”

I think it’s time we get beyond this tired storyline and confront the realities of the moment. Just as God didn’t ordain that newspapers should drive 25 percent-plus profits from their daily monopolies, God didn’t set the payout rules that drive current web business models. It’s time to reboot the conversation and devise business models that represent real-world needs. The two big needs: maintaining the free flow of global news and information and figuring how to pay people to create journalism and other useful content we all need.

Meeting both those goals is the key, and it won’t be done with a single bold stroke. It’s time though for a reckoning. That reckoning can rejigger the relationships between the new mass media of our day—Google (NSDQ: GOOG), Yahoo (NSDQ: YHOO), MSN, AOL (NYSE: TWX) and increasingly the emerging Facebooks—and news producers.

How to achieve that reckoning, after the jump

That reckoning means moving beyond the fatigued arguments of another day. “Fair use,” which some believe is rooted in the UK’s 1709 Statute of Anne, has served a great public purpose. The simple idea: stimulate the public discourse by allowing judicious excerpting, while maintaining producer value through copyright. As web companies first started “excerpting” in the early days of the web, it set corporate attorneys atwitter. Was search engine or other excerpting “fair use”? I was involved in too many of those conversations at Knight Ridder, and with counsel of other newspaper companies.

The short answer is: we still don’t know. Newspaper companies—fearing they would lose in court—have failed to pursue the question of fair “fair use,” in the digital age. In fact, in the immortal words of Harvey Cox, “not to decide is to decide,” and in their indecision, newpaper companies have lost.

AP, Britain’s PA, Canada’s CP and AFP all huffed and puffed about fair use, threatening legal action and won licensing contracts with newspapers. Those are the “multimillion dollar” payouts Eric Schmidt noted as he professed surprise at all the publisher consternation that had erupted in San Diego prior to his Tuesday keynote.

Individual publishers, though, have not won licensing deals. One reason newspaper companies haven’t banded together to threaten suit and demand payment: “anti-trust.” That’s another legal concept that’s served the country well. It, like, “fair use,” though didn’t anticipate the upending chaos of the Internet.

So two good concepts, made in part obsolescent, by the way we communicate and the way journalism now operates.

It’s time, especially at this time of national re-thinking and re-jiggering, to get beyond “fair use” to “fair share.” It’s as American a concept as you can devise, and it’s one that plays directly to our moment in history.

Read or watch Eric Schmidt’s remarks, and you’ll find them quite reasonable. He’s a smart guy who can put the digital world into context. They are, however, disconnected from the reality of our times. Here he is talking to a newspaper industry that has already seen five bankruptcies, newsroom cuts of greater than 20% and a downsizing future as far as any impartial observer can see, and he’s talking to them about the transformative power of the mobile experience, how ad models will work out in the end and micropayments. We need to be talking about macropayments.

Astounding.

It’s like offering a five-year plan to a group that’s found itself afloat on a deserted island with little food. Sure, we can argue that some of those publishers are somewhat responsible for their straits, and I’d be the first to agree. The fact, though: they are still stuck on the island. And we’re all held hostage by their plight, as the flow of what we know about our communities, our nation, our governmental policies and our business practices is reduced daily. We don’t know what we don’t know, but we do know we know less than we used to.

How might fair share work and how can we justify it?

First, it’s important to stress that this isn’t about Google and isn’t about newspapers. Both have their strong and weak points, but it’s not about them. It’s about us.

On the first hand, it’s about that excerpting of fair use and re-interpreting it for digital times. That excerpting (indexing, snippetizing, etc.) is done most successfully by Google, but also by the other search engine and by tens of thousands of sites. That’s what’s behind this week’s AP’s anti-piracy push.

On the other hand, it’s not about newspapers and providing a Legacy Media Bailout Act of 2009. It’s about enabling the production of authentic, independent newsgathering by providing a going-forward way of compensating the work it takes to produce it. That work can go on in within the confines owned by multinationals like News Corp (NYSE: NWS). Thomson Reuters (NSDQ: TRIN) and Gannett (NYSE: GCI) or rented by the recently (and deservedly) awarded Voice of San Diego or the Huffington Post or Slate or TechCrunch. It’s the free flow of news production we care about, not preserving the companies.

So fair share would simply recognize that the first stage of web monetization has been, well, a bit simple. There’s little nuance to it, with way too much value accruing to the search and aggregation players, and far too little to the content producers. There’s nothing particularly evil about this; it’s just what happened. (For context, consider the parallel discussion of “conversion attribution” in the online ad world, a spirited argument that has fallen off the table recently.)

Google is our proxy here for search engines and other indexers of content. Why? It’s the leader, providing 25 percent to 35 percent of the traffic to news websites (while the combined traffic contributions of the others push that number to well over half the traffic many news sites receive). It indexes news websites, and out of more than 4,000 news sources, produces Google News. Ask Google how fair this arrangement is, and it will cite two things: 1) the fair-use legal argument; and 2) the amount of traffic it sends news websites. What it is, in fact, saying is, hey, that’s fair, that’s even.

Wait a minute, maybe legally fair. But even? How do balance value here? The way it works now is that whatever value Google can derive from snippetizing news is fair game, and even. If it produces a million in revenue related to news, that was fair. If it produces billions related to news, well, that’s fair and even too.

In fact, Google derives lots of value from the news it presents, and we have no clear accounting of that value.

Consider a few diverse data points here:

  * Here’s the grossest number: $21.7 billion. That’s Google’s 2008 annual revenue. Big number, and growing at a 24% rate. How has Google weathered the recession? Well, better than most, we think. We’ll see first quarter results Tuesday. Compare that number to how much advertising revenue the U.S. newspaper industry has lost. It pulled in $47 billion in 2005 and will come at maybe $36 billion in 2009. Certainly, there’s not a one-to-one transfer of those dollars, but do all the forensics you want, and you’ll find dollars formerly spent on newspaper ads are now spent on Google ads. Advertising makes up about 97% of Google’s revenues.

  * Google News, launched in 2002,  is now a big player. Number 11 in the U.S., with more than 11 million monthly unique visitors, according to Nielsen. A month ago, it struck a new match in what now emerged as a wildfire, when it added its ad placements, for the first time, to Google News. How much in ad revenue will Google News create? We don’t know yet, but we will be able to guesstimate this year. If we look at February Nielsen data, though, we see that Google News produced a little less than 10% of the Google’s overall uniques (11,379M of 127,142M). That puts it seventh in the Google cosmos, behind Search, Maps, Image Search, Gmail, Account and Toolbar Features. In Time Spent, though, it’s third. On average, 16 minutes, 44 seconds per person. That’s behind Search (29 minutes, 16 seconds) and Gmail (predictably the highest at 2 hours, 48 minutes, 27 seconds). That’s 16 minutes of engagement—of readers’ attention—every month in Google News. You can see why Google finally got around to monetizing it. Of course, Google will note that none of that $21.7 billion in 2008 revenue came from Google News, so publishers have no reason to get out their calculators…yet. 

  * The business value of Google News to Google….and that $21.7 billion. “Huh,” would be one way to gauge Google’s response to that point. Yet, what has Google built but an incredible multi-purpose brand, in which the separate pieces (Web Search, Finance, Video, GMail, Images, Maps and More) all connect to each other, building the experience, and somehow, throwing off $21.7 billion. So, clearly there’s a value to Google when Google News users come to the site and then click on other Google links. How much? Google says it hasn’t calculated that impact. That seems hard-to-believe. Let’s say that 20% of the referring pages to Google Search or Maps or Video come from News. Then, by traditional web economics, the referring site should derive some economic benefit, some rev share from the ads sold on the landing pages.

  * Two-thirds of Google’s revenue now derive from Google-owned sites; less than a third from its partners. Not more than several years ago, those numbers were reversed. The we’re-not-a-destination portal has become a portal, driving more revenue on its “own” pages, sharing increasingly less of it with partners.   

  * Google’s cost of sales is remarkably low. John Battelle has pointed out, pegging it at less than 10%. Of course, its cost of sales is low: It largely doesn’t pay suppliers for the raw material enabling the business.

Add it all up, and you can make a pretty good argument that Google is deriving a disproportionate share of the value from the content that it has so magically aggregated, searched and presented. You can see that mere snippets—intended to advance the public discourse—have had unintended consequences, enabling huge businesses and depriving oxygen from those who create the raw material from which snippets are harvested. You can make an argument that Fair Share wouldn’t be about radically changing Google’s magic or the wider web’s mojo. It would be about nuance, of recognizing value along the chain, from the production of (news) content through its harvesting and presentation.

We’re not saying Google doesn’t deserve money for its magic. We’re just saying it should fairly share the wealth. What’s fair? Well, some percentage of gross revenues. As big as the web business has become, it’s only in its infancy. Online advertising will continue to outpace ad spend overall, and content producers want to get on the ramp. 

Assuming some percentage of payout for supply, what’s the mechanism? Royalty pools have long been used by legacy content aggregators—think LexisNexis and Factiva here. Who’s in the royalty pool? Certainly, media upstarts as well as 150-year-old companies. It should be a democratic pool, certainly with some kind of minimum threshold daily (weekly, monthly?) content production.

How would you allocate revenue to those in the royalty pool? Here’s my favorite answer: Create an algorithm. How Silicon Valley can we get? Figure out a threshold of content supplied, kind of content, reader usage, number of links, etc., and apply Silicon Valley smarts to a long-standing business tradition. Google will tell you it gives them a headache to think about such a complicated formula, but, let’s remember that’s why we’ve educated (and Google has employed) so many computational linguists. Let’s say the first algorithm isn’t just right. Let’s go to rev 2.0.