But what if your biggest source of traffic and readers is something you aren’t even really paying attention to, and something that is extremely hard to track in the same way as Google or Twitter or Facebook? That’s the reality of web publishing today, according to Alexis Madrigal at The Atlantic — who writes about the influence of what he calls “dark social” on engagement and traffic patterns. While everyone is busy watching Twitter and Facebook because they are easy to track, Madrigal argues that most social traffic still comes from old-fashioned or difficult-to-track sources like email and chat messages:

“This vast trove of social traffic is essentially invisible to most analytics programs. I call it DARK SOCIAL. It shows up variously in programs as “direct” or “typed/bookmarked” traffic, which implies to many site owners that you actually have a bookmark or typed in http://www.theatlantic.com into your browser. But that’s not actually what’s happening a lot of the time. Most of the time, someone Gchatted someone a link, or it came in on a big email distribution list, or your dad sent it to you.”

Most of your social traffic is hidden from you

As evidence, Madrigal provides some data from Chartbeat, the Betaworks spin-off that focuses on real-time analytics for publishers, looking at everything from the amount of time readers spend on a page to how far down they got in an article before they decided to click away. Chartbeat, which we have written about a number of times, is one of the few analytics engines that tries to break down that “direct” category into sub-categories like email and what the service calls “direct social” — meaning everything from apps (for chat or other social features) to instant messaging.

For The Atlantic, the impact of this kind of direct social traffic outweighs any other kind of social network or service like Facebook, Twitter or Reddit: according to data from Chartbeat, the magazine’s website gets almost 60 percent of its social traffic from these hard-to-track sources. Facebook is still a large referrer for the site, generating about 21 percent of the social traffic, and Twitter is also fairly large at 11 percent — but the “dark social” category is larger than all of the other social services combined, and has more than twice the impact that Facebook does.

Chartbeat’s numbers also showed that this direct-social traffic was a large contributor for other sites that the service tracks, according to Madrigal — and Chartbeat is used by some of the largest publishers in the media business, including ESPN and the New York Times. On the average, the company’s stats showed that close to 70 percent of the social traffic to these sites came from email, instant messaging, chat apps and other sources (as Madrigal points out, at many websites including The Atlantic, social traffic far outweighs traffic that comes from search, and that gap is still growing).

The one obvious conclusion to take away from all of this is that measuring user engagement and sources of traffic is probably a lot harder than most publishers think — and they likely already thought it was pretty hard. It’s bad enough that comScore and Compete and Nielsen and Google Analytics all provide different numbers, and it’s almost impossible to tell who is right (especially since all of these sources often disagree with a publisher’s internal statistics). Now there is a huge source of traffic that is even harder to measure: email is trackable in the aggregate, but how do you track instant messaging?

The only solution is to create engaging content

This problem is compounded by the shift to mobile content consumption as well, since chat apps and instant messaging and other direct communication methods are even more prevalent in the mobile world than on the desktop. Links are passed from social network to apps to chat to email, and tracking them quite quickly becomes almost impossible. That’s part of the reason why almost all web publishers get surprised by posts or stories that blow up traffic-wise days or weeks after they first appeared, with no obvious sign of how or why they hit that invisible tipping point.

So how are publishers and media companies supposed to deal with this problem? Madrigal’s solution is an appealing one, at least for those who create content — he says the only dependable way of generating real traffic and engagement is to actually write things that people care about or are interested in. In other words, the “content is king” approach. As he describes it:

“The only real way to optimize for social spread is in the nature of the content itself. There’s no way to game email or people’s instant messages. There’s no power users you can contact. There’s no algorithms to understand.”

It’s not that getting people to share content via email or chat app or instant messaging service is that different from trying to get them to share it on Twitter or Facebook — the same general rules apply, in the sense that it has to be engaging and interesting and shareable, and all of those other things we are supposed to be doing with our content. But the difficult part is that it’s hard to track in the same way publishers watch Facebook “likes” or page followers, or Twitter re-tweets and other metrics. And so it’s difficult to tell whether it’s working and why, or what you should do differently.

In a sense, what Madrigal is describing just reinforces the fact that much of what content companies do is more of an art than a science — even though social-media gurus and analytics providers would like to make it sound like something that can be quantified and measured from every aspect. And maybe that’s not such a bad thing, even if it does make our jobs harder.

Post and thumbnail images courtesy of Flickr user George Kelly and Alexis Madrigal

Launched three years ago as a way of giving publishers and media companies better real-time analytics related to their content, Chartbeat announced Monday that it has closed a new round of Series B financing worth $9.5 million from a series of venture funds including Index Ventures and Draper Fisher Jurvetson. The company — which was developed by New York-based incubator Betaworks, creator of Bit.ly and News.me — said it has also upgraded its analytics dashboard with new features that give websites and publishers better ways to measure actual reader engagement. And in the age of digital content, being able to do that accurately is a crucial weapon for publishers in the fight to show advertisers that their content has value.

Tony Haile, Chartbeat’s CEO, says one of the goals for the company from the beginning was to produce better analytics so that publishers could understand who was interacting with their content and when — on a page-by-page and author-by-author level — in something approaching real time. While most analytical tools for websites and publishers such as Google Analytics and specialized tools like Omniture pay most of their attention to pageviews and other page-based metrics, Haile says that Chartbeat wanted to give content creators a much more granular view of what readers were doing with their stories, and to track all that in real time.

So using the Chartbeat dashboard — which a number of major media entities like ESPN and the New York Times do — a publisher can see that a specific blog post or news story is getting a lot of readers who are coming from social media such as Twitter and Facebook, and a smaller number from search or direct traffic. They can see where those users are coming from, in real time, and how long they are spending on the page, and whether they have reached the bottom of the page or left without scrolling down.

One of the biggest differences with Chartbeat when compared to something like Google Analytics, says Haile, is that while Google and other programs only check once to see if a visitor has opened a page — and then show that page as being read until it is closed — Chartbeat continually pings every few seconds to see if the page is still open, and therefore it has a far more accurate reading of which pages are actually getting engagement from readers, as opposed to simply being open in a browser tab somewhere.

That’s important in an age where advertisers are increasingly looking for engagement with content as a metric of value, instead of just a raw page-load or browsing behavior — and are getting what they want from social networks like Facebook, which has been capturing a growing share of online advertising revenue. In another new feature, Haile says that Chartbeat also shows publishers how their metrics around engagement and social sharing compare to others in their industry (although the industry data is anonymized so no specific competitors are mentioned).

From a user’s perspective, there is just one big problem with Chartbeat: namely, that it quickly becomes addictive to be able to see the exact number of readers a post has, where they came from, and how long they spent on the page (even though in many cases those numbers may be lower than a writer might hope for). The speedometer-style dials and gauges that show each incoming and outgoing visitor, the map that updates with the location of the latest reader, and the table of posts that shift around based on which is getting the most traffic are almost hypnotic.

Chartbeat’s funding round — which brings the amount it has raised to $12.5 million — was led by Josh Stein at Draper Fisher Jurvetson and Saul Klein at Index Ventures, and Haile said the funding group also included many of the angel investors who backed Chartbeat in the beginning. In addition to Google Analytics and Omniture, the company’s competitors include startups like Go Squared and Woopra. John Borthwick, CEO of Betaworks, will be at PaidContent 2012 in New York in May.

Post and thumbnail images courtesy of Flickr user Skyhawk4life

Chartbeat says it has 2,500 paying customers, including Gawker Media, the Chicago Tribune and DailyKos. The new funding round was led by Index Ventures. Other investors include O’Reilly AlphaTech Ventures, Freestyle Capital, Lowercase Capital and Jason Calacanis.