Now Pearson is giving developers the opportunity to remix some of its best-known books for themselves, through two new API data sets:

• It is making available 90,000 raw images, of the kind how-to and guide book publisher Dorling Kindsersley (DK) has become famous for, including of human anatomy, city maps, scientific images and stock photography.
• Meanwhile, Pearson’s Penguin book publisher is making available 48 classic titles from 31 authors including the Brontë sisters, Oscar Wilde and Thomas Hardy.

The two new APIs go live on what is the first birthday of Pearson’s Plug & Play developer program. Pearson wants to regard some of its content as raw building blocks for third-party developers to make new apps and services from.

The group began its developer program in 2011 with three APIs, so that developers could re-use its Eyewitness Guides city data, Longman dictionary definitions and Financial Times press releases to create new products.

This March, the giant unveiled its first chargeable API that allows developers to re-use a library of 2,500 food recipes from its cooking books, with developers needing to pay after hitting a certain monthly usage.

Pearson, which made around $2 billion from digital content last year, will claim credit for embracing small-scale developers using this home-spun outreach tactic. But it likely won’t make available in this way full premium content from its Financial Times or education services.

So far, Pearson is showcasing 12 real-world re-uses of its material. Commercial payback to the group is not known. We have asked.

After giving a kind of whirlwind tour of the open-source movement in his talk, including the rise of Linux, Shirky devoted much of his discussion to Github — a kind of crowdsourced platform for maintaining code that Linux creator Linus Torvalds also created, which allows anyone to edit, to “fork” or create their own version, and to track the changes that others make. It’s not a big stretch to get from that idea to the idea of crowdsourcing legislation, which is what Shirky seems to have in mind, and there have already been some attempts at doing this via Github: for example, a German software developer has uploaded all of Germany’s laws to the platform so that citizens can recommend and track changes.

It’s an intriguing idea: that a simple software platform aimed at collaboration could change the way legislation is developed and implemented, much like the process that powers Wikipedia has created a crowdsourced encyclopedia that evolves and changes over time. But is it realistic? There were plenty of skeptics who said Wikipedia would never succeed, and yet it has an excellent track record when it comes to reliability, despite some hiccups in the process, such as the recent incident involving author Philip Roth. That said, however, there are also plenty of critics who believe that the “cabal” of editors who control the crowd-powered encyclopedia have too much authority.

Of course, some would argue that we’re already in that kind of situation with most governments anyway, and therefore Github couldn’t make things any worse. And Shirky is not the only one to make this argument: developer Abe Voelker has proposed a “Github for laws” that would take exactly the same approach to crafting and crowdsourcing legislation. There have also been some initial experiments with similar ideas — including Iceland’s new constitution and similar types of project in Finland and Ireland — which shows that others are also open to the concept.

One of the problems with applying a technical solution like Github to a massive cultural and political process like government, however, is that creating laws — even small ones — is very different from changing a piece of code so that Linux can duplicate Windows-style typefaces, or changing the Wikipedia entry on George Bush. And if even those kinds of prosaic examples can lead to the equivalent of a Linux or Wikipedia holy war, which in many cases they have, what hope do we have that politicians can actually use a similar process to change the way that government works? As Shirky suggests in his talk, there’s also a pretty entrenched bureaucracy that has become part of most governments and likely has no interest in relinquishing that control to the crowd.

In his book “Here Comes Everybody,” Shirky described the potentially massive impact of crowdsourcing and crowd-powered social change, and his admiration of Github seems to be part of an attempt to find tools that will help us deal with the tidal wave of human-driven collaboration. This is something we clearly need, so it’s worthwhile to start looking at solutions — and while Github may not be the answer, at this point just about anything is probably worth a shot.

Post and thumbnail images courtesy of Flickr user Fabio Venni

Some of the comments during a back-and-forth discussion we had on Twitter were interesting, so I thought I would excerpt them here, and also include a Storify collection of the debate as well. I’m hoping to talk more about this and other topics with the former Twitter CEO at GigaOM’s RoadMap conference in November.

During his time as Twitter’s CEO, Williams had to deal with the beginnings of Twitter’s transformation from a cool project into an actual revenue-generating company, as it started to acquire third-party apps and caused a backlash among developers that is very similar to the one it is facing today. Williams left Twitter in 2010 when he was replaced by Dick Costolo, and formed a startup incubator called Obvious Corp. with former Twitter colleague Biz Stone (which recently launched a new-media publishing platform called Medium) but the former CEO has remained an advisor to the company and a board member.

While Twitter was trying to figure out in 2010 which external services it wanted to incorporate and which it wanted to leave alone — a process that angel investor Chris Dixon compared to “a drunk guy with an Uzi” — Williams admitted that the company had screwed up its relationship with developers, and Twitter held a whole conference for developers called Chirp that was supposed to try and repair that relationship. In reality, however, the tensions between where Twitter wanted the company to go and how that was going to affect third-party apps remained just below the surface, and erupted again recently after moves like the announcement of new API rules and the shutting off of features to services like Tumblr and Instagram.

What role has the ecosystem played?

Our Twitter discussion started when Williams mentioned a phrase from my recent post about the company’s ongoing struggle with being open vs. controlling the network. I argued that much of the early power and growth of the network came from being open, since many of the things we associated with Twitter — such as the @ mention for users, the hashtag, and even the retweet — were not developed by the company but came from the users themselves, in many cases assisted by third-party apps. But Williams said that this influence is “a common myth but completely overblown”:

@mathewi "virtually all of the network’s power and growth has come from outside the company itself"—a common myth but completely overblown

—
Evan Williams (@ev) September 06, 2012

At this point, Anil Dash — who used to work for blogging platform Six Apart and now has a media consulting firm — agreed with Williams that the focus on how much of a role third-party apps played in Twitter’s success is overstated:

@mathewi I think the "Twitter was made by third party things" is mostly nerd triumphalism, not factual. Shaq did more than any indie app.

—
Anil Dash (@anildash) September 06, 2012

Dash also noted that some of the elements we associate with Twitter — even hashtags, which Chris Messina (now of Google) was the first to use on Twitter — were used in other ways on the internet before Twitter came along, and others noted that the @ symbol was also used on services such as Internet Relay Chat. Williams then pointed out that if it wasn’t for the company’s decision to incorporate and support those features, they would never have become part of Twitter to begin with.

@mathewi @anildash This is how products evolve. You have 1M ideas—some come from usage, some from inside. You pick and choose carefully.

—
Evan Williams (@ev) September 06, 2012

I tried to argue that the point wasn’t to try and determine which played a larger role, the ecosystem or Twitter itself and the decisions it made (some of which irritated users, such as the decision to implement retweets in a different way). The point for me is that the relationship between users — and third-party services — and Twitter has always been much more symbiotic than it has a traditional company-user dynamic. And a big part of that was a wide-open API that let tweets flow wherever they wanted to, something Twitter has been busy shutting down.

Ethan Kaplan, a developer who is a vice-president at Live Nation and used to work for Warner Brothers Records, put it well when he said that all developers really want is for Twitter to admit the relationship is symbiotic, rather than parasitic:

@mathewi @chrismessina ultimately what we all want is twitter to stop treating their ecosystem as parasitic. It isn’t. It’s symbiotic.

—
Ethan Kaplan (@ethank) September 06, 2012

And Chris Messina — who noted that the third-party app Tweetie (which Twitter ultimately acquired and turned into the official iPhone app) was the first to support hashtags — said that one of the things the company has failed to do is to make it clear who it is making all of its recent changes for. As I’ve pointed out before, it argues that it is doing so for users, but is that really the case? I have to admit that I’m not convinced.

@ethank @mathewi is it that? It feels like there's not sufficient transparency behind the motivation for the changes. Who're they *for*?

—
Chris Messina™ (@chrismessina) September 06, 2012

I’ve embedded the full version of the Storify below, with as many of the comments as I could find (apologies to those whose contributions I missed). Interestingly enough, Twitter has said that the new API rules aren’t meant to apply to services like Storify, even though the company seems to fall into the wrong quadrant of customer product lead Michael Sippey’s by-now-infamous chart.

[View the story "Evan Williams on Twitter and its ecosystem" on Storify]

Evan Williams on Twitter and its ecosystem

Storified by Mathew Ingram · Thu, Sep 06 2012 15:45:46

The Tumblr news didn’t come as that much of a surprise to anyone that has been following recent events, since Twitter has already cut off other apps such as Instagram. In fact, Matt Buchanan at Buzzfeed wrote a post on Wednesday about how the blog network could be the next target for Twitter and within a matter of hours Tumblr lost the ability to connect to Twitter.

In the case of Instagram, Twitter’s removal of those connection rights — which allowed users to find and connect with any Twitter followers who also use the photo-sharing app — seemed as though it might have been driven in part by a desire to play hardball with Instagram’s new owner, Facebook. But Tumblr isn’t owned by a competitor: If anything, the blog network has been a close partner of Twitter’s, up to and including building in support for the newly introduced “Twitter cards” that show expanded information about tweets.

Twitter cut Tumblr off even though it is a partner

Despite those ties, Twitter decided to shut off Tumblr’s ability to show users their Twitter friends, a decision that even one Twitter engineer apparently doesn’t agree with. And Tumblr was clearly disappointed by the move, saying in a statement delivered to a number of blogs:

“To our dismay, Twitter has restricted our users’ ability to “Find Twitter Friends” on Tumblr. Given our history of embracing their platform, this is especially upsetting. Our syndication feature is responsible for hundreds of millions of tweets, and we eagerly enabled Twitter Cards across 70 million blogs and 30 billion posts as one of Twitter’s first partners… We are truly disappointed by Twitter’s decision.”

The key to the move, and to the similar action taken against Instagram, is contained in the only comment that Twitter has so far made publicly about either decision — after cutting off access to the friend-finder ability on Instagram, the company said simply that: “We understand that there’s great value associated with Twitter’s follow graph data, and we can confirm that it is no longer available within Instagram.” Twitter pointed to this statement after the Tumblr decision as well, saying it had nothing to add.

As designer and developer Dustin Curtis of Svbtle described in a post about Twitter’s recent behavior, a huge amount of Twitter’s value to both users and external services is tied up in its follower graph — that is, the index of all a user’s friends and connections, which in turn are a direct representation of their interests. That “interest graph” is what gives Twitter any power it might have to target advertising, to customize search results, to promote tweets, and all the other things it is trying to do in order to monetize its platform and justify its market value.

Unlike Facebook, where users are normally connected with their friends via other means, that interest graph represents all the power that Twitter has over a user. So it’s not surprising the company would want to control that feature as closely as possible, and even turn it into a monetization strategy (it’s not clear whether Twitter asked Tumblr to pay for access or whether it was just removed). The only question now is whether apps like Flipboard — which also had close ties to Twitter, until CEO Mike McCue left the board of directors — will suffer the same fate.

The future is driving eyeballs to television programs

As it cuts off the third-party developers (and in some cases users) who helped generate much of its success, signs of where Twitter is headed are also abundant: They can be seen in the deals that the company has signed with corporate partners such as NBC, which led to an official Twitter hub where curated information about the Olympics appeared — and also caused a significant amount of controversy when the company suspended the account of a journalist who was critical of NBC. In a comment about the partnership, Twitter’s VP for media, Chloe Sladden, said:

“It’s not fair to describe Twitter as a spoiler mechanism. What we saw is that it was an amazing daytime teaser trailer driving people into prime time.”

In other words, despite the critical protests of users who popularized the hashtag #NBCfail — because of the TV network’s decision to post tweets about events that weren’t going to be broadcast in the U.S. for hours — and despite the fact that the NBC deal forced Twitter to geo-block anyone outside the U.S. from seeing anything on its official hub, the company was more than happy with the relationship because it drove lots of people to watch television. As media relationships become a bigger part of Twitter’s future, which they will almost certainly do, that kind of argument is going to define the company’s vision of success.

The only question that remains is whether enough users want Twitter to become that kind of media entity, with all the controls and restrictions and advertising messages that come with it. It’s possible that — as some have argued — the third-party developers who are complaining about their treatment by the company are no longer relevant, and that those users who have been supporting alternatives like App.net are simply misguided. Or Twitter may have miscalculated badly, and sealed its fate as yet another media entity scrambling to promote its ads to a declining user base, just as MySpace did.

Post and thumbnail images courtesy of Flickr user Rosaura Ochoa

Beneath all the sound and fury from developers, however, is a kernel of truth that Twitter would do well to consider: namely, that one of the reasons why external apps and services have been — and continue to be — such an important part of the company’s growth and success is that many of its own products are frequently underwhelming at best. If the point of the new API changes is to control more of the ecosystem and the Twitter experience, then the company had better make sure that experience is as good as it can possibly be, or it risks losing the very user base it is hoping to monetize, as others have in the past.

As Harry McCracken notes in a post at Techland, the description of the changes from consumer product lead Michael Sippey does a pretty poor job of explaining what kind of behavior Twitter is in favor of and what kind it isn’t, and it doesn’t really give users any kind of guidance at all when it comes to which apps or services they should feel comfortable using. The confusing table included in the post — with quadrants for different apps and abstract descriptions rather than names — obscured a lot more than it revealed, as highlighted by the fact that many people couldn’t tell whether Storify was one of the “good” apps or one of the bad ones and the director of platform Ryan Sarver was forced to try to clarify that with a tweet.

The new rules have their defenders, including some who argue that Twitter is at least providing some firm guidance for developers, since its attitude toward third-party apps and services has been the subject of a lot of fear and uncertainty. Others have made the point that placing limits on API use makes perfect sense for a company that is trying to generate revenue from its network, as opposed to giving every developer with an app a free ride, and that the limits are not onerous (although Bottlenose founder Nova Spivack argues that Twitter could actually make as much or more money by licensing the use of its API).

But while the limits on API use and the requirements for how Twitter can be used may not look extreme, the message behind them seems to be clear. As entrepreneur and venture investor Chris Dixon put it, ”If you make a Twitter client, you should stop and make something else.” Instapaper developer Marco Arment has a similar view of the changes, saying they are obviously designed to make it difficult for other services to make use of what Twitter sees as its core functionality, to the point where one clause about how tweets must be displayed even appears to threaten popular aggregation apps like Flipboard. As Arment put it:

“Apps cannot interleave chronological groups of Twitter posts with anything else. This is very broad and will bite more services and apps than you may expect. It’s probably the clause that caused the dispute with LinkedIn, and why Flipboard CEO Mike McCue just left Twitter’s board.”

Squashing third-party apps means pain for users

The fact that Flipboard and Tweetbot — a popular mobile client — and possibly even services like Storify are threatened by Twitter’s moves highlights an important point: The company claims that these changes are being made to provide a “consistent user experience,” implying that all it really wants is to save users from irritating or poorly designed services. But the reason why people use apps and services like Flipboard, Tweetbot and Hootsuite in the first place is that they provide something useful that Twitter doesn’t. How does throttling or even extinguishing those kinds of apps help users? Just like the decision to pull tweets out of Google search, users are the ones who ultimately seem to pay for these kinds of moves.

One of the things that makes Tweetbot appealing as a mobile Twitter client, at least to me, is that it is consistently faster and better designed than the official mobile app and has a number of useful features that Twitter’s app doesn’t. As more than one person has pointed out, the company’s mobile web app and even its regular website also leave a lot to be desired in terms of usability, and the iPad app and Mac OS X apps appear to be the redheaded stepchildren of the family: They get few (if any) updates, and in some cases they may not even meet Twitter’s new display and usage guidelines.

It’s true that relatively few people use third-party apps, and so some have argued the developer angst and outcry isn’t worth paying attention to. But if this rationale is taken far enough, it turns into a kind of “we can do whatever we want, and users will have to put up with it” attitude, and that could be very dangerous indeed. As I have argued before, MySpace and Digg are a examples of companies that put the demands of revenue generation and business models ahead of what their users wanted, and they paid the price. They may not have had third-party developers, but the outcome was the same.

In a debate with John Gruber of Daring Fireball, who says Twitter is effectively telling developers to “drop dead,” Anil Dash argued that the company’s restrictions aren’t that different from what Apple has done with its app store and developer community. But unlike Twitter, Apple had a successful and attractive platform that developers were clamoring for access to. The platform Twitter is now trying to monetize would not have achieved much of its value if it wasn’t for the developers it is now spurning. Will it have the same value if they leave?

Furthermore, Apple’s focus has also always been on users and the user experience, and its requirements for developers — however draconian they seemed – have stemmed from that impulse. Twitter wants to portray its changes and restrictions in the same way, but it is a much harder argument to buy. It feels as though the company’s need to justify its $8 billion market value is taking precedence over everything else, and developers — and users — are getting caught in the crossfire.

Post and thumbnail images courtesy of Flickr users Mark Strozier and Rosaura Ochoa

Twitter sent some shock waves through the technology community with a blog post on Friday that talked about its plans for the future, and suggested that those plans don’t necessarily involve third-party services and apps. Although the company phrased its statement as a move designed to standardize the experience for Twitter users, developers and others in the broader Twitter ecosystem clearly took the post as a warning shot across the bow — especially since the company simultaneously shut down a cross-posting partnership it had with LinkedIn . It seems clear that Twitter wants to control the network as tightly as possible so that it can monetize it more easily, but doing so also comes with substantial risks.

In his blog post, consumer product manager Michael Sippey talked a lot about the introduction of features such as “expanded tweets,” which show more information from providers like GigaOM and the New York Times when a link is included in a tweet. He said the company wants to broaden that program to more publishers, as well as giving them tools to display expanded tweets and other features on their sites — but he also made it obvious that developers who stray outside of the lines are taking a big risk:

[W]e’ve already begun to more thoroughly enforce our Developer Rules of the Road with partners, for example with branding, and in the coming weeks, we will be introducing stricter guidelines around how the Twitter API is used.

Twitter has burned the ecosystem before

These comments set off warning bells for a number of developers, who said they were concerned that Twitter was going to crack down on any third-party app or service. One developer on Hacker News said that in his view, Twitter was trying to shut down third-party services so that they could “inflict a homogenized, boring, monoculture on their user base [that] they can monetize, which will make the experience progressively worse.” Said Turntable.fm developer Jonathan Kupferman:

Twitter seems to be mercilessly killing all developer apps of any interest businessinsider.com/twitter-linked… Light the match, hello #burningplatform

—
Jonathan Kupferman (@jkupferman) June 29, 2012

This isn’t the first time that Twitter has upset the developer community by throwing its weight around. In 2011, there was widespread criticism of the service for the way it issued new rules around use of the Twitter API — and also the way it behaved towards those who crossed the line by shutting off their access without even a warning, as it did in the case of entrepreneur Bill Gross and his Ubermedia network. At the time, one critic accused the company of “nuking” the Twitter ecosystem.

The company also came under fire in 2010 for the way it handled relations with third-party developers after it bought an app called Tweetie. Hunch founder Chris Dixon said Twitter was “acting like a drunk guy with an Uzi” by telling developers not to bother developing Twitter apps, and a number of companies and investors that had been putting money and time into the Twitter ecosystem stopped doing so. So some of the negative reaction to Sippey’s post stems from being burned twice already.

Some observers have argued that Twitter is just doing what it has to do in order to control its network and build a sustainable business, and that third-party developers don’t have any right to expect favorable treatment, since they are piggybacking on its API and resources. Longtime Twitter users, however, say the service’s behavior is a betrayal of all of the other services and apps that helped generate most of the goodwill it is now busy monetizing. As John Abell of Reuters pointed out on Friday, much of the value that users find in Twitter comes from the way it connects to other services.

Twitter's value is its integration with other networks. Cutting them off is like being on the wrong side of history. allthingsd.com/20120629/twitt…

—
John C Abell (@johncabell) June 30, 2012

Anti-user moves torpedoed both MySpace and Digg

And there is a very real risk to this kind of aggressive focus on control and monetization, as a commenter on Hacker News pointed out: restricting the ways that users can access and display their tweets, whether through strict API rules or moves like the LinkedIn shutdown, could irritate the user base that Twitter is relying on to click ads and do all the other things it is planning around monetization. Ultimately, the company could ruin the experience that made Twitter so compelling in the first place, in the same way that MySpace and Digg did.

There are plenty of reasons why MySpace failed, including the conflicting desires of a giant corporate owner like News Corp., but it also started to hemorrhage users because it focused more on monetization through ads and other elements than it did on maintaining a good experience for users. Digg did something similar — in an attempt to build a bigger company and leverage its user base for profit, it added a whole range of “services” and features that were designed mainly to appeal to corporate customers and advertisers. The end result was a wholesale desertion of Digg for other communities like Reddit.

Twitter has a tiger by the tail — it has an active user base in the hundreds of millions, it has become an almost indispensable tool for both news junkies and the media (although this carries risks as well) and it is starting to see some favorable responses to its ad model. But it is also a community, where the users provide the vast majority of the content that is being monetized, and while screwing around with that relationship may appear to make short-term financial sense, it could end in disaster.

Post and thumbnail images courtesy of Flickr users Rosaura Ochoa and See-ming Lee

“Operators treat partners like vendors.” That quote comes form Google director of global Android partnerships John Lagerling, who said it during a panel discussion this week at Management World in Dublin, where it was captured by Light Reading. It’s a telling statement, especially coming from one of the most important players in the mobile industry.

Several years ago, operators could have gotten away with treating everyone else in the mobile ecosystem as a mere supplier. After all, they ran the whole show. If your service, device or technology was ever going to make it onto an operator’s network, you had to kiss the ring. A lot has changed in a few years.

These days, Google and Apple have far more clout in the global wireless industry than any individual carrier. Apple’s decisions on which radio technologies to support can cause seismic shifts in the industry. And any new iteration of the Android OS can spur new waves of new service innovation and app development. That’s power that no single operator has ever held.

The mobile industry has evolved, but if Lagerling is speaking the truth, then the carriers haven’t evolved with it. That Bellhead, old-school telco mindset was in evidence at the U.S. mobile industry’s biggest show, CTIA Wireless. The show has been suffering for several years and I believe the main cause for its slow demise is its stubborn focus on carriers. The show simply hasn’t embraced the new reality.

To be fair, the disdain runs both ways. Most over-the-top (OTT) app developers view the operators as mere dumb pipes, which isn’t going to ingratiate with them Verizon and AT&T anytime soon. I’ve heard developers lament the loss of unlimited data plans like old hippies pining for the Summer of Love. Mobile carriers may be treating developers and platform providers like mere vendors supplying the nuts and bolts of their services, but the developers are treating operators like mere utilities into which they plug their apps.

The thing is carriers need developers a lot more than developers need the carriers. And the carriers know it. Wireless analyst Chetan Sharma sums up their conundrum quite eloquently in his latest U.S. market update:

We are at a critical juncture of the industry evolution. The OTT phenomenon is shifting the tectonic plates at a rapid pace. What seemed like a minor irritant only a few quarters back is become a nuisance virus that is eating away the core. Some operators have gone into panic mode while others have stepped back, assessed the situation, embraced it, and will try to exploit the opportunity. The truth of the matter is that the two biggest apps – voice and messaging didn’t really evolve a period of two decades. When the last big invention was interoperability and that too a decade ago, you know things are ripe for disruption. Thanks to the availability of always-on IP networks, new and nimble players are pushing the boundaries of what’s possible.

Operators are now floating the idea of OTT providers fronting the costs of delivering their traffic over mobile broadband networks. (We may even see the first instance of it in the coming weeks from Verizon.) The operators are talking about revenue sharing, and “sharing” implies a partnership of equals.

Research presented by VisionMobile today at the Open Mobile Summit found that while Android and Apple’s iOS are currently the most-used platforms by developers — at 67 percent and 59 percent respectively — the mobile web is not that far behind, with 57 percent developing for the cross-platform platform. That represents a leap of some 16 percent for the mobile web over last year:

And what about when the question gets turned around, to ask what platforms are getting abandoned in the future by developers?

Unsurprisingly, the list is led by Symbian, which Nokia (NYSE: NOK) is outsourcing to Accenture and has promised it would only continue to support until 2016 (and of course that might get revised again). Other “losers” include WAC — the platform created by a consortium of operators but yet to yield any actual products. Java is getting less attention, too, as feature phones become less popular; as is PalmOS as HP (NYSE: HPQ) puts more effort into its new WebOS — although that, too, comes up on the list (sorry, HP):

The growing interest in, and activity on, the mobile web is being borne out by lots of developments of late on the platform. High profile web app launches from the likes of Twitter and the FT have given the platform a new look from consumers.

And although the mobile web is still limited in its scope compared to the capabilities of native apps, that appears to be slowly changing: Gavin Stirrat, MD at Millennial Media, noted how the latest release of Safari for iOS lets developers incorporate the use of the iPhone’s accelerometer into ad content. Features like this will ultimately make the switch from the two platforms more seamless, and significantly more attractive for developers.