Won’t get fooled again

Some (actually many) years ago I was working as an Information Manager at the Audit Commission.  I became increasingly frustrated by the difficulty of joining up data across, the organisation.  There were loads of opportunities to add more value to what we were doing by taking data collected in different contexts and telling stories based on that data.  As an organisation, that's what we were really good at doing, telling stories based on data but we also valued innovation very highly and the key way that people saw they could innovate was in data collection and analysis.  This was also at a time when people were  getting used to the idea that they could use computers to do things that just hadn't been practical before.  In addition, managers were given a great deal of freedom to procure small-scale software to collect, collate and analyse data in interesting (but closed) ways – the priority in a study was for auditors to be able to create value for clients by telling them something useful, anything that achieved that was more important than any wider or longer-term use of the data.

I ended up boring all around me with two regular rants.  One was about agreeing on standards for data collection and ecosystem-wide metadata, the other was about thinking about horizontal standardisation rather than vertical integration when developing new software, to separate out how we dealt with each of the   Data, Application and Presentation layers and to ensure that when we developed or procured something new, we could easily get the data out to reuse later.  The hardest thing was getting people to understand that you could think about these things separately.  They found it really difficult to think about data independently of the process for collecting it, analysing and presenting it.  For a long time, people kept designing things where the data was locked in.  I drew a venn diagram that showed all of the applications in current and recent use and pointed out that all our procurement was either partially duplicating something that was already there or else filling in ever-tinier niche gaps between existing applications.  My point was that if we standardised our data management, we could focus attention on what we did with the data and re-use existing date to save some effort re-inventing the wheel.

I think that those of us who tell stories on the web using data are at a similar stage now.  I think that means all of us who are writing or making media on the web, but also anyone who is using social tools.  The $1bn sale of Instagram to Facebook today has been another wake-up call.  My own Instagram feed is only a few days old, but I'm following people who have been there for a while and see almost universal disappointment among long-term users and a desire to find something else that isn't currently tainted by Facebook.

But shifting over to another service that still holds onto your content and helps you to manage your social graph isn't going to work for long.  The lesson from Instagram is that if a service is any good it will get swallowed up by one of the big boys (probably Facebook) and if it isn't any good, well, you probably won't want to be using it.  The time you'll have using the next platform will just get shorter and shorter as the bubble inflates – how many people do you think today have thought "Blimey, $1bn shared between 10 people in less than two years? I'll have a piece of that!"  The value to me of the services provided has slipped below the break-even point.  As long as I've felt I'm getting a good deal, I've been relatively happy to be a hamster in my cage.  But I feel the prospects of getting a good deal from vertically integrated social applications is getting slimmer and slimmer. And every time we switch cages, we have to leave something behind. In Instagram for example, you may be able to extract your photos, but what about the comments and lists of likes?  No matter what the promises about being able to get at "your content" most tech products don't see it as yours, it's theirs and it's their route either to revenue generation or to selling out to someone with deep pockets.  

I don't think it's enough to say that I'm just not going to use Product X.  We need to build a different model, one in which we hold and manage our own data stores and we have a choice of tools that we use to share that data with the world.  I would rather pay money for an honest service that simply processed my data in interesting, useful and innovative ways than to pay for my experience by losing control over the things that I make and say and the online relationships that I have with people.

Originally posted on Lloyd’s posterous