The open spending data that isn’t

Posted: July 2nd, 2010 | Author: Chris Taggart | Filed under: Where Does My Money Go | Comments Off

The following guest post is from Chris Taggart of OpenlyLocal, who advises the Where Does My Money Go? project on local spending data, and is a member of the Open Knowledge Foundation’s Working Group on Open Government Data. This is a cross-post — Chris’ original post here.

When the coalition announced that councils would have to publish all spending over £500 by January next year, there’s been a palpable excitement in the open data and transparency community at the thought of what could be done with it (not least understanding and improving the balance of councils’ relationships with suppliers).

Secretary of State for Communities & Local Government Eric Pickles followed this up with a letter to councils saying, “I don’t expect everyone to do it right first time, but I do expect everyone to do it.” Great. Raw Data Now, in the words of Tim-Berners Lee.

Now, however, with barely the ink dry, the reality is looking not just a bit messy, a bit of a first attempt (which would be fine and understandable given the timescale), but Not Open At All.

As a member of the Local Public Data Panel, I’ve worked with other members and councils to draw up some clear and pragmatic draft guidelines for publishing the local spending data. We’ve had a great response in the comments and in conversations, and together with some lessons I did on importing the existing data, I think these will allow us to do a second draft soon.

One thing we weren’t explicit in that first draft – because we took it for granted – was that the data had to be open, and free for reuse by all. Equality of access by all is essential.

So I’ve been watching the activities of Spikes Cavell’s SpotlightOnSpend with some wariness and now those fears seem to have been borne out, as the company seems to set out not to consume the open data that councils are publishing, but to control this data.

The idea seems to be that councils should give Spikes Cavell privileged access to their detailed invoice information, which the company then adds to their proprietry and definitely non-open database, and then publishes an extract of this information on the SpotlightOnSpend website. Exactly what information they get, and under what terms isn’t disclosed anywhere.

The website’s got most of the buzzwords: transparency, accessible, efficiency. It’s even got a friendly .org.uk domain. If that’s not enough to convince councils, liberally sprinkled around the site is an apparent endorsement from the Secretary of State himself:

I’m really excited about the opportunities of transparency and it’s something this government is utterly committed to. spotlightonspend demonstrates that, when innovative businesses work with far-sighted public bodies, we can inform the public, reduce costs and improve democracy both locally and nationally.

Eric Pickles

Secretary of State

Communities and Local Government

However, when you go to the data and click on the download link this is what you get:

Note the “This data is for your personal use only” (not to mention the fact that the use of a captcha’ to screen out machines downloading the data means, er, you can’t use machines to automatically download the data, which is sort of the point of publishing the data in a machine-readable way).

Never mind, surely you can just head over to the council’s website and download the data from there? No chance. This is what you get on the Guildford website:

You can search and view this financial data using a new Spotlight on Spend national website. Just follow the link found in the offsite links section of this page.

What about Mole Valley Council:

This data is now available on the spotlight on spend website. You can look at categories and individual suppliers to see how much has been spent in each area or you can download all the data to see individual transactions.

But what about Windsor & Maidenhead, who are closely affiliated with the project, and who are publishing data on their website? Well, download the data from SpotlightOnSpend and it’s rather different from the published data. Different in that it is missing core data that is in W&M published data (e.g. categories), and that includes data that isn’t in the published data (e.g. data from 2008).

So the upshot seems to be this, councils hand over all their valuable financial data to a company which aggregates for its own purposes, and, er, doesn’t open up the data, shooting down all those goals of mashing up the data, using the community to analyse and undermining much of the good work that’s been done.

It’s worth linking here to the Open Knowledge Foundation’s draft guidelines on reporting of Government Finances (disclosure: I helped draw them up), of which the first point is ‘Make data openly available using an explicit license’. And let me be absolutely clear here: this is not open data, not a desirable approach, will not achieve the results of transparency or of equality of access, and is not good for the public sector.

I’m hoping this is a matter of councils and the Secretary of State not understanding the process and implications of giving this data to Spike Cavell on a privileged basis. If not, perhaps it could be the first test case for the newly setup of Public Sector Transparency Board to rule on.

Opening up government finances

Posted: June 3rd, 2010 | Author: Chris Taggart | Filed under: Where Does My Money Go | Comments Off

With a string of announcements this week and the COINS database (the UK’s biggest source of information on public spending) about to be released tomorrow, it’s an exciting time for open data in the UK at the moment! When I first played around with the idea of opening up the basics of local government data (which turned into OpenlyLocal), I never imagined I was entering an area that little more than a year later would become such an exciting area, combining two of the hottest online trends, open government data and local data.

But still, there’s a hell of a long way to go, and one of the areas where there’s furthest to travel, and most to do is finance, specifically where the money’s being spent, who it’s being spent with, and also where it comes from. As the old journo saw goes: follow the money.

I had my first taste of the problems when I took a pretty much unused (and locked) spreadsheet, the 2006-07 Local Spending Report, and over the course of a weekend, unlocked it cleaned it up, imported it into a database and allowed people to do what the spreadsheet didn’t — make comparisons on local spending across councils and in areas.

However, the information was fairly heavily aggregated, was for just one period, and didn’t allow comparison with other financial reports.

So at the last OKCon a month or so ago, I sat down with some of the good people from Where Does My Money Go to discuss in some general principles for presenting government finances as data, to allow it to be properly analysed, combined with other data, and follow the flow of money to and from all branches of government, central and local. Now, the first draft has been published:

http://www.opengovernmentdata.org/finances/

The hope was that we could establish some general principles that would be applicable not just to government finances in the UK, but also for other countries too. Some of the key points are:

Machine-readable — we need the information as data that we can do things with.
Fine enough granularity so we can understand what’s going on, both in terms of categories, time periods, and transactions of any sort of size.
Using standard IDs to allow definitive identification and matching of bodies, areas and categories.

Obviously we’d welcome comments from both the UK and other countries. It’s also worth noting that there are two overlapping but slightly different areas: the accounts, and the transactions. Ultimately if you have access to the transactions you can work out the accounts, but it may be worth teasing out the distinctions, particularly in light of the UK moves (see below).

At the same time as doing this, in the UK things have been moving on apace, with the new coalition government announcing that by January 2011 all spending by local government over £500 must be published, which in government terms is a blink of an eye.

In addition, Will Perrin and I, who both sit on the Local Public Data Panel, were also asked for advice by Camden Council, who in the best traditions of open data wanted to get on and release their data a lot earlier than this.

Within literally a few days, and with much helpful advice from many of the other Local Public Data Panel members, a first draft was done, and was published yesterday on data.gov.uk. This clearly is very much a UK document, is concerned with local spending, and is framed by the goal of publishing spending over £500. However, like the Open Knowledge Foundation document, it’s meant as a first draft, and a focal point for discussion and I’d encourage all, whether open data and transparency advocates, or those working in local government (including police, health authorities etc) to add their comments to this document too.

Analysing and visualising UK public spending

Where Does My Money Go?

The open spending data that isn’t

Opening up government finances

Where Does My Money Go?

Menu

Get Involved

Recent posts

Twitter: wdmmg