World Government Data

January 25, 2010 4:42 PM

Governments around the globe are opening up their data vaults allowing us to check out the numbers for ourselves. This is the Guardian’s gateway to that information. Search for government data here from the UK, USA, Australia and New Zealand — and look out for new countries and places as they are added. Read more about this on the Datablog.

They have an API available. Even though all of these government data sites have enormous quantities of data, they are not in the same formats. What the Guardian has done is put them into a unified form, meaning developers have the opportunity to write applications that compare data between different countries. If you want the data in Atom or JSON just change the "/search" to "/search.atom" or "/search.json" in the url.

Here are some sample visualizations that have already been created.

posted by netbros (13 comments total)

I particularly like the US's data on whose phone calls they've been surveilling without warrants.
posted by ZenMasterThis at 5:34 PM on January 25, 2010 [1 favorite]

I like this. Very nice link. I had heard they were going to do this awhile back. Definitely want to look into that API feed. wonder what their content sydication prices are like
posted by rateit at 5:44 PM on January 25, 2010

How long until people start using these to stalk individual people? There's this illusion that statistics are anonymous, that gets shattered pretty quick once someone uses something like "income by race by zip code" to find out how much the only Pacific Islander family in small-town Nebraska makes...
posted by qvantamon at 5:56 PM on January 25, 2010

The times did some wrapping of government data into a more useful api as well. Their congress api, for example. Detailed on the Open blog which looks pretty interesting, actually.
posted by shothotbot at 6:04 PM on January 25, 2010 [1 favorite]

...unless you're asking about David Kelly.
posted by mullingitover at 6:07 PM on January 25, 2010

The cool thing about this is how it opens up more chances for citizens to participate in government beyond just voting every so often. I'd love to see a google map of planned construction, schools vs. property values, that sort of thing.
posted by Mr. Gunn at 6:11 PM on January 25, 2010

I particularly like the US's data on whose phone calls they've been surveilling without warrants.

Where do you see that...or were you trying to derail asap?
posted by hal_c_on at 6:46 PM on January 25, 2010

No one bit on his troll so it's best to just ignore him now.
posted by smackfu at 6:52 PM on January 25, 2010

Part of me is excited about this. This information is ours we paid for it, just wait for the mashups.

Another part of me snarks goes "Wow, it's like being able to do you own cack handed analysis of random old data without any statistical training! Look! The presence of hospitals *causes* deaths!"
posted by Damienmce at 7:11 PM on January 25, 2010 [4 favorites]

qvantamon:

The Census Bureau censors any data that might lead to identification, in your case they will fudge the numbers so that is impossible to find that ONE individual or business.

It's also why detailed statistics aren't available in very small areas.
posted by stratastar at 7:49 PM on January 25, 2010

Featured Australian dataset? Find a public barbecue!

Thanks for the stereotype Guardian (NOT sarcastic).
posted by wilful at 9:29 PM on January 25, 2010

I think I just heard David McCandless explode...
posted by Molesome at 4:07 AM on January 26, 2010 [1 favorite]

Here's the thing: I'm most definitely excited to have more access to the data, but I'd really like the governments to better centralize and standardize their data (though I realize the Guardian is trying to remedy some of that) before the Guardian scrapes it.

Here's an example of what I mean: a student walked up to the reference desk and asked for national health expenditure statistics for Canada and the USA. OK, for Canada it's fairly easy (CIHI) though I'm sure there's also other sources. For the US, though, I went to GAO, NIH, HHS, and finally found the information at the CDC (the CDC! Not my first thought, frankly for national health spending stats) and I certainly could have gone to other sources (data.gov, FedStats, even the BLS). That's way, way too many agencies and websites to be practical. I'd like to see data.gov succeed in its mission (to unify all these distinct data streams) so that the Guardian's new thing can succeed in its mission (to perform that unification on an international level).

They look like they're currently pulling all of their US federal data from data.gov, which explains their low numbers for the US (well, there's a lot of data there and some of it is from the state level but being the US government, there's still quite a lot of federal data to grab). In any case, a great start but there's so much to build on that I wonder how long their enthusiasm (and money) will last.
posted by librarylis at 4:16 PM on January 26, 2010