Data.gov - a new toy for data geeks, librarians, and transparency advocates everywhere

May 21, 2009 7:46 PM

Data.gov is live!

The Obama Administration is trying to bring transparency and a little sunshine to those cold, hidden government databases, like the FBI, NOAA, and the Census Bureau.

Data.gov is getting some mixed reviews. There’s not a lot of data in it yet, although you can suggest your favorite federal database (you do have a favorite federal database, right?) It may not be any better than what private citizens have already done.

The Sunlight Foundation has already announced a competition to build the best application and visualization for the data.

What can you find?

posted by gingerbeer (39 comments total)

If cortex likes this kind of data as much as he likes data about Metafilter, he's gonna swoon.
posted by ocherdraco at 7:51 PM on May 21, 2009

How about "expense payments for British MP's @ Disney"?
posted by R. Mutt at 7:51 PM on May 21, 2009

I hit post a little too fast and forgot the link to the official announcement.
posted by gingerbeer at 7:53 PM on May 21, 2009

Yey!
posted by leotrotsky at 7:54 PM on May 21, 2009

There’s not a lot of data in it yet

Only 47 databases, many of which are very specialized. Still, it's a start. That usgovxml site is pretty neat-o.
posted by Blazecock Pileon at 7:54 PM on May 21, 2009

Wow, that's cool. They only have 47 data sets so far and they're kind of random. Not that I haven't always wanted to know the locations of All the world's copper smelters, but without having lots of other industries it's not that helpful. Unless you've got some copper that needs smelting I guess.

But yeah this could potentially be very cool.
posted by delmoi at 7:56 PM on May 21, 2009

yes.
posted by Salvor Hardin at 7:56 PM on May 21, 2009

Mighty fine web design on all these new government sites.
posted by Garak at 7:57 PM on May 21, 2009 [3 favorites]

Yeah, I am sort of imagining cortex donning one of those special swimsuits and then just diving in and swimming about like a dolphin. Then, after he blasts out a ring of analysis from his bloghole, he bites into it and just for a second ... it looks a little bit like a giant donut.
posted by adipocere at 7:59 PM on May 21, 2009 [9 favorites]

delmoi: Not that I haven't always wanted to know the locations of All the world's copper smelters

I thought I was the only one!
posted by Kattullus at 8:09 PM on May 21, 2009

I hope we find some really juicy and irrelevant correlations. I wish I remembered how to use SPSS.
posted by bxyldy at 8:12 PM on May 21, 2009

Looks pretty. Let's see if it works.
posted by blucevalo at 8:29 PM on May 21, 2009

They even use recaptcha! I'm impressed.
posted by Lemurrhea at 8:45 PM on May 21, 2009 [1 favorite]

This is so fucking cool. I rarely am so excited about running regression analysis. Ok, maybe I'm always a little excited about running regression analysis. Anyway, they're using recaptcha!
posted by The White Hat at 8:46 PM on May 21, 2009 [3 favorites]

This is going to be such a gigantic timesuck when there are more datasets.
posted by rtha at 8:50 PM on May 21, 2009

I can't even wrap my head around how unbelievably awesome and world-changing this could, and will be. When the availability, tools, formats and uses for this start getting realized, we are going to see a huge shift in not only what we can accomplish, but our relationship with information, intellectual property, and collaborative research. This is relevant, connected, and the start of something phenomenal.
posted by iamkimiam at 9:04 PM on May 21, 2009 [2 favorites]

Just for kicks, it could be neat to do a Metafilter competition/collaboration project with these sets. Maybe users pick a set, and find a way to use or reinterpret the data through your own field/lens. For example, i'd be interested in seeing how an artist, engineer, biologist, linguist, statistician, economist, teacher, and realtor would approach the Active Mines and Mineral Plants in the US set.
posted by iamkimiam at 9:14 PM on May 21, 2009

It's like I've died and gone to geographical copper smelting distribution heaven.
posted by cortex at 9:16 PM on May 21, 2009 [13 favorites]

There are some NIH/NSF research databases I'd love to get on there. But that will never happen.
posted by dw at 9:25 PM on May 21, 2009

I wish I remembered how to use SPSS.

You think you've got problems? The last time I used it, it was running on a DEC20 via batch files.

I recently tried to run it under Windows and was completely bemused by the point and click aspects.
posted by PeterMcDermott at 9:26 PM on May 21, 2009

If they put the trade data from the census bureau and the department of commerce on there, I will be in heaven...
posted by gemmy at 9:37 PM on May 21, 2009

When does it become sentient?
posted by mazola at 9:45 PM on May 21, 2009

This will be very cool, eventually.
posted by schyler523 at 9:57 PM on May 21, 2009

From the FAQ:

What is metadata?
Metadata is "data about data."

Dada!
posted by humannaire at 10:02 PM on May 21, 2009 [1 favorite]

Metadata.
posted by mazola at 10:15 PM on May 21, 2009 [3 favorites]

So we feed this into Wolfdog|Alpha and index it with googlebots and feed and cross-reference the index right back into Wolfshirt|Woofwoof with crosslinks to wikipedia and spam blacklists and the next thing we know we're being hunted down by creepy fifty foot tall spiderbots with a concise knowledge of our shopping preferences and population demographics, a scary taste for showing off it's appetite for crunching complicated math as well as a murderous knowledge of physics and a rather snarky sick sense of humor.

Be right back. I need to finish hacking this blu-ray laser diode into a pumped and doped secondary crystal. Gonna need a laser blaster real soon now.
posted by loquacious at 10:27 PM on May 21, 2009 [9 favorites]

This has the potential to rock my world, so am very, very excited to see this post.

Here's hoping that government in-fighting doesn't kill this one.

Just for perspective, this is what data from the government used to look like--and that site is still live.
posted by librarylis at 10:41 PM on May 21, 2009

The fact that they're offering ESRI data is what excites me. If only Australia would offer a simple portal for GIS data. Think of all the useless but pretty maps I could make!

Even if they were all about copper smelters.
posted by Serial Killer Slumber Party at 11:04 PM on May 21, 2009

Vivek Kundra spoke at a government webaster conference recently. He wants to do away with traditional government websites and just have views into the data. Anything we can do to get this information out of the hands of the communications offices of individual government agencies is a step in the right direction.
posted by HumanComplex at 6:39 AM on May 22, 2009

I'd love to see a database listing of the DHS No-Fly list, cross referenced with the number of people prevented from flying based on the list, with annotation as to how many of them were actual "terror suspects". Something tells me the overlap is astoundingly small.
posted by FatherDagon at 9:39 AM on May 22, 2009

The National Map is pretty sweet.
posted by slogger at 11:18 AM on May 22, 2009

Category: All Categories
Department: Department of Homeland Security

No Matching Records Found
posted by 256 at 12:42 PM on May 22, 2009

yo wolfdawg I heard you like metadata so I put data in your data so you can alpha while you beta
posted by speicus at 12:50 PM on May 22, 2009 [2 favorites]

So we feed this into Wolfdog|Alpha and index it with googlebots and feed and cross-reference the index right back into Wolfshirt|Woofwoof.....

I too think it's great that the MarkovFilter is back.
posted by inigo2 at 12:56 PM on May 22, 2009

I imagine the DHS No-Fly list and others are not going online any time soon, if ever. My wife's maiden name always flagged something in some system, requiring the airport staff to make a call or two to be made before they could issue her tickets. We were never told why, and it's not an issue now that she's Mrs. filthy light thief, so we don't think of it much any more.
posted by filthy light thief at 2:37 PM on May 22, 2009

To be fair, she is neé Bombdaplane.
posted by klangklangston at 3:20 PM on May 22, 2009

The National Map is pretty sweet.

Yeah, but having worked with ArcIMS, it could be a lot better. They didn't spend much time making the interface user friendly. If you want to see what I mean, click on the "identify" tool on the left and then click on the map.
posted by desjardins at 2:14 PM on May 24, 2009

Meant to italicize the comment to which I was referring.
posted by desjardins at 2:14 PM on May 24, 2009

I like the national map myself
posted by michaelehline at 10:14 PM on May 26, 2009