The Control Revolution And Its Discontents
- "the long process of algorithmisation
over the last 150 years has also, wherever possible, replaced implicit rules/contracts and principal-agent relationships with explicit processes and rules."
(625 kb zip file) "It's a treasure trove for researchers that I'm sure is unmatched in the world of modern polling:
[Daily Kos has] assembled all the raw data for every single Daily Kos/SEIU poll conducted in 2011 into a single file. That's 46 polls, including questionnaires ... in a nifty 623 KB package. No one else releases information this granular, so if you've ever wanted to take a deep, deep dive into raw polling data, this is your chance."
Artist and film-maker
, Hito Steyerl
, asks us to stand shoulder to shoulder with our digital equivalents. Digital images are Things (like you and me)
- a plethora of compressed, corrupted representations
pushed and pulled through increasingly policed and capitalised information networks. If 80% of all internet traffic*
- a liberated excess withdrawn**
from accepted channels of communication - perhaps it is in The Poor Image
we find our closest kin? [more inside]
- providing tools for creating interactive data visualizations for the web
are the most valuable contributors to communities because they contribute the highest quantity and quality of comments." As anonymous
online contributors struggle to remain non-identifiable
, Disqus data show pseudonymous commenters are the best
. (most recently previously)
"You feel euphoric you know. Because it's one of the best buzzes personally I've had in my life. Better than any drug. And you know it was just that....It was a feeling of standing up straight against an institution that's been historically has always been brutal, wicked and bad mind towards young people especially young black people."
In collaboration with the LSE, the Guardian's Reading the Riots
project has used a mixture of quantitative and qualitative methodologies to explore the causes of England's summer of disorder.
Mr. Data Converter
takes CSV, Excel, or tab-delimited data and coverts it into web-friendly formats, including HTML tables, PHP arrays, JSON properties and MySQL tables. via
The Music Ngram Viewer
from Peachnote tracks appearances of any given note or chord sequence in a corpus of 60,000 optically scanned public-domain classical scores, ranging from the 17th century to the present -- a la what Google Ngram Viewer
does for words and phrases. A fuller description with examples.
And if you don't like the Google-esque GUI, you can download the raw data
and mess with it yourself. (Via Music Hack Day Boston
The Billboard Wayback Machine
is an interactive that lets you explore the Billboard charts spanning from 1964 to 2011
In August 2011, 35 ACLU affiliates filed 381 requests in 32 states with local law enforcement agencies seeking to uncover when, why and how they are using cell phone location data to track Americans
. So how long do American cell phone carriers retain information about your calls, text messages, and data use? According to data gathered by the US Department of Justice, it can be as little as a few days or up to seven years, depending on your provider
. (Via / More)
is a visualization of complementary flavors
, sourced from patterns found in lots and lots of recipes. For those of us not blessed with a chef's instincts.
InForm: Turning Data into Meaning.
An exhibit at the Adobe Museum of Digital Media.
U.S. Poverty Rate, 1 in 6, at Highest Level in Years (NYT) -
An additional 2.6 million people slipped below
the poverty line in 2010, census officials said, making 46.2 million people in poverty in the United States
, the highest number in the 52 years the Census Bureau has been tracking it, said Trudi Renwick, chief of the Poverty Statistic Branch
. That represented
15.1 percent of the country. The poverty line in 2010 was at $22,113 for a family of four
MST3kdbx: Six Degrees of Peter Graves.
Did you know Coleen Gray was in The Leech Woman and The Phantom Planet?
Like the IMDB obsessive cinephile friend you never friend MST3Kdbx indexes and connects together every actor in every movie shown on Mystery Science Theater 3000 [via mefi projects
A corpus analysis of rock harmony
[PDF] - The analyses were encoded using a recursive notation, similar to a context-free grammar, allowing repeating sections to be encoded succinctly. The aggregate data was then subjected to a variety of statistical analyses. We examined the frequency of different chords
and chord transitions ... Other results concern the frequency of different root motions, patterns of
co-occurrence between chords, and changes in harmonic practice across time.
More information, analysis, and explanation here
Every year since 2005, Nicholas Feltron has logged the progress of his life – his meals, locations, conversations, pets, travel, everything – in minute and exacting detail, summarizing his activities in what he calls "Annual Reports" featuring beautiful infographics
Last year, Feltron's father died. Rather than talking about himself for the 2010 Annual Report, Feltron memorialized the entire life of his father
The Harvard Study of Adult Development is the longest prospective study of mental and physical well-being ever conducted. For 72 years, researchers at Harvard have been following 824 individuals through war, career, marriage and divorce, parenthood and grandparenthood, and old age. Designer Laura Javier
took ten of those cases and visualized them in the Elements of Happiness
. [via flowingdata]
A Mismeasured Mismeaurement of Man.
Stephen Jay Gould's classic The Mismeasure of Man
argues that 19th century scientist Samuel George Morton
inflicted his own racial biases on his data
to demonstrate that Caucasians had larger brains than other races. A new paper in the Public Library of Science: Biology
debunks Gould's account by remeasuring the same skulls Morton used. Whatever biases Morton may have had, they are not reflected in the data
The U.S. Department of Housing and Urban Development has launched a new interactive mapping tool
for Community Planning and Development agencies, interested agency partners, and the public. [more inside]
Mining the Mother of all Data Dumps
We now have a relatively massive haul of digital data from the OBL strike. There are several forensic toolkits in use by the private (commercially available)
sector as well as open-source
. Best practices
include inventorying all the sources, cloning the sources so as to not damage pristine data, recovering any partial or damaged content, making the cloned sources read-only, adhering to legally-admissible tools standards, and documenting everything. There is an excellent source titled Digital Forensics and Born-Digital Content from the Council on Library and Information Resources [pdf
, Resource Shelf
]. But what to do next*? [more inside]
a new online magazine by Google UK. [more inside]
is the New York Times'
current Data Artist in Residence. He creates information-rich animations, most recently of the latest Kepler candidate extrasolar planets [previously]
; also a global render of people's uses
is a designer, artist, and biker: his work includes visualisations of Facebook breakups over the course of a year
and Hollywood box office revenues, 1986 - 2008
is an "information journalist"; his blog, InformationisBeautiful.net
, has been linked
on the blue
, but you might enjoy this overview of his work and others at TED
. Similarly, Hans Rosling, also mentioned previously
. [more inside]
Stanford's Visualization Group
has produced a data cleanup web app called Wrangler
that works like straight up magic
becoming a commonly deployed tactic by the London Met, students from the University College London are fighting back with Sukey
, launched this morning. [more inside]
give their hopes and dreams for data, data tools and data science
Already, Google has provided Google Refine
) to help clean your datasets. While great visualizations
can be created with online tools
or by combining R (great posts previously
), with ggplot2
, and even Google Motion Charts With R
(already built into Google Spreadsheets
Need data? Needlebase
, helps non-programmers scrape, harvest, merge, and data from the web. Or if you’re introspective, Your Flowing Data
provide tools to measure and chart details of your own life.
"They're not out to make a quick buck, they're looking to protect the integrity of the franchise and its mythology."
1998's Star Trek Insurrection
went through a number of different plots before becoming the film we ultimately saw. Starting out as Star Trek: Stardust
, the first take on the idea involved Captain Picard going all Heart of Darkness
on a former friend from his Starfleet Academy days in a bid to find the Fountain of Youth. That treatment evolved into a remarkably Avatar
ish story called simply Star Trek IX
in which Picard must go upriver to kill a malfunctioning Data as part of a Federation/Romulan alliance to displace strange alien natives from a planet teeming with a valuable and rare ore (spoiler: Picard actually kills Data in this treatment, and Tom Hanks was supposed to have a major role somewhere).
Let the late Michael Piller
guide you through the writing of Insurrection
in his unpublished book Fade In: The Making of Star Trek: Insurrection
(his "last great gift to the fans and to aspiring writers everywhere") in which he presents his original story treatments, story notes from his bosses at Paramount, surprisingly reasonable Trekker-type reactions from actors Patrick Stewart and Brent Spiner, and much more. First made freely available by TrekCore.com
, Piller's family has since asked that it be removed, but you'll still find the file roaming the Internet if you boldly go looking for it
. [more inside]
The New York Times presents an interactive map of America's population
separated by race, income, and education, according to census data from 2005 to 2009. One dot for every 50 people. (Previously
) [more inside]
Elizabeth Warren on setting up the Bureau of Consumer Financial Protection
- lecture starts here
, but really starts getting good here
: "I feel like this is a boring speech." stay for the Q&A
hosts competitions to glean information from massive data sets, a la the Netflix Prize
. Competitors can enter free, while companies with vast stores of impenetrable data pay Kaggle to outsource their difficulties to the world population of freelance data-miners. Kaggle contestants have already developed dozens of chess rating systems which outperform the Elo rating currently in use
, and identified genetic markers in HIV associated with a rise in viral load
. Right now, you can compete to forecast tourism statistics
or predict unknown edges in a social network
. Teachers who want to pit their students against each other can host a Kaggle contest free of charge
is World Statistics Day, so help yourself to a metric (haha sorry)
ton of publicly available data at UNdata
, ICSPR (registration required to download data sets)
, and data.gov
). You can also explore, visualize and animate a variety of publicly available data sets with Google Labs' Public Data Explorer
25 most dangerous neighborhoods 2010.
Click through the maps for some more specific data.
, Making sense of complex issues
through data and design
. Visualizing is a place to showcase your work, get feedback, ensure that your work is seen by lots of people and gets used by teachers, journalists, and conference organizers to help educate the public about various world issues.
A Tour through the Visualization Zoo
. A survey of powerful visualization techniques, from the obvious to the obscure.