60 posts tagged with BigData.
Displaying 1 through 50 of 60. Subscribe:

Wins above replacement ozeki

538 crunches the numbers behind 255 years of professional sumo tournaments.
posted by Chrysostom on May 13, 2016 - 12 comments

“Me gustaría un dato por favor - extra grande, con papeles.”

The BBC: “Panama Papers affair widens as database goes online”
The ICIJ has made the Panama Papers data available for download. It is a Neo4j database, but has been saved as CSV files so most anyone should be able to mess with it, provided they have enough disk space.
posted by Going To Maine on May 9, 2016 - 21 comments

"or have otherwise obtained"

Data broker Acxiom has released a tool About The Data to let you see (some of) the data they have and sell on you. [more inside]
posted by the man of twists and turns on Feb 27, 2016 - 21 comments

Your Code in Spaaace!

In the ISS there are two Astro Pi computers, Ed and Izzy, equipped with Sense HATs, two different camera modules (visual and IR), and stored in rather special cases. They are now running code written by UK school children - the winners of a competition. The data will be feeding back soon! [more inside]
posted by Stark on Feb 5, 2016 - 3 comments

quantified sneezes

Thomas Blomseth tracked his sneezes for five years, over 60,000 of them, and may have ended his pollen allergy
posted by the man of twists and turns on Jan 12, 2016 - 13 comments

As if we all have the same online experience

One day Harvard professor Latanya Sweeney googled herself with a reporter friend sitting next to her. An ad popped up inquiring about her arrest record. She had never been arrested. "It must be because you have one of those Black Names!" the friend said. "That's impossible," she replied, "Computers can't be racist." But then she started doing research. [more inside]
posted by Potomac Avenue on Dec 22, 2015 - 60 comments

Love In The Age Of Big Data

You might expect love to be the last frontier breached by data. It is the Antarctic of the human experience, richly feeding the oceans of our emotions, yet somehow remaining elusive and unknown. Philosophers have argued over it for millennia without arriving at a satisfactory definition. Poets like Erich Fried capture its strange mix of pleasure and pain, the sense of its essential ungovernability: “It is foolish, says caution / It is impossible, says experience / It is what it is, says love.” [slhuffpo]
posted by ellieBOA on Dec 21, 2015 - 12 comments

The Empire Strikes Back

Thursday was a banner day for Bernie Sanders, whose campaign reached two million donations and won two key endorsements. So it came as a shock Friday when Sanders was hamstrung by, of all things, a Clinton data scandal. NGP VAN, the Democratic Party's main vendor for data services, mistakenly lowered the firewalls isolating each campaign's voter info -- and one Sanders staffer peeked. While the (now-fired) staffer claims they were just trying to gauge the scope of the exposure, the Clinton camp accused their rival of downloading valuable data. DNC chair Debbie Wasserman Schultz agreed, barring the entire campaign from NGP VAN in response -- potentially crippling their sprint to Iowa. Already dinged for shielding Clinton with favorable debate schedules, the DNC dropped the ban following outcry and a Sanders lawsuit (which Sanders campaign manager Jeff Weaver said might expose collusion). Crisis averted, though not without adding some potential fireworks to tonight's Democratic debate on ABC.
posted by Rhaomi on Dec 19, 2015 - 401 comments

Why Not Both?

Is it a Pokémon, or is it big data?
posted by Going To Maine on Nov 25, 2015 - 29 comments

Frankly, you sound a little paranoid

If someone had told me even a few years ago that such a thing wasn’t pure coincidence, I would have had my doubts about that someone. Now, however, I reserve my doubts for the people who still trust. There are so many ghosts in our machines—their locations so hidden, their methods so ingenious, their motives so inscrutable—that not to feel haunted is not to be awake. That’s why paranoia, even in its extreme forms, no longer seems to me so much a disorder as a mode of cognition with an impressive track record of prescience. --Walter Kirn on modern paranoia in The Atlantic [more inside]
posted by chavenet on Oct 14, 2015 - 33 comments

"About $43,000 a year."

What's the Difference Between Data Science and Statistics?Not long ago, the term "data science" meant nothing to most people-even to those who worked with data. A likely response to the term was: "Isn't that just statistics?" These days, data science is hot. The Harvard Business Review called data scientist the "Sexiest Job of the 21st Century."  So what changed? Why did data science become a distinct term? And what distinguishes data science from statistics?
posted by tonycpsu on Oct 13, 2015 - 38 comments

Snake Oil Supplements?

Scientific evidence for popular health supplements from information is beautiful [Snake Oil not included]
posted by chavenet on Sep 22, 2015 - 42 comments

Patent Data Visualization

PatentsView is a new patent data visualization platform from the US Patent and Trademark Office. The PatentsView beta search tool allows members of the public to interact with nearly 40 years of data on patenting activity in the United States. Users can explore technological, regional, and individual-level patent trends via search filters with multiple viewing options. The database links inventors, their organizations, locations, and overall patenting activity using enhanced 1976-2014 data from public USPTO bulk data files.
posted by jedicus on Sep 20, 2015 - 5 comments

Gonna try with a little help from my friends

Facebook Wants to Redline Your Friends List. Facebook recently filed a patent for its technology tracking how users connect within a network. Three of the uses described focus on how this information could be used to filter spam, tailor search results, or determine access to a third-party website. The fourth use explains how the credit ratings of those in a user's social network could be used to determine loan eligibility. This is raising concerns about data discrimination.
posted by earth by april on Sep 3, 2015 - 41 comments

Map of Jazz

Map of Jazz A visualization of collaboration in jazz through mapping players by session, for roughly 14,000 sessions. Full methodology described here (PDF)
posted by klangklangston on Aug 24, 2015 - 13 comments

Where is Google taking us?

I listen to one of the two or three key brains behind the Search algorithm itself, Ben Gomes, who speaks 10 to the dozen of “natural language generation” and “deep learning networks” (and, inevitably, of the “holy grail” of answering users’ questions before they have been asked). [more inside]
posted by Little Dawn on Jul 5, 2015 - 52 comments

Fatal police shootings in 2015 approaching 400 nationwide

The Washington Post is compiling a database of every fatal shooting by police in 2015, as well as of every officer killed by gunfire in the line of duty.
Overall, blacks were killed at three times the rate of whites or other minorities when adjusting by the population of the census tracts where the shootings occurred.
posted by Little Dawn on May 31, 2015 - 74 comments

The Vine Nerds

But to a growing cadre of A-list winemakers, there’s actionable intelligence in the data. Many of Fruition’s clients are altering their irrigation techniques, turning laggard vineyards into top performers and using far less water than they ever imagined. Along the way they’re extracting lessons that could extend far beyond this rarified corner of agriculture. By gaining insight into the relationships between water, sunlight, yield, and taste, Fruition Sciences is showing the way for farmers of all stripes to increase productivity and quality in a world of shifting weather patterns and decreasing supplies of freshwater.
posted by the man of twists and turns on May 10, 2015 - 18 comments

China announces it is scoring its citizens using big data

China rates its own citizens - including online behaviour: "The Chinese government is currently implementing a nationwide electronic system, called the Social Credit System, attributing to each of its 1,3 billion citizens a score for his or her behavior. The system will be based on various criteria, ranging from financial credibility and criminal record to social media behavior. From 2020 onwards each adult citizen should, besides his identity card, have such a credit code." [more inside]
posted by kliuless on May 5, 2015 - 77 comments

History as data science

History Lab has "focused on digitizing, structuring and visualizing large sets of declassified US government documents. This is a starting point for showcasing how computational techniques can aid historical research." Can big-data analysis show what kinds of information the government is keeping classified? [more inside]
posted by the man of twists and turns on Apr 27, 2015 - 4 comments

"For example, we could transmit the contents of the Internet."

Seth Shostak, director of the Center for SETI Research at the SETI Institute, suggests in a NYT Op-Ed that we should "offer the aliens Big Data."
Such a large corpus — with its text, pictures, videos and sounds — would allow clever extraterrestrials to decipher much about our society, and even formulate questions that could be answered with the material in hand.
Previously, Stephen Hawking has disagreed.
posted by Little Dawn on Mar 28, 2015 - 73 comments

♫Do you want to scrape a website?♫

Did you ever just want a bunch of web data as painlessly as possible but don't know a thing about command-line webscrapers (curl, wget) or parsing libraries (BeautifulSoup, JSoup, pandas)?
import.io will try to auto-magically hash any website you give it into structured data. (Here's MetaFilter.)
Need a bit more control over those results?
Kimono gives you a point-and-click environment for choosing page elements and pagination indicators. (Requires a Chrome add-on or browser bookmarklet.)
posted by Going To Maine on Mar 17, 2015 - 36 comments

The Algorithmic Self: On Being Made by the Numbers

"The first step toward protecting the self in an age of algorithmic manipulation is to recognize such manipulation as a problem." Frank Pasquale, writing for The Hedgehog Review, grapples with "The Algorithmic Self." [more inside]
posted by MonkeyToes on Mar 16, 2015 - 5 comments

Big Data looks at your poop

The team gathered samples over the course of a year from sewage treatment plants in 71 different cities in 31 states, chosen for their geographic spread and range of obesity rates. The leanest city sampled was Steamboat Springs, Colorado, with an obesity rate of 13.5 percent, while the heaviest was St. Joseph, Missouri, with a rate of 37.4 percent. ... "If we had a city with a higher percentage of obese people, we would see a higher percentage of the kinds of bacteria associated with obesity,"
posted by mecran01 on Mar 8, 2015 - 31 comments

you have probably heard of 'big data,' well get ready for Huge Data.

The DevOps League is your fast track to DevOps certification by the industry's leading Thought Lords. Supported by one manone stunning man – you will deep dive into such important concepts such as Dark Data, Data Beans and Terry the Data Goblin, to fully learn what DevOps really means. (But beware of the Realm of Dark Jon Hendren or 2014's Worst DevOps Article Headers.)
posted by slogger on Feb 25, 2015 - 37 comments

The Code We Can’t Control

David Auerbach for Slate discusses the dangers of the algorithm-driven data collection and organization of Big Data in a review for law professor Frank Pasquale's book on the subject, The Black-Box Society. [more inside]
posted by NoxAeternum on Jan 15, 2015 - 31 comments

Where form follows data.

Amsterdam City Dashboard: a City as Urban Statistics
posted by infini on Dec 15, 2014 - 8 comments

A Dispatch From The Future

Evgeny Morozov writes for the New Yorker: The Planning Machine, on Project Cybersyn (previously) and Big Data.
Greg Grandin in The Nation responds: The Anti-Socialist Origins of Big Data
posted by the man of twists and turns on Oct 29, 2014 - 46 comments

Grooming Students for A Lifetime of Surveillance

The same technologists who protest against the NSA’s metadata collection programs are the ones profiting the most from the widespread surveillance of students.
posted by the man of twists and turns on Oct 24, 2014 - 27 comments

The real problem with Big Data and ubiquitous surveillance

The question is not so much “do you trust the CIA/NSA/MI6/etc?”. It’s “Do you trust every single sysadmin working for these organisations? Every single analyst? Every single middle manager?”
posted by MartinWisse on Sep 24, 2014 - 23 comments

Down The Rabbit-Hole

For the very first time, the sales of one million sex toys and 45,000 of their reviews have been analysed to reveal what we do in our most intimate and uninhibited moments. Research by Jon Millward, who also brought us Deep Inside. (Previously) [All links NSFW]
posted by chavenet on Sep 9, 2014 - 20 comments

"Visualize race and income data for your community, county, and [USA]."

Justice Map [more inside]
posted by the man of twists and turns on Aug 29, 2014 - 17 comments

How to Invent a Person Online (Is online anonymity even possible today?)

On April 8, 2013, I received an envelope in the mail from a nonexistent return address in Toledo, Ohio. Inside was a blank thank-you note and an Ohio state driver’s license. The ID belonged to a 28-year-old man called Aaron Brown—6 feet tall and 160 pounds with a round face, scruffy brown hair, a thin beard, and green eyes. His most defining feature, however, was that he didn’t exist. I know that because I created him.
posted by spock on Jul 28, 2014 - 34 comments

Theater Map of Ukraine

"...it's social media that has helped build the public case against Russia" in Ukraine. One example is liveuamap.com, who "gather information from open sources and put it on the [Google] map" using familiar Google Maps markers for a Reds (Pro-Russian) vs Blues (Pro-Ukraine) theater map. Shaded regions indicate the Donetsk People's Republic (DNR; Red), Lugansk People's Republic (LNR; Purple), the MH17 crash site (Yellow), and the MH17 ceasefire zone (green). The posts linked to by each marker include a link to the source via a chain icon at the bottom of the post.
posted by jwells on Jul 23, 2014 - 11 comments

I Know Where Your Cat Lives

Only 15 million, riiiight. A data experiment out of Florida State University maps the location of 1 million of the 15 million publicly available online images tagged with the word "cat." Using a supercomputer and the map coordinates imbedded in their metadata, I Know Where Your Cat Lives shows where each image was taken, to within an estimated 7.8 meters accuracy. [more inside]
posted by gottabefunky on Jul 22, 2014 - 72 comments

Mapping Census Data

Datashine: Census is a site from UCLs Big Open Data: Mining and Synthesis project which provides an easy interface to map UK population data. [more inside]
posted by Just this guy, y'know on Jul 3, 2014 - 2 comments

All of them blue and with numbers and lens flare.

Big Data Pictures is a tumblr for visualizations of big data.
posted by Going To Maine on Jun 20, 2014 - 10 comments

The Internet With a Human Face

"These big collections of personal data are like radioactive waste. It's easy to generate, easy to store in the short term, incredibly toxic, and almost impossible to dispose of. Just when you think you've buried it forever, it comes leaching out somewhere unexpected." A talk by Maciej Ceglowski, founder of Pinboard, about why we have Big Data and why it's frightening. [more inside]
posted by 23 on May 27, 2014 - 48 comments

May 16 is National Bike To Work Day

Strava, the bike and run tracking system, is using their database to create Strava Metro, to sell to urban planners for commute data. But unless you're the Oregon DoT, London, or Alpine Shire, you might find the Strava bike and run heatmaps more useful. [more inside]
posted by the man of twists and turns on May 15, 2014 - 35 comments

How to not let the internet know you're pregnant

"And finally, I'm actually here today to win the 'Most Creative Use of Tor' award," she said, followed by roars of laughter in the audience. "I really couldn't have done it without Tor, because Tor was really the only way to manage totally untraceable browsing. I know it's gotten a bad reputation for Bitcoin trading and buying drugs online, but I used it for BabyCenter.com."-- How Janet Vertesi tried and hid her pregnancy from the internet and big data. (Direct link to her presentation.)
posted by MartinWisse on Apr 30, 2014 - 64 comments

InBloom wilts under privacy heat

Controversial education tech company InBloom has shut down over student data privacy concerns. Backed with $100 million in grants from the Bill & Melinda Gates Foundation and the Carnegie Corporation of New York, InBloom quickly announced nine states (CO, DE, GA, IL, KY, LA, MA, NC, NY) as partners, with more than 2.7 million students enrolled, with the goal of using big data to direct education emphasis and other decisions. With a recent decision by New York state to halt participation in any project involving storing student data in the way InBloom had planned (and the deletion of any such data already stored), all nine states had either put data sharing plans with InBloom on hold, made them voluntary, or pulled out completely. [more inside]
posted by 2bucksplus on Apr 22, 2014 - 29 comments

Practical information for wannabe Glenn Greenwalds

"The first journalist to attempt reporting on the Wikileaks cables was David Leigh of The Guardian. The material arrived as a single 1.7GB CSV file containing 251,287 U.S. diplomatic cables from 1966 to 2010. If you’ve ever tried to open a 1.7GB file, you know you probably can’t. Microsoft Word and Excel will plain refuse. Windows Notepad and Mac TextEdit will try, but slow to a crawl." At Opennews Source, Jonathan Stray has written a helpful beginners' guide to dealing with large amounts of documents for journalists and interested lay people.
posted by MartinWisse on Mar 17, 2014 - 18 comments

The economy of movement

Databall. With an ocean of new statistical information available, the NBA could be on the verge of understanding the value of every single movement on the court.
posted by antonymous on Feb 8, 2014 - 44 comments

Distributed Data

Academics launch torrent site to share papers and datasets [more inside]
posted by eviemath on Feb 4, 2014 - 12 comments

Data Love. Porn Data

Sexualitics tries to contribute to human sexuality understanding through a Big Data approach. Studies (PDF), Datasets and Porngrams (maps the evolution of words frequencies in the titles of porn videos).
posted by motdiem2 on Jan 30, 2014 - 18 comments

A story 10 years in the making

Measuring societal zeitgeist by counting mood words across millions of books correlates with the economic misery index shifted forward a decade. "When are we most miserable, according to literature? Ten to eleven years after an economic downturn." Paper: Books Average Previous Decade of Economic Misery.
posted by stbalbach on Jan 14, 2014 - 10 comments

Data You Can Believe In

The data analysis group that used Facebook and set top TV data to help Barack Obama win the latest election is taking its talents to the private sector. (SL NYTimes)
posted by reenum on Sep 30, 2013 - 16 comments

Singularity flip side

47% of US jobs under threat from computerization according to Oxford study. The study reveals a trend of computers taking over many cognitive tasks thanks to the availability of big data. [more inside]
posted by stbalbach on Sep 24, 2013 - 121 comments

A world in upheaval

A map of every protest everywhere since 1979 (some caveats are noted in the accompanying article).
posted by MartinWisse on Aug 26, 2013 - 18 comments

The Age of Networked Matter

An Aura of Familiarity: Visions from the Coming Age of Networked Matter. The Institute for the Future commissioned six science fiction writers to create short stories for their Age of Networked Matter research project. "We asked our collaborators to envision a world where humans have unprecedented control of matter at all scales, and to share with us a glimpse of daily life in that world. It was a process meant to make the future tangible." Three of the stories have appeared so far. [more inside]
posted by homunculus on Jun 28, 2013 - 9 comments

Page: 1 2