Fifteen months ago, Donald declared and we commented; two months earlier, Hillary did likewise. And now, here we are near the end of an divisive and damaging election. As Donald's campaign struggles under many allegations [BBC] [NBC News] [Guardian] [New York Times] and increased conversation on abuse, Hillary pulls out a 7 point lead in a Fox poll, a gap in the Real Clear Politics poll average and a large victory chance in 538 (though, cautionary words about poll bounces). Michelle Obama spoke about the language of this election (FPP title from her speech) [BBC] [New Yorker] [Washington Post] and in The Guardian: "She lent her extraordinary ability to say what people are feeling to every English-speaking woman in the world". Elsewhere, Trump-stooge Chris Christie is facing a criminal summons and Utah could be a three-way race which leads to a small possibility of President Evan. [more inside]
How can I become a Data Scientist.......the first answer in this Quora thread is a pretty concise profile of this hot (and hyped) new career choice written by William Chen, whose data science blog Storytelling with Statistics has got some cool stuff in it. Like the Probability Cheat Sheet. [more inside]
The Sleeping Beauty Problem is a problem in probability (rumored to have originated at MIT) that appears trivially simple, yet has inspired some rather sophisticated arguments. [more inside]
Battleship Probability Calculator by C. Liam Brown. Finds the best squares to try during the game.
You're sitting down with your friends to play a boardgame, and you find yourself in a conundrum: how do you choose a first player? Sure, you could roll a standard die and take highest number, but what if there's a tie? That could take forever! Besides, wouldn't you rather be mathematically sure that everyone has a fair shot at each spot in the turn order? Of course you would!
"I amused myself for over a year thinking about the impacts of different toilet seat administration policies and how to measure them – doing calculations in my head, considering ratios of Standing events to Sitting events, and I slowly began to understand some of the specific differences in the basic policies that know to be administered most often. Finally, I decided to perform a probabilistic analysis". Essential Toilet Seat Analytics.
How to tell correlation from causation - "The basic intuition behind the method demonstrated by Prof. Joris Mooij of the University of Amsterdam and his co-authors is surprisingly simple: if one event influences another, then the random noise in the causing event will be reflected in the affected event."
The NYT calculates the probability of pregnancy using 15 common birth control methods, for up to 10 years of both "typical" and "perfect" use. Protip: the graphs do slidey comparison things on mouseover!
In a Multiverse, What Are the Odds? "Testing the multiverse hypothesis requires measuring whether our universe is statistically typical among the infinite variety of universes. But infinity does a number on statistics." (previously) [more inside]
Hyperreal numbers: infinities and infinitesimals - "In 1976, Jerome Keisler, a student of the famous logician Tarski, published this elementary textbook that teaches calculus using hyperreal numbers. Now it's free, with a Creative Commons copyright!" (pdf—25mb :) [more inside]
Network Theory Overview - "The idea: nature and the world of human technology are full of networks! People like to draw diagrams of networks. Mathematical physicists know that in principle these diagrams can be understood using category theory. But why should physicists have all the fun? This is the century of understanding living systems and adapting to life on a finite planet. Math isn't the main thing we need, but it's got to be part of the solution... so one thing we should do is develop a unified and powerful theory of networks." (via ;)
...to leave a smile on your face, by Helder Guimarães: Individual vs Crowd | Chaos | Freedom | Trick [more inside]
GaMuSo is an application of BioGraph-based data mining to music, which helps you get recommendations for other musicians. Based on 140K user-defined tags from last.fm that are collected for over 400K artists, results are sorted by the "nearest" or most probable matches for your artist of interest (algorithm described here). [more inside]
Walter Hickey at Business Insider looks at when you should buy a Powerball ticket and whether to take the lump sum or annuity if you win.
The thrill and rush of possibly winning started to wear off after about the twentieth losing ticket. Each card had a couple of “Life” symbols on them, and every time you got a second you just dreamed of seeing the third one under the remaining graphite. However it never appeared and never will and it just kind of turned depressing. How could people put themselves through this humiliation and teasing every day of their lives?
The classic criticism of the lottery is that the people who play are the ones who can least afford to lose; that the lottery is a sink of money, draining wealth from those who most need it. Some lottery advocates . . . have tried to defend lottery-ticket buying as a rational purchase of fantasy—paying a dollar for a day's worth of pleasant anticipation, imagining yourself as a millionaire. But consider exactly what this implies. It would mean that you're occupying your valuable brain with a fantasy whose real probability is nearly zero—a tiny line of likelihood which you, yourself, can do nothing to realize. . . . Which makes the lottery another kind of sink: a sink of emotional energy. [via]
Is Psychometric g a Myth? - "As an online discussion about IQ or general intelligence grows longer, the probability of someone linking to statistician Cosma Shalizi's essay g, a Statistical Myth approaches 1. Usually the link is accompanied by an assertion to the effect that Shalizi offers a definitive refutation of the concept of general mental ability, or psychometric g." [more inside]
Bayesian analysis shows redshirts are not most likely to die on Star Trek:TOS. Although Enterprise crew members in redshirts suffer many more casualties than crew members in other uniforms, they suffer fewer casualties than crew members in gold uniforms when the entire population size is considered. Only 10% of the entire redshirt population was lost during the three year run of Star Trek. This is less than the 13.4% of goldshirts, but more than the 5.1% of blueshirts. What is truly hazardous is not wearing a redshirt, but being a member of the security department. The red-shirted members of security were only 20.9% of the entire crew, but there is a 61.9% chance that the next casualty is in a redshirt and 64.5% chance this red-shirted victim is a member of the security department. The remaining redshirts, operations and engineering make up the largest single population, but only have an 8.6% chance of being a casualty.
The Nature of Computation - Intellects Vast and Warm and Sympathetic: "I hand you a network or graph, and ask whether there is a path through the network that crosses each edge exactly once, returning to its starting point. (That is, I ask whether there is a 'Eulerian' cycle.) Then I hand you another network, and ask whether there is a path which visits each node exactly once. (That is, I ask whether there is a 'Hamiltonian' cycle.) How hard is it to answer me?" (via) [more inside]
Tails of the Unexpected: "Normality has been an accepted wisdom in economics and finance for a century or more. Yet in real-world systems, nothing could be less normal than normality. Tails should not be unexpected, for they are the rule." An eminently human-readable explanation of why normal models fail to describe the uncertainties of our abnormal world. [more inside]
H _ _ _ m _ n, Y a _ _ _ e e, _ _ t t _ _ _ h i p, _ h u t _ s & L a _ _ e r _ , R _ _ k , _ _ n d y _ _ _ _ , and _ _ r t s.
The year was 1945. Two earthshaking events took place: the successful test at Alamogordo and the building of the first electronic computer. Their combined impact was to modify qualitatively the nature of global interactions between Russia and the West. No less perturbative were the changes wrought in all of academic research and in applied science. On a less grand scale these events brought about a [renaissance] of a mathematical technique known to the old guard as statistical sampling; in its new surroundings and owing to its nature, there was no denying its new name of the Monte Carlo method (PDF). -N. MetropolisConceptually talked about on MeFi previously, some basic Monte Carlo methods include the Inverse Transform Method (PDF) mentioned in the quoted paper, Acceptance-Rejection Sampling (PDFs 1,2), and integration with and without importance sampling (PDF).
An "Exciting Guide to Probability Distributions" from the University of Oxford: part 1, part 2. (Two links to PDFs)
"So, if the probability of finding an egg with two yolks is 1/1000 - then to find the likelihood of discovering four in a row you simply multiply the probabilities together four times. One thousand to the power of four brings us to the grand total of one trillion...
If true that would mean the event that occurred in Jen's kitchen was a trillion-to-one event. But is it true? No is the short answer."
If true that would mean the event that occurred in Jen's kitchen was a trillion-to-one event. But is it true? No is the short answer."
Counting is one of the first and simplest concepts most people are taught. But when you get beyond simple 123s, counting can become an advanced subject all its own. Essentially the science of counting, combinatorics is a key component of everything from abstract algebra to probability (PDF). [more inside]
Are We Alone In the Universe? New Analysis Says Maybe. In a new paper published on arXiv.org, astrophysicist David Spiegel at Princeton University and physicist Edwin Turner at the University of Tokyo argue...using a statistical method called Bayesian reasoning...that the life here on Earth could be common, or it could be extremely rare — there's no reason to prefer one conclusion over the other. [more inside]
In the recent MIT symposium "Brains, Minds and Machines," Chomsky criticized the use of purely statistical methods to understand linguistic behavior. Google's Director of Research, Peter Norvig responds. (via) [more inside]
Measure-theoretic probability: Why it should be learnt and how to get started. The clickable chart of distribution relationships. Just two of the interesting and informative probability resources I've learned about, along with countless other tidbits of information, from statistician John D. Cook's blog and his probability fact-of-the-day Twitter feed ProbFact. John also has daily tip and fact Twitter feeds for Windows keyboard shortcuts, regular expressions, TeX and LaTeX, algebra and number theory, topology and geometry, real and complex analysis, and beginning tomorrow, computer science and statistics.
Nontransitive dice are sets of dice (A, B, C, etc.) with counterintuitive properties: die A beats die B and die B beats die C, but die C beats die A. [more inside]
Durango Bill's Home Page. With topics that include: 3D end-to-end tour of the Grand Canyon, the origin and formation of the Colorado River, and examples of river systems that cut through mountain ranges instead of taking easier routes around them in Ancestral Rivers of the World. [more inside]
The day after a senator from Illinois, is elected president, the Pick 3 lottery in Illinois comes up 666. It's happened before, notably in Pennsylvania (12 times, including one time as part of a scam and once earlier this year, in Maryland. Some are jokingly (I hope) calling him the antichrist as a result. Others, namely numbers geeks like me, are spending their lunch hours looking up the history of lotteries drawing triple numbers and sharing it with MetaFilter.
THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS by Nassim Nicholas Taleb. "In the following Edge original essay, Taleb continues his examination of Black Swans, the highly improbable and unpredictable events that have massive impact. He claims that those who are putting society at risk are "no true statisticians", merely people using statistics either without understanding them, or in a self-serving manner.
A discovery leads to questions about whether the odds of people sharing genetic profiles are sometimes higher than portrayed. Calling the finding meaningless, the FBI has sought to block such inquiry.
On May 13, security advisories published by Debian and Ubuntu revealed that, for over a year, their OpenSSL libraries have had a major flaw in their CSPRNG, which is used by key generation functions in many widely-used applications, which caused the "random" numbers produced to be extremely predictable. [lolcat summary] [more inside]
The Monty Hall Problem has struck again, and this time it’s not merely embarrassing mathematicians. If the calculations of a Yale economist are correct, there’s a sneaky logical fallacy in some of the most famous experiments in psychology." The NY Times' John Tierney reports on new research into cognitive dissonance as examined through the famous Monty Hall Problem. [A previous MetaFilter thread about the Monty Hall Problem: Let's Make A Deal!]
Poker hand simulator. Get a feel for the odds before you bet the farm.
If you need a foolproof way to decide whether to kill someone or are simply curious as to whether probability is still operating as a factor in your existence (and find yourself out of change but near a computer with an internet connection), you can just use flip a coin.
Interactive mathematics miscellany and puzzles, including 75 proofs of the Pythagorean Theorem, an interactive column using Java applets, and eye-opening demonstrations. (Actually, much more.)
How many group photographs do you have to take to get one in which nobody is blinking? Nic Svenson and Dr Piers Barnes work it out.
We've talked about quantum computation a few times before, but how much do we really know? Metafilter, instruct thyself. Don't forget to learn some advanced probability and computational complexity (Scott Aaronson has more). Whoa, that's a lot o' learning, so let's so check out the much easier, and much cooler "sleeping puppy" experiment. I can only dream that will help break quantum mechanics' association with animal abuse. Then, there's the Free Will Theorem that just came out (some discussion on it) and another paper with a new look at an old problem. The latter describes another way of solving ye olde, super importanto Einstein-Podolsky-Rosen paradox using the relational interpretation of quantum mechanics (lots of discussion running around). Whew. We don't need the crackpot ramblings of What the #$*! Do We Know? when we've got real physics to keep us up at night. So, who wants to become a physicist? (t'Hooft has some thoughts for those who want to go theoretical.)
Experts can suck at predicting the future. Their intuitive sense of probability is no more developed than lay-people's. A classic experiment is to present two indistinguishable choices are presented, but with unequal probability of reward. Humans look for complex patterns, which don't exist, and preform quite poorly. Rats quickly recognize the choice with higher probability, and preform optimally.
Everybody's an expert, but does expertise promote better predictions?
Know less than nothing!? What could negative knowledge possibly mean? In short, after I tell you negative information, you will know less... "In this week's issue of Nature, however, Michal Horodecki and colleagues present a fresh approach to understanding quantum phenomena that cannot be grasped simply by considering their classical counterparts." [via slashdot :]
Incredible -- but true coincidences are fascinating, and pleasing, to the psyche. I tend to agree with John Littlewood (a University of Cambridge mathematician) that "...in the course of any normal person's life, miracles happen at a rate of roughly one per month." In other words, statistically speaking, unusual coincidences are to be expected in a world teeming with billions of humans. Still, I find such coincidences stangely inspiring. More can be found here.
The Logic of Diversity "A new book, The Wisdom of Crowds [..:] by The New Yorker columnist James Surowiecki, has recently popularized the idea that groups can, in some ways, be smarter than their members, which is superficially similar to Page's results. While Surowiecki gives many examples of what one might call collective cognition, where groups out-perform isolated individuals, he really has only one explanation for this phenomenon, based on one of his examples: jelly beans [...] averaging together many independent, unbiased guesses gives a result that is probably closer to the truth than any one guess. While true — it's the central limit theorem of statistics — it's far from being the only way in which diversity can be beneficial in problem solving." (Three-Toed Sloth)
An Intuitive Explanation of Bayesian Reasoning. [Page contains Java]
Page: 1 2