the 'gold standard' of statistical validity, are not as reliable as many scientists assume.
posted by dhruva
on Feb 14, 2014 -
How to think about "Science Studies Prove My Position", for politicians and all non-scientists. Any collation of measures (the effectiveness of a given school, say) will show variability owing to differences in innate ability (teacher competence), plus sampling (children might by chance be an atypical sample with complications), plus bias (the school might be in an area where people are unusually unhealthy), plus measurement error (outcomes might be measured in different ways for different schools). However, the resulting variation is typically interpreted only as differences in innate ability, ignoring the other sources. This becomes problematic with statements describing an extreme outcome ('the pass rate doubled') or comparing the magnitude of the extreme with the mean ('the pass rate in school x is three times the national average') or the range ('there is an x-fold difference between the highest- and lowest-performing schools'). League tables, in particular, are rarely reliable summaries of performance.
posted by Dashy
on Nov 25, 2013 -
Statistics Done Wrong
is a guide to the most popular statistical errors and slip-ups committed by scientists every day, in the lab and in peer-reviewed journals. Statistics Done Wrong assumes no prior knowledge of statistics, so you can read it before your first statistics course or after thirty years of scientific practice.
posted by Foci for Analysis
on Oct 27, 2013 -
Is Psychometric g a Myth?
- "As an online discussion about IQ or general intelligence grows longer, the probability of someone linking to statistician Cosma Shalizi's essay g, a Statistical Myth
approaches 1. Usually the link is accompanied by an assertion to the effect that Shalizi offers a definitive refutation of the concept of general mental ability, or psychometric g
." [more inside]
posted by kliuless
on Apr 11, 2013 -
"Brain training games don't actually make you smarter.
" Looking at recent meta-analyses and replication attempts of studies showing increased cognitive abilities gained from brain-training games, the New Yorker article comes to the conclusion that the results are suspect and these games haven't been shown to improve cognitive abilities broadly. Currently, brain training is a multi-million-dollar business.
posted by tykky
on Apr 9, 2013 -
With a database of over 5,000 scientists, from Nobel prize winners to postdocs and PhD students, Sense About Science
works in partnership with scientific bodies, research publishers, policy makers, the public and the media, to change public discussions about science and evidence. They make these scientists available for questions from civic organizations and the public looking for scientific advice from experts
, campaign for the promotion of scientific principles in public policy
, and publish neat guides to understanding science intended for laypeople. [more inside]
posted by Blasdelb
on Feb 28, 2013 -
The practice of lying to one's children to encourage behavioral compliance was investigated among parents in the US (N = 114) and China (N = 85). The vast majority of parents (84% in the US and 98% in China) reported having lied to their children for this purpose. Within each country, the practice most frequently took the form of falsely threatening to leave a child alone in public if he or she refused to follow the parent. Crosscultural differences were seen: A larger proportion of the parents in China reported that they employed instrumental lie-telling to promote behavioral compliance, and a larger proportion approved of this practice, as compared to the parents in the US. This difference was not seen on measures relating to the practice of lying to promote positive feelings, or on measures relating to statements about fantasy characters such as the tooth fairy. Findings are discussed with reference to sociocultural values and certain parenting-related challenges that extend across cultures. [HTML]
-- [PDF] [more inside]
posted by Blasdelb
on Jan 23, 2013 -
The Nature of Computation
- Intellects Vast and Warm and Sympathetic
: "I hand you a network or graph, and ask whether there is a path through the network that crosses each edge exactly once, returning to its starting point. (That is, I ask whether there is a 'Eulerian' cycle.) Then I hand you another network, and ask whether there is a path which visits each node exactly once. (That is, I ask whether there is a 'Hamiltonian' cycle.) How hard is it to answer me?" (via
) [more inside]
posted by kliuless
on Dec 1, 2012 -
"We have little trouble recognizing that a chess grandmaster’s victory over a novice is skill, as well as assuming that Paul the octopus’s ability to predict World Cup games is due to chance. But what about everything else?" [Luck and Skill Untangled: The Science of Success
posted by vidur
on Nov 20, 2012 -
is an exhaustive website devoted to the life
and works of the statistical pioneer and "father of eugenics" Francis Galton, inventor of the scatterplot, the correlation coefficient, fingerprint identification, and who knows what else. Almost all of Galton's books
are reproduced here, some in scanned form and some in searchable .pdf, from his major books to his letters to Pigeon Fancier's Journal
. A short selection after the fold. [more inside]
posted by escabeche
on Apr 25, 2012 -
provides a RESTful
interface to the popular open-source statistical package R
, enabling the user to perform calculations and create publication-quality or web-embeddable visualizations via standard web requests.
posted by Blazecock Pileon
on Nov 10, 2011 -
is a veteran American cartoonist best known for his delightful comic-book guides to science and history, many of which have previews online. Chief among them is his long-running Cartoon History of the Universe
(later The Cartoon History of the Modern World
), a sprawling multi-volume opus documenting everything from the Big Bang to the Bush administration. Published over the course of three decades, it takes a truly global view -- its time-traveling Professor thoroughly explores not only familiar topics like Rome and World War II but the oft-neglected stories of Asia and Africa, blending caricature and myth with careful scholarship (cited by fun illustrated bibliographies
) and tackling even the most obscure events with intelligence and wit
. This savvy satire carried over to Gonick's Zinn
chronicle The Cartoon History of the United States
, along with a bevy of Cartoon Guides
to other topics, including Genetics, Computer Science, Chemistry, Physics, Statistics, The Environment
, and (yes!) Sex
. Gonick has also maintained a few sideprojects, such as a webcomic look at Chinese invention
, assorted math comics
), the Muse magazine
mainstay Kokopelli & Co.
(featuring the shenanigans of his "New Muses"
), and more
. See also these lengthy interview snippets
, linked previously
. Want more? Amazon links to the complete oeuvre inside! [more inside]
posted by Rhaomi
on Jun 6, 2011 -
Following the Journal of Personality and Social Psychology
's decision to publish Daryl Bem's writeup of 8 studies
(PDF) purporting to show evidence for precognition (previously
), researchers from the University of Amsterdam have written a rebuttal
(PDF) which finds methodological flaws not only in Bem's research, but in many other published papers in experimental psychology. Could this prove to be psychology's cold fusion moment
? [more inside]
posted by yourcelf
on Jan 8, 2011 -
'Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong.'
Dr. John P. A. Ioannidis, adjunct professor at Tufts University
School of Medicine is a meta-researcher. 'He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences.' [more inside]
posted by VikingSword
on Oct 18, 2010 -
Do we live in a world where there is magic and meaning, or is it all just chance?
Radiolab meets two young women who share a nearly unbelievable story of coincidence and fate. Then they consult with statisticians for a very different take on the same story.
This short audio documentary is charming and delightful. A Lucky Wind
won a Best Documentary: Honorable Mention Award in the 2009 Third Coast / Richard H. Driehaus Foundation Competition as well as the 2009 AAAS Kavli Science Journalism Award (Radio Documentary). [more inside]
posted by storybored
on Aug 25, 2010 -
...Or how our most common statistical methods really weren't meant to be used that way and why that study result is likely spurious. Since mefites like to argue about stats, here's some background for us all (and I'm not talking correlation vs causation)!
posted by mandymanwasregistered
on Mar 18, 2010 -
Mercenary Epidemiology: Data Reanalysis and Reinterpretation for Sponsors With Financial Interest in the Outcome.
(.pdf link) When should scientists be required to release their raw data for (potentially hostile) re-analysis? A letter to the editors of Annals of Epidemiology from David Michaels, Ph.D., MPH, public health blogger
, author of the book Doubt Is Their Product
, and, as of December 2009, the Assistant Secretary of Labor for OSHA,
unanimously confirmed by the Senate despite the dismay of some
. Michaels interviewed at Science Progress
about Doubt Is Their Product
(podcast, with transcript.)
posted by escabeche
on Feb 11, 2010 -
Whatever one's opinion of its possible limitations, the 2006 Iraq mortality survey produced epidemiological evidence that coalition forces have failed to protect Iraqi civilians... If, for the sake of argument, the study is wrong and the number of Iraqi deaths is less than half the infamous figure, is it acceptable that "only" 300,000 have died? Last November, with no explanation, the Iraqi Ministry of Health suddenly began citing 150,000 dead, five times its previous estimate. Is that amount of death acceptable? In January, the United Nations reported that more than 34,000 Iraqis were killed violently in the last year alone. Is that acceptable?
Regarding The Number
, the result of what one of the study's authors calls an episode more deadly than the Rwandan genocide
... [more within]
posted by y2karl
on Mar 7, 2007 -
The Logic of Diversity
"A new book, The Wisdom of Crowds
] by The New Yorker
columnist James Surowiecki, has recently popularized the idea that groups can, in some ways, be smarter than their members, which is superficially similar to Page's results
. While Surowiecki gives many examples of what one might call collective cognition, where groups out-perform isolated individuals, he really has only one explanation for this phenomenon, based on one of his examples: jelly beans [...
] averaging together many independent, unbiased guesses gives a result that is probably closer to the truth than any one guess. While true — it's the central limit theorem
of statistics — it's far from being the only way in which diversity
can be beneficial in problem solving." (Three-Toed Sloth)
posted by kliuless
on Jun 20, 2005 -