stimulusing

April 3, 2010 7:31 PM

Veronique de Rugy, NRO contributor and George Mason fellow, says her research indicates that stimulus funding was disproportionately directed towards Democratic congressional districts. Nate Silver begs to disagree. De Rugy responds here; Silver responds here. Others say that this is a model "for the quick, effective peer-review that the internet facilitates." Perhaps this is a new model for peer review?
posted by lalex (27 comments total)

Flame warblogging becoming Peer Review 2.0?
posted by mccarty.tim at 7:43 PM on April 3, 2010

Nate Silver, the respected blogger at 538.com...

Fail.
posted by MikeKD at 7:53 PM on April 3, 2010

"Nate Silver, the respected blogger at 538.com..."

fail.

?
posted by lalex at 8:25 PM on April 3, 2010

It's fivethirtyeight.com, not 538.com.
posted by alexei at 8:32 PM on April 3, 2010

His site is fivethirtyeight.com, not 538.com (from de Rugy in the "De Rugy responds here;" link). It doesn't reinforce her attention to detail to mess up that much (and, no, 538.com doesn't redirect to the proper site).
posted by MikeKD at 8:36 PM on April 3, 2010

MikeKD, is there a reason that you immediately called FAIL?
posted by lalex at 9:36 PM on April 3, 2010

Perhaps an "epic fail" was in order.
posted by mek at 10:02 PM on April 3, 2010

Let's see. Veronique de Rugy, partisan hack, working for a right-wing-welfare think tank makes inflammatory accusations in testimony to Congress which is thoroughly demolished by common sense analysis. She blames her error on poor government data, but even though she knew the data was incomplete, it did not cause her to hesitate for an instant in her dubious conclusions. Somehow this becomes the new model for peer review? Lord help us. This is just one more instance of conservatives inventing their own reality leaving everyone else the interminable and frankly hopeless task of cleaning up the mess.
posted by JackFlash at 10:47 PM on April 3, 2010 [4 favorites]

Being an economist at George Mason is only slightly more credible than being an analyst at the Heritage Foundation, a special assistant to Rupert Murdoch, or an advisor to Pinochet. The difference is that this particular right-wing hack who couldn't cut it on the open market without ideological affirmative action happens to be paid with tax payer dollars. Which is its own sort of mind fuck for the libertarian/failed mathematicians that they employ.

Do you think that 2+2 =4? Are you unable to do more complicated math? Do you hate the poor? Come work at George Mason! All the academia with half the credibility and none of the peer review.
posted by allen.spaulding at 10:50 PM on April 3, 2010

JackFlash: in viewing this as an example of the "new model for peer review" it's Silver who'd be the peer reviewer of de Rugy's work, not de Rugy as the peer reviewer of the government's work. So I don't get your "Somehow this becomes the new model for peer review? Lord help us."
posted by logopetria at 11:58 PM on April 3, 2010

lalex: "Silver responds here."

de Rugy is also to be commended for having released portions of her dataset** on the Mercauts Center website...

There are two other variations that I find less impluasible...

I find it less impausible that the funds could have been directed toward...

...even after controlling for other demographic variabes...

...they are no proof of casuation at *any* order of magnitude...

de Rugy is correct that many demographic variables are correlated with one another, which makes model speification more difficut and can lead to potential problems with overfitting. However, these demographic variables are also correlated with the poltical representation in the Congress. Moreover, because the stimulus consists of many different 'layers' (categories of projects), it is quite plausible that many different demographic variables (as well as intercations between two or more such variabes) could come to bear on how funds were ultimately distributed.

This is a sticky (albeit common) problem. The best way to handle it would probably be to make several different specificiations of the model and to publish them explicitly.

Silver is one smart SOB, but goddamn does he need to invest in a copyeditor. Or even a simple spellcheck, which I'm pretty sure every major word processor, web browser, and blogging platform has baked in by now. FiveThirtyEight's analysis is top-notch, but seeing multiple typos in every single post is getting very distracting, especially when there's no excuse for it.
posted by Rhaomi at 12:42 AM on April 4, 2010

I read Rhaomi's comment, and then the post title and, yeah. oll.
posted by iamkimiam at 1:12 AM on April 4, 2010

Nate "Poblano" Silver is a one man peer review army. Fivethirtyeight.com is one of the best political websites out there, and its emergence in the 2008 campaign was one of the best aspects of that horrid year.

Nate Silver for president. Yes nerds can.
posted by fourcheesemac at 5:38 AM on April 4, 2010

Also, I think it's fair to point out that referring to the site as "538" is house style on that very blog. Check it out. It's not an "epic fail" except at the level of linking.
posted by fourcheesemac at 5:42 AM on April 4, 2010

She can control for capitol easily with dummy variables. There's no need to drop them from the sample and re-estimate her equations.
posted by scunning at 6:13 AM on April 4, 2010

Is the substance of her argument just Table 4 in this? Seems kind of preliminary to be testifying on it before Congress. I wonder if the republican indicator is statistically significant if she uses Huber-White robust standard errors, which is standard. The r-squared is small (0.04), and as there are few controls, I wouldn't hold my breath on this holding up with more scrutiny (scrutiny on the model, not her motivations).
posted by scunning at 6:18 AM on April 4, 2010

Scratch that - downloaded the data and estimated it with robust standard errors, and excluding "unemploymentchange" which is not in her data, the coefficient on "republican" and "marginaly" is the same and even more precise.

. regress logdollars tenure republican leadership marginaly appropriations construction meaninc

Source | SS df MS Number of obs = 436
-------------+------------------------------ F( 7, 428) = 3.45
Model | 9.63002687 7 1.37571812 Prob > F = 0.0013
Residual | 170.915715 428 .399335782 R-squared = 0.0533
-------------+------------------------------ Adj R-squared = 0.0379
Total | 180.545742 435 .415047682 Root MSE = .63193

------------------------------------------------------------------------------
logdollars | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
tenure | .0023586 .0037243 0.63 0.527 -.0049617 .0096789
republican | -.234922 .0631733 -3.72 0.000 -.3590904 -.1107535
leadership | -.0519316 .1065654 -0.49 0.626 -.2613882 .157525
marginaly | -.1607593 .0836406 -1.92 0.055 -.3251568 .0036381
appropriat~s | .0355848 .0909001 0.39 0.696 -.1430813 .2142509
construction | -.013629 .0163218 -0.84 0.404 -.0457099 .0184519
meaninc | -4.90e-07 1.60e-06 -0.31 0.759 -3.63e-06 2.65e-06
_cons | 8.189609 .1897399 43.16 0.000 7.816671 8.562547
------------------------------------------------------------------------------

. regress logdollars tenure republican leadership marginaly appropriations construction meaninc , robust

Linear regression Number of obs = 436
F( 7, 428) = 4.02
Prob > F = 0.0003
R-squared = 0.0533
Root MSE = .63193

------------------------------------------------------------------------------
| Robust
logdollars | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
tenure | .0023586 .0034539 0.68 0.495 -.0044301 .0091473
republican | -.234922 .0620565 -3.79 0.000 -.3568954 -.1129485
leadership | -.0519316 .0962185 -0.54 0.590 -.2410513 .1371881
marginaly | -.1607593 .0723499 -2.22 0.027 -.3029647 -.0185539
appropriat~s | .0355848 .0889185 0.40 0.689 -.1391865 .2103562
construction | -.013629 .0168218 -0.81 0.418 -.0466927 .0194347
meaninc | -4.90e-07 1.37e-06 -0.36 0.722 -3.19e-06 2.21e-06
_cons | 8.189609 .1790648 45.74 0.000 7.837653 8.541565
------------------------------------------------------------------------------
posted by scunning at 6:26 AM on April 4, 2010

Yay, KosFilter!
posted by ZenMasterThis at 6:36 AM on April 4, 2010

Now I'm going to be obsessed with this data all day. Grr. The dataset is interesting. I don't know what to make of the statistical significance on the republican indicator, because changing the model can bring it down. If all you are doing is moving variables in and out from the equation and estimating with OLS, then "republican" seems to have on average a significance in the range 5-15%. Percent that voted for Obama, on the other hand, is strongly correlated. But this is again where one needs to get a lot more information about these districts. You could maybe approach this with regression discontinuity as the framework, and see if the closely won districts that went Republican received any difference in funding. Here's another model below. If you include "meaninc", then the republican indicator goes to significance 0.21. The "percent Obama" remains strong. "Marginaly" also seems more robust than "republican".

. regress logdollars republican tenure leadership perobama difference marginaly goplead appropriations unemployment

Source | SS df MS Number of obs = 436
-------------+------------------------------ F( 9, 426) = 3.30
Model | 11.7821932 9 1.30913258 Prob > F = 0.0007
Residual | 168.763548 426 .396158564 R-squared = 0.0653
-------------+------------------------------ Adj R-squared = 0.0455
Total | 180.545742 435 .415047682 Root MSE = .62941

------------------------------------------------------------------------------
logdollars | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
republican | -.1345283 .0831062 -1.62 0.106 -.2978776 .028821
tenure | .001723 .0038121 0.45 0.652 -.0057699 .009216
leadership | -.0706761 .1484542 -0.48 0.634 -.3624701 .2211179
perobama | .0071237 .0034464 2.07 0.039 .0003497 .0138977
difference | .0001065 .0022571 0.05 0.962 -.00433 .0045429
marginaly | -.1568453 .096417 -1.63 0.105 -.3463575 .0326669
goplead | .031284 .1946887 0.16 0.872 -.3513861 .4139541
appropriat~s | .0384515 .0911641 0.42 0.673 -.1407359 .217639
unemployment | -.0427284 .0303218 -1.41 0.160 -.1023274 .0168705
_cons | 7.813197 .1825817 42.79 0.000 7.454324 8.17207
------------------------------------------------------------------------------
posted by scunning at 6:36 AM on April 4, 2010

She says this in the document, "We downloaded all FY09 Q4 recipient reports for contracts and grants from the official Recovery.gov Web site. [1] These are self-reports submitted by the recipients of stimulus contracts and grants." I wonder if self-report could be correlated with party affiliation. Are Democratic beneficiaries more likely to report (as opposed to more likely to receive) funding?
posted by scunning at 6:42 AM on April 4, 2010 [1 favorite]

Being an economist at George Mason is only slightly more credible than being an analyst at the Heritage Foundation, a special assistant to Rupert Murdoch, or an advisor to Pinochet.

Vernon Smith won the Nobel in 2002.
James Buchanan won the Nobel in 1986.
Gordon Tullock co-wrote (with Buchanan) The Calculus of Consent.
posted by Slap Factory at 8:19 AM on April 4, 2010

scunning: Are Democratic beneficiaries more likely to report (as opposed to more likely to receive) funding?

Weren't there Republican state governors and/or legislatures who were quite vocal that their states would not be participating in the stimulus program? If this trickled down to Republican-aligned business owners who refused that filthy Obamacash, no wonder they're now finding a preponderance of Democratic-oriented funding recipients.

Or I should just just go off and read Nate's analyses - I fear the Veronique de Rugy* ones would make my ears bleed.

*Shouldn't she be known as VERONICA FREEDOM to right-wing circles?
posted by hangashore at 9:47 AM on April 4, 2010 [1 favorite]

In viewing this as an example of the "new model for peer review" it's Silver who'd be the peer reviewer of de Rugy's work, not de Rugy as the peer reviewer of the government's work. So I don't get your "Somehow this becomes the new model for peer review? Lord help us."

Peer review is supposed to stop the dissemination of bad information before it is released to the public. Once bad information is released it is too late to undo the damage. As Mark Twain said "A lie can travel halfway around the world while the truth is putting on its shoes."

De Rugy got her fifteen minutes of fame in front of Congress and the television cameras and now it is all over Fox News, CNN and talk radio. Do you think any of those sources have even heard of Nate Silver? Real peer review would have detected the flaws in de Rugy's analysis and rejected it before publication. Instead, the lie is out there and can't be undone.

Look at the polls for the number of people who still believe to this day that Saddam Hussein was responsible for 9/11. Many psychological studies of the misinformation effect have shown that once a false idea is implanted that any attempt to correct the misinformation actually has the perverse effect of strengthening the original misconception.

Bad information always drives out the good. That is why "peer review" after the fact, as described in here, is useless and may actually result in greater misunderstanding by simply creating more publicity for the controversy. Muddying the waters is a very effective propaganda technique used, for example, by the tobacco companies or more recently by the oil and coal industry. You can't un-muddy the waters. Invented controversy just plays into their hands by creating confusion and doubt.
posted by JackFlash at 10:10 AM on April 4, 2010 [5 favorites]

I'm intrigued by the piece of this kerfuffle in which de Rugy claims she has data on unemployment rates by congressional district even though the dataset she made public only shows it by MSA. Mostly because I'd be really interested in seeing unemployment data at a finer geography than county or MSA, so I kind of hope she's not bluffing, but if she is, bluffing about the existence of nonexistent data is kind of impressive.
posted by yarrow at 3:31 PM on April 4, 2010

Mostly because I'd be really interested in seeing unemployment data at a finer geography than county or MSA

You can pull 2008 unemployment by US House district straight from the American Factfinder at the Census.
posted by ROU_Xenophobe at 5:29 PM on April 4, 2010

Yes, the American Community Survey data - did not know they presented by congressional district, thanks! I was thinking about BLS data. In the circles I run in ACS data is viewed with great skepticism at smaller geographies but I'm not really up on the nuances of why.
posted by yarrow at 6:28 PM on April 4, 2010

Weren't there Republican state governors and/or legislatures who were quite vocal that their states would not be participating in the stimulus program?

Yeah; here in Florida I know at least some available stimulus funds have gone and continue to go unclaimed due to political considerations--primarily, Republicans in the legislature not wanting to be seen as making accommodations with the enemy.
posted by saulgoodman at 7:00 PM on April 4, 2010