October 2009
Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Authors' Committee

Chair:

Matt Blackwell (Gov)

Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries

Categories

Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 4.24-en


« Elements of Statistical Learning (Online) | Main | Sources of Randomness »

21 October 2009

Multiple Instruments

I recently found a paper by Angus Deaton that attempts to (1) discount the usefulness of instrumental variables for making causal inferences in development economics and (2) discount the usefulness of field experiments. He has definitely stirred the pot a little and is now part of an interesting debate, although the discussion seems to be more focused on Deaton's controversial claims about experiments.

I think Deaton overlooks some of the benefits of experimental research but his criticism of instrumental variables seems dead on, especially on the use of multiple instruments (see pages 12-13). Intuitively, we might think that having many instruments makes for better causal inference -- if one doesn't work out, then the others will pick up the slack. Following this logic, studies that use multiple instruments and "test" for exogeneity with overidentification tests have become popular in the instrumental variables literature. Essentially, these tests boil down to re-estimating the model with subsets of the instruments and showing that the estimated coefficients don't change dramatically. This can mean one of two things: (a) not just one, but all of the instruments are exogenous, or (b) not just one, but all of the instruments are endogenous. Personally, I think the probability of finding even a single good instrument for a given problem is small, so when shown a research design with multiple instruments, I need some serious convincing that miraculously all of the instruments are valid.

I am probably overly skeptical and I am very sympathetic to heroic attempts to solve difficult problems of causal inference to answer important questions. Still, it seems that having multiple instruments can become an embarrassment of riches. A good instrument is so hard to come by that having too many starts to lend evidence against an empirical argument rather than for it.

Posted by Richard Nielsen at October 21, 2009 12:41 PM