November 2006
Sun Mon Tue Wed Thu Fri Sat
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    

Authors' Committee

Chair:

Matt Blackwell (Gov)

Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries

Categories

Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 4.24-en


« November 22, 2006 | Main | November 28, 2006 »

27 November 2006

Designing and Analyzing Randomized Experiments in Political Science

I just read a paper by Yusaku Horiuchi, Kosuke Imai, and Naoko Taniguchi (HIT) on "Designing and Analyzing Randomized Experiments." HIT draw upon the longstanding statistics literature on this topic and attempt to “pave the way for further development of more methodologically sophisticated experimental studies in political science.” While experiments are becoming more frequent in political science, HIT observe that a majority of recent studies do not randomize effectively and still ignore problems of noncompliance and or nonresponse.

Specifically, they offer four general recommendations:

(I) Researchers should obtain information about background characteristics of experimental subjects that can be used to predict their noncompliance, nonresponse, and the outcome.

(II) Researchers should conduct efficient randomization of treatments by using, for example, randomized-block and matched-pair designs.

(III) Researchers must make every effort to record the precise treatment received by each experimental subject.

(IV) Finally, a valid statistical analysis of randomized experiments must properly account for noncompliance and nonresponse problems simultaneously.

Take a look. I agree with HIT that these issues are not new, yet too often ignored in political science (exceptions acknowledged). HIT illustrate their recommendations using a carefully crafted online experiment on Japanese elections. Statistically, they employ a Bayesian approach using the general statistical framework of randomized experiments with noncompliance and nonresponse (Angrist, Imbens, and Rubin 1996; Imbens and Rubin 1997; Frangakis and Rubin 1999, 2002). There is also interesting new stuff on modeling causal heterogeneity in this framework (a big topic in of itself).

Posted by Jens Hainmueller at 12:19 PM