November 2009
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30          

Authors' Committee

Chair:

Matt Blackwell (Gov)

Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries

Categories

Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 4.24-en


« November 17, 2009 | Main | November 26, 2009 »

21 November 2009

Violations of SUTVA

Network methods and methods for causal inference are popular areas of research in social sciences. Often they are considered separately due to a fundamental difference in their basic assumptions. Network methods assume that individual units are interdependent, that one network member's actions have consequences for other members of the network. Methods for causal inference, in contrast, often rest on the Stable Unit Treatment Value Assumption (SUTVA). SUTVA requires that the response of a particular unit depends only on the treatment to which he himself was assigned, not the treatments of others around him. It is a useful assumption, but as with all assumptions, there are circumstances in which it is not credible. What can be done in these circumstances?

When researchers suspect that there may be spillover between units in different treatment groups, they can change their unit of analysis. Students assigned to attend a tutoring program to improve their grades might interact with other students in their school who were not assigned to the tutoring program and influence the grades of these control students. To enable causal inference, the analysis might be completed at the school level rather than the individual level. SUTVA would then require no interference across schools, a more plausible assumption than no interference across students. However, this approach is somewhat unsatisfactory. It generally entails a sharp reduction in sample size. More importantly, it changes the question that we can answer: no longer can we learn about the performance of individual students, we can only learn about the performance of schools.

I have not come across a more satisfactory statistical solution for circumstances in which SUTVA is violated. In an interesting new paper, Manski provides some bounds on treatment effects in the presence of social interactions. Unfortunately, these bounds are often uninformative, since when SUTVA is violated random assignment to treatment arms does not identify treatment effects. Sinclair suggests using multi-level experiments to empirically identify spillover effects. This approach (which relies on multiple rounds of randomization to test if treatment effects are overidentified, as we would expect if there were no spillovers) is appealing, as the process of diffusion within networks is of great scientific interest. However, it does not help identify treatment effects when spillovers are present. Neither can we simply assume that effects estimated under SUTVA represent upper bounds on the true effects, because it is possible that interference across units intensifies the treatment effects rather than diluting them. Manski's paper seems like a useful foray into an open area of research. Let me know of other work on methods for causal inference in network-like situations where interference across units is likely.

Posted by Deirdre Bloome at 4:50 PM