Violations of SUTVA

Network methods and methods for causal inference are popular areas of research in social sciences. Often they are considered separately due to a fundamental difference in their basic assumptions. Network methods assume that individual units are interdependent, that one network member's actions have consequences for other members of the network. Methods for causal inference, in contrast, often rest on the Stable Unit Treatment Value Assumption (SUTVA). SUTVA requires that the response of a particular unit depends only on the treatment to which he himself was assigned, not the treatments of others around him. It is a useful assumption, but as with all assumptions, there are circumstances in which it is not credible. What can be done in these circumstances?

When researchers suspect that there may be spillover between units in different treatment groups, they can change their unit of analysis. Students assigned to attend a tutoring program to improve their grades might interact with other students in their school who were not assigned to the tutoring program and influence the grades of these control students. To enable causal inference, the analysis might be completed at the school level rather than the individual level. SUTVA would then require no interference across schools, a more plausible assumption than no interference across students. However, this approach is somewhat unsatisfactory. It generally entails a sharp reduction in sample size. More importantly, it changes the question that we can answer: no longer can we learn about the performance of individual students, we can only learn about the performance of schools.

I have not come across a more satisfactory statistical solution for circumstances in which SUTVA is violated. In an interesting new paper, Manski provides some bounds on treatment effects in the presence of social interactions. Unfortunately, these bounds are often uninformative, since when SUTVA is violated random assignment to treatment arms does not identify treatment effects. Sinclair suggests using multi-level experiments to empirically identify spillover effects. This approach (which relies on multiple rounds of randomization to test if treatment effects are overidentified, as we would expect if there were no spillovers) is appealing, as the process of diffusion within networks is of great scientific interest. However, it does not help identify treatment effects when spillovers are present. Neither can we simply assume that effects estimated under SUTVA represent upper bounds on the true effects, because it is possible that interference across units intensifies the treatment effects rather than diluting them. Manski's paper seems like a useful foray into an open area of research. Let me know of other work on methods for causal inference in network-like situations where interference across units is likely.

Posted by Deirdre Bloome at November 21, 2009 4:50 PM