February 2007
Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28      

Authors' Committee

Chair:

Matt Blackwell (Gov)

Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries

Categories

Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 4.24-en


« February 20, 2007 | Main | February 22, 2007 »

21 February 2007

How do I cheat with potential outcomes?

As some folks know, I'm on the legal academic job market this year. My job talk paper is on the application of the potential outcomes framework for causation to legal matters, particularly anti-discrimination issues that arise in litigation. As I've presented the framework, I've highlighted one of its advantages as being the fact that much of the hard work of separating covariates from intermediate outcomes and balancing covariates can (and should) be done without access to the outcome variable. The idea is that without access to the outcome variable, it is harder for a researcher (or, God forbid, an expert witness) to model-snoop, i.e., to fit model after model until finding one that "proves" a pet theory.

In a few schools, reaction to the claim of increased objectivity has been chilly. Skeptics have said, in essense, "I don't know enough about statistics to argue with you, but I'm REALLY SURE that your method is just as manipulable as, say, regression, even if you don't have access to the outcome variable when you do the hard work." The skeptics have then asked me to tell them how to manipulate the potential outcomes framework (i.e., to tell them why they are right and I am wrong), assuming no access to the outcome variable.

Any ideas on this? I'm able to think of one way it can be done (although the results of "my" way would not be nearly as bad as those from model-snooping), but I'd prefer not to stifle any comments folks might have by putting forth my own thoughts.

Posted by James Greiner at 3:33 PM