November 2005
Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      

Authors' Committee

Chair:

Matt Blackwell (Gov)

Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries

Categories

Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

Archives

Notification

Powered by
Movable Type 4.24-en


« November 15, 2005 | Main | November 17, 2005 »

16 November 2005

Fun with bad graphs

Just read this nice entry on Andrew Gelman's blog about junk graphs . Somebody complemented the entry by posting a link to another site by Karl Broman in the Biostatistics department of Johns Hopkinson. In case you missed this please take a look. We all make these mistakes, but it's actually really funny...


Posted by Jens Hainmueller at 2:09 PM

Experts and Trials II: True Opinion & Slant

John Friedman

I ended my last post by showing, in the context of the brief model I sketched, what the optimal outcome would look like. In practice, though, the court suffers from two problems.

First, it cannot conduct a broad survey, but must instead rely on those testimonies presented in court. Each side will offer an expert whose "true opinion" is as supportive of their argument as possible, regardless of whether that expert is at all representative of commonly accepted views in the field. Second, the court cannot distinguish between an expert's true opinion and her "slant." Experts probably suffer some cost for slanting their views away from their true opinions, so one should not expect most slants to be large. But the legal parties will look to pick experts who suffer as little a cost from slanting as possible, so that, in equilibrium, the slants could be quite large.

Given these strategies from the legal parties, what does the court see? Each side presents an expert (or slate of experts) with the most favorable combination of "true opinion" and "slant." Even if the court could disentangle the two components of testimony, the court would only see the endpoints of the distribution of "true opinions" among the potential pool of experts. But since they cannot even distinguish the slant, the court actually sees only a noisy signal of the extremes of the distribution.

Finally, I have already argued that the experts chosen will be those most able (or willing) to slant their opinions, so that the ratio of signal to noise – or of "true opinion" to "slant" - for the experts will be very low, in expectation. When the court performs the required signal extraction problem, very little signal remains. Because of the optimizing action of each party, the court will draw very little inference from any of the witnesses in many cases, ironically nullifying the effect of the efforts of the experts. No one deviates from this strategy, though; if one side presented a more representative expert, while the other played the old strategy, the evidence would appear lopsided.

I noted in my last post that the "first-best," or socially optimal solution, would be for the court to collect a representative sample of the opinions of experts for their decision. Even when the parties present their own experts, each side would be better off if they could somehow commit not to use "slant" in their expert's opinions, since the decision in the case would be less noisy. But the structure of the problem makes such an agreement impossible.

Jim is correct when he remarks that, given the adversarial nature of the legal system, expert testimony could not happen any other way. We should not celebrate this fact, though; rather, we should mourn it. We are stuck in a terrible equilibrium.

Posted by James Greiner at 4:59 AM