26 November 2009
I went to law school before I ended up as a graduate student, so I read with some interest a recent essay by Vanderbilt Law Professor Herwig Schlunk entitled "Mamas, don't let your babies grow up to be...lawyers" (an online version is at the Wall Street Journal's Law Blog).
Maybe the title gives it all away, but the gist is that a legal education might not always pay off. While I wholeheartedly agree with this, I'm less enthusiastic about the author's methodology. The author essentially constructs three hypothetical law students: "Also Ran," a legal slacker who attends a lower-ranked law school; "Solid Performer," a middling kind of person who attends a middling kind of law school; and "Hot Prospect," a high-flying and well-placed law student. The essay then more or less "follows" them through their legal "careers" to see if their discounted expected gains in salary match what they "paid" in terms of opportunity costs, tuition, and interest on their student loans. (I know, I'm using a lot of air quotes here.) Unsurprisingly, a legal education isn't a very good investment for any of the individuals.
What's interesting about the paper is that it's essentially an exercise in counterfactuals -- what Also Ran would have earned after going to law school, what Hot Prospect would have made had she not gone, etc., etc. To that extent, it's very fun think about. But, on the flip side, that's kind of what it is -- a thought experiment. Maybe an interesting extension would be a do an empirical causal analysis -- maybe matching pre-law undergraduates along a slew of covariates and then seeing how the "treatment" of law school affects or does not affect their earnings. I'd certainly find that a lot more persuasive (although I imagine that the kind of data that you'd need to pull this off would be well-nigh impossible to collect).
Last Thursday, I posted about the recent government recommendations regarding breast cancer screening in women ages 40-49. At least one of you wrote me to say that one of my calculations might have been slightly off (they were), and so I did some more investigation on this issue, as well as on new recommendations on cervical pap smears. (Sorry --it took
me a few days to get around to all of this!)
To back up a second, here's what the controversial new recommendations (made by the US Preventative Services Task Force) say:
So all of this got me thinking that this could maybe be a straightforward application of the "rare diseases" example of Bayes' Rule (which many people see in their first probability course). I did some (more) digging around in one of the government reports, and here's
how the probabilities break down:
Now bear with me while I go through the mechanics of Bayes' Rule. For women in their 40s, here are the pertinent probabilities:
P(cancer) = 1/69
P(no cancer) = 1-1/69 = 68/69
P(positive|no cancer) = 97.8/1000
P(negative|no cancer) = 1 - 97.8/1000 = 902.2/1000
P(positive|cancer) = 1 - 1/1000 = 999/1000
And for women in their 50s:
P(cancer) = 1/38
P(no cancer) = 1-1/38 = 37/38
P(positive|no cancer) = 86.6/1000
P(negative|no cancer) = 1 - 86.6/1000 = 913.4/1000
P(positive|cancer) = 1 - 1.1/1000 = 998.9/1000
The probability we are interested is the probability of cancer given that a woman has tested positive, P(cancer|positive). Using Bayes' Rule:
P(cancer|positive) = P(positive|cancer)*P(cancer)/P(positive)
P(cancer|positive) = P(positive|cancer)*P(cancer)/P(positive|no
We now have all of the moving parts. Let's first look at a woman in her 40s:
P(cancer|positive) = (999/1000*1/69)/(97.8/1000*68/69+999/1000*1/69)
and for a woman in her 50s:
P(cancer|positive) = (998.9/1000*1/38)/(86.6/1000*37/38+998.9/1000*1/38)
All this very simple analysis suggests is that mammograms do appear to be a less reliable test for younger women. Whether these recommendations make sense is another matter. Insurance companies might use these recommendations as an excuse to deny coverage for women with higher than average risk. In addition, as some of you noted, a 13% risk is nothing to sneeze at, and it's much higher than the 1/69 rate for women in their 40s (though comparable to a woman's lifetime 12% risk). Lastly, I also refer folks to Andrew Thomas's post , where he discusses the metrics used by the task force and notes that the confidence interval for women in their 40s lies completely within the confidence interval for women in their 50s.
I also did some very brief investigation regarding the new cervical cancer guidelines. For those of you unfamiliar with this story, the American College of Obstetricians and Gynecologists recently issued recommendations that women up to the age of 21 no longer receive pap test and that older women receive paps less often -- also advice contrary to what women have been told for decades.
It was much harder to pinpoint the false positive and false negative rates involved with pap tests (a lot of medical jargon, different levels of detection, and human and non-human error made things confusing). I did manage to find this article in the NEJM. The researchers there looked at women ages 30 to 69 (a different subgroup, unfortunately, from the under 21 group), but they do report that the sensitivity of Pap testing was 55.4% and the specificity was 96.8%. This corresponds to a false negative rate somewhere around 44.6% and a false positive rate somewhere around 3.2%. (Other references I've seen elsewhere hint that the false negative rate could be anywhere from 15 to 40%, depending on the quality of the lab and the cells collected.)
The other thing to note is that cervical cancer is very rare in young women and, unlike forms of cancer, it grows relatively slowly. According to the New York Times, 1-2 cases occur per 1,000,000 girls ages 15 to 19. This, combined with the high false negative rates, resulted in the ACOG recommendations.
My sense is that the ACOG recommendations are on more solid footing, but if people have comments, I'd be keen to hear them.