24 January 2011
We hope that you can join us for the first Applied Statistics Workshop of the term this Wednesday, January 26th when we will be happy to have Maya Sen from the Department of Government. She will be presenting joint work with Adam Glynn, also in the Department of Government. You will find an abstract below. As always, we will serve a light lunch and the talk will begin around 12:15p. (Note that this talk is not on the website schedule yet due to technical issues.)
“Female Socialization: How Having Daughters Affect Judges’ Voting on
Women’s Issues” (with Adam Glynn)
Department of Government
CGIS K354 (1737 Cambridge St. map)
Wednesday, January 26th 12 noon
Social scientists have long maintained that women judges might behave different than their male colleagues (e.g., Boyd et al. (2010)). This is particularly true when it comes to highly charged social issues such as gender discrimination, sexual harassment, and the status of gender as a suspect classification under federal law. Less studied has been the role that a judge’s family might have on judicial decision making. For example, we may think that a male judge with daughters might have different views of gender discrimination and sexual harassment than a male judge without any daughters. This paper takes a look at the question causally by leveraging the hypothesis that, conditional on the number of total number of children, the probability of a judge having a boy or a girl is independent of any covariates (Washington 2008). Looking at data from the U.S. Courts of Appeals, we find that conditional on the number of children, judges with daughters consistently vote in a more liberal fashion on gender issues than judges without daughters. This effect is particularly strong among Republican appointed judges and is robust and persists even once we control for a wide variety of factors. Our results more broadly suggest that personal experiences — as distinct from partisanship — may influence how elite actors make decisions, but only in the context of substantively salient issues.
UPDATE (1/25/11): Correct a typo in the abstract. Judges become more liberal on gender issues with daughters, not more conservative.
18 January 2011
A few months ago, I read this review of instrumental variables in political science by Allison Sovey and Don Green. I enjoyed it tremendously, so I was pleased to see that it just came out in the American Journal of Political Science. The (mis)use of instruments in the political science literature has been driving me crazy, so I'm hoping that Sovey and Green's article will help raise the bar. In some ways, the dataset they created shows that things are already getting better. More and more articles are offering justifications for their choices of instruments and more articles are using "just-identified" models to avoid the "embarrassment of riches" problem that comes from using multiple instruments. On the other hand, a plurality of articles still fail to give any justification for their instruments, so there is a long way to go. Hopefully compliance with the check-list Sovey and Green provide will become the new standard in the literature.
But wait, there's more...
Sovey has another paper (with Peter Aronow) that may be the future of instrumental variables. It is now well known that IV set-ups generally identify a local average treatment effect (LATE) which is rarely the quantity of interest for researchers. Aronow and Sovey show how to recover sample average treatment effects by estimating compliance scores for each unit (even in the face of two-sided non-compliance!) and then using these weights to estimate the treatment effect if every unit in the sample had complied with their treatment. This idea strikes me as very smart. It also strikes me as crazy, but possibly crazy enough that it might just work. If I ever find an instrument I actually believe for a problem I actually care about, I'll be trying this out.
13 January 2011
I just read an interesting article in the New Yorker (hat tip to John Sheffield) that gives an entertaining introduction to the so-called "decline effect" in scientific discovery. Apparently, at least a few scientists have had trouble reproducing the large effect sizes of initial studies on various topics (I'm shocked, shocked!). Some argue that the trend is general -- the initial studies on any topic will find big effect sizes that will be harder and harder to replicate over time. My guess is that this is simply a story about the problems of searching for "significance," but people interviewed in the article offer other explanations as well, including the possibility that nature is out to get scientists by teasing them with results and then making them go away.
"But wait," you say, "wouldn't the discovery of the 'decline effect' also be subject to the decline effect?" Is this yet another situation where people focus on apparently confirmatory cases while ignoring cases that don't confirm their hunch? I'm hoping "yes", if only because it would be deliciously ironic.
And if all this doesn't make you depressed about the durability of published findings, try this gem by John Ioannidis.
5 January 2011
You say you wanted an update on that ESP paper where a professor of psychology “time-reversed” some classic experiments? The New York Times has you covered. Want to see more discussion of null hypotheses and Bayesian analysis in the NYT? Also covered:
Many statisticians say that conventional social-science techniques for analyzing data make an assumption that is disingenuous and ultimately self-deceiving: that researchers know nothing about the probability of the so-called null hypothesis.
The so-called null hypothesis? Take that, Fisher! Oh, there’s more:
Instead, these statisticians prefer a technique called Bayesian analysis, which seeks to determine whether the outcome of a particular experiment “changes the odds that a hypothesis is true”…
Also, the last paragraph of the story seems very relevant:
So far, at least three efforts to replicate the experiments have failed.
The Economist has an article that has made the rounds on the Monkey Cage and with Drew Conway. The article bemoans the glut of PhD degrees granted in the U.S. and cries oversupply. It points to the rise of post-docs and adjuncts as cheap labor in the education and research markets. While the article does paint a perhaps overly bleak portrait of the current academic environment and the potential for those who leave it, I think the reaction has been a little strange. Since it’s slightly off-topic, I’ll banish most of my argument below the fold.
From Josh Tucker at the Monkey Cage:
Like it or not, academia is a meritocracy. It may be a highly flawed meritocracy susceptible to overvaluing labels or fads of the day, but ultimately tenure is bestowed on those who earn the respect of their peers, and the more of your peers that respect you, the more job offers you are going to get and the more money you are going to make.
And yet, tenure and meritocracy are goals at odds. The average age of professors when they received tenure is 39 in the United States, meaning that many will spend roughly half of their academic life with tenure. I don’t think it is a stretch to say that academic merit ceases to be a relevant criteria for employment after tenure. Of course, it matters for further professional success, but that is not what’s on the table. The most recent generation of scholars are likely to live quite long and productive lives, meaning tenure looms large for budgets.
To bring to back to sports, baseball teams do not hire players for life. The certainty of tenured faculty positions shifts most of the uncertainty in the academic labor market to graduate students and junior faculty. Does this mean we should reduce the number of PhD candidates we admit? Not necessarily, but we should not ignore the fundamental cracks in the academic system that we have seen in the last economic downturn. The academy (and especially the way we teach) will change drastically in the next 10-20 years. We would be wise to be ahead of those change instead of trying to catch up.
We could, though, try to reduce the information asymmetries that exist for potential PhD students. But it is hard to deny this sentence from the Economist article:
The interests of academics and universities on the one hand and PhD students on the other are not well aligned.
Though, even when they are aligned you see problems due to selection effects: professors who are dispensing advice are the ones who made it. Very few potential graduate students understand the market before they decide to earn a PhD. This might actually be getting better due to (of all things!) the often insipid and almost always anonymous job-rumour mongering forums that have cropped up for various disciplines. Even when people are attempting to give honest advice, they fail to anticipate impending problems. When I applied to graduate school 5 years ago, people made the job market sound like the land of milk and honey. You just go pick your job off the tree! No one anticipated the crash in the number of available jobs, or at least they did not reveal this information to me. This problem exists even more strongly for law schools as they are unaware of the life at top law firms and the likelihood of receiving such jobs. Yet law students have more paths available to them after they graduate compared to PhDs. The paths to success in academia are fairly limited to the traditional model.
And lastly, this story from Drew:
On my first day of graduate school one of my professors said, “Congratulations on being accepted to the program. While most people will not understand it, you have one of the greatest jobs one the planet. People are going to pay you to think, and I think that is pretty cool.”
I heard this numerous times as well and may have even said it myself. And yet, I think there is an inherent tension between this idea and Josh’s assertion that we succeed by convincing others of our ideas. The most surprising thing I have learned in graduate school, embarrassingly enough, is that we are not in the business of creating ideas, we are in the business of selling ideas and, in some sense, selling yourself. I think potential PhD students should know this.