November 2008
Sun Mon Tue Wed Thu Fri Sat
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29

Authors' Committee


Matt Blackwell (Gov)


Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Recent Comments

Recent Entries



SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
The Education Wonks
Empirical Legal Studies
Free Exchange
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science



Powered by
Movable Type 4.24-en

« October 2008 | Main | December 2008 »

23 November 2008

Which Crash Was Most Like This?

Which past market environment was most like today's? This post shows that in terms of correlation to returns, the closest of all major markets to the S&P 500 in the last year was the Nikkei in 1991.

The S&P 500 peaked in mid-October of 2007, and has been down, down, down since then. Common comparisons include the Nikkei in 1990, the Dow Jones in 1929, and the NASDAQ crash of 2000.

But did those downturns really feel like today's? The plot below shows weekly returns overlaid for these four bear markets. All the plots start at the (then) all-time week-ending high of the market index in question.

They're not really all that similar. The 1929 and 2000 crashes both fell much more dramatically from their all-time highs initially. In contrast, this crash started slowly, gradually accelerating its decline until those dark days of August and September 2008. One depressing fact is that we're actually worse now than we were 58 weeks after the crash of '29!

Are there any precedents for this kind of decline? To answer that I scraped all week-by-week returns for all major indices, U.S. and international, from Yahoo! Finance. Then I programmatically scanned each index's past for a 58-week period where the returns have high correlation with the S&P 500's returns over the past 58 weeks since its October high. The winners, in order of correlation:

  • The Nikkei 225 from March 1991 is the closest, and the big winner. This is interesting because it is the cusp of the Nikkei's so-called ``second wave'' of declines, after it had stabilized following its better-known 1990 crash.
  • Germany's DAX 30 starting September 2000. The European markets all had a more pronounced downturn in the wake of the tech bubble crash than did the U.S. indices besides the NASDAQ.
  • The S&P 500 in May 1969 and in July 1973. Both of these were the crest of major bear markets, the latter owing to an oil crash.

In particular, the ebbs and flows of the Nikkei in 1991 are eerily close to those of the S&P 500 in the last year!

Although no one knows what our future holds, all but one of these indices went on to rally in the following two years. Perhaps it is some consolation that none declined much further, and most gained. The S&P 500 in 1969 recouped all of its losses by two years later. The worst of the bunch was the German DAX 30, which was about flat at its trough after two years.

Posted by Kevin Bartz at 11:22 AM

21 November 2008

Estimation of the Stereotyped Ordered Regression Model

While reading Xiaogang Wu and Donald Treiman's paper entitled "Inequality and Equality under Chinese Socialism: The Hukou System and Intergenerational Occupational Mobility" in American Journal of Sociology (2007, 113: 415-445) , I was directed to a technical paper written by John Hendrickx (2000), describing how to use "mclgen" and "mclest" in Stata to estimate the Stereotyped Ordered Regression Model (SOR) in social mobility studies.

SOR is similar to conventional ordinal Logit models, but with a scaling metric to scale the effects of the independent variables on the dependent variables so that the effects of an independent variable vary by the values of the dependent variable. In addition, SOR does not assume strict ordering among values of the dependent variable, which is perfect for studying occupational mobility as occupation is orderable but without strict order. Another desirable property that SOR has is that it specifies an inheritance parameter measuring intergenerational occupational immobility, i.e., the extent to which father and son have the same occupation.

These features make SOR appear to outperform ordinal Logit models in social mobility studies.

Click here to consult Hendrickx' paper for more details of the SOR model and the syntax of using "mclgen" and "mclest" in Stata.

Posted by Weihua An at 9:40 PM

19 November 2008

Election Wrap-up: Ballot Design

I like the noise of democracy.
--James Buchanan

There has been quite a bit of popular and scholarly interest in the mechanics of voting over the last decade, especially after the 2000 Florida Presidential election threw the concepts of butterfly ballots, residual votes and chads into the spotlight. The recount of the U.S. Senate race in Minnesota between Norm Coleman and Al Franken has brought the voting-error fun right on back. Minnesota Public Radio has compiled a list of challenged ballots for you to judge (via kottke). You can even use the Minnesota state statues governing voter's intent. I think the write-in for "Lizard People" is one of the best.

It is refreshing to see that in spite of all of the attention toward electronic voting problems, the old paper method can still make a mess. Things have changed a bit since the blanket ballots of the nineteenth-century, but ballot design still has quite a few problems. The most obvious case is the butterfly ballot of Palm Beach County in 2000 which almost certainly changed the outcome of the presidential election (see Wand, et al (2001)). Laurin Frisina, Michael Herron, James Honaker, and Jeff Lewis recently published an article in the Election Law Journal about undervoting in Florida's 13th Congressional District, a phenomenon they attribute to poor (electronic) ballot design. Other examples abound.

The good folks at AIGA put together an interactive guide for designing ballots and the problems with current designs. A lot of these suggestions are really spot on and would help to solve a lot of the errors in the Minnesota ballots. Especially important are the "if you make a mistake..." guidelines. This was posted at the New York Times in late August, which seems to me to be plenty of time for registrars to get these issues worked out. On the other hand, some of the Minnesota ballot problems do seem to transcend clear design. Depressingly, this probably brings a smile to faces of anti-plebian elites.

If you are a sucker, like me, for images of old ballots, you can find plenty of old California ballots at the voting technology project. Melanie Goodrich put this together. The real gem of this collection is the Regular Cactus Ticket of 1888.

Posted by Matt Blackwell at 2:10 PM

18 November 2008

NYT article on measuring racial bias

In today's paper, the NYT reports on an interesting debate between two groups of researchers regarding studies on unconscious racial bias (``In Bias Test, Shades of Gray''). The discussion centers around the usefulness of an online test, the Implicit Association Test, which measures how quickly respondents associate ``good'' or ``bad'' words with blacks or whites. How useful are such tests? It does seem crude as metric for racial bias (try it yourself here). But I suspect that they have raised awareness and deserve credit for involving a wide audience. Yet despite its timid recommendations and disclaimers when the results are displayed the test could also be misleading: what if you're characterized as racially bias (but are not)? What if you're characterized as unbiased (but are and should be told)?

Posted by Sebastian Bauhoff at 12:03 AM

17 November 2008

Glynn on "Assessing the Empirical Evidence for Mechanism Specific Causal Effects"

Please join us Wednesday, November 19th, when Adam Glynn--Government Department--will present his research, "Assessing the Empirical Evidence for Mechanism Specific Causal Effects". Adam provided the following abstract:

Social scientists often cite the importance of mechanism specific causal
knowledge, both for its intrinsic scientific value and as a necessity for
informed policy. In this talk, I use counterfactual causal models to re-assess
the empirical evidence for two oft cited examples from American and comparative
politics: the voting habit effect that is not due to campaign attention and the
effect of oil production on the likelihood of civil war onset that is due to
the weakening of state capacity. Utilizing decompositions of direct and
indirect effects, I discuss a number of identification strategies, and
demonstrate through sensitivity and bounding analysis that the evidence for the
aforementioned examples is weaker than is typically understood.

The applied statistics workshop meets at 12 noon in room K-354, CGIS-Knafel (1737 Cambridge St) with a light lunch. Presentations start at 1215 pm and usually end around 130 pm. As always, all are welcome and please email me with any questions

Update: Adam provided this paper as background for his presentation

Posted by Justin Grimmer at 7:13 PM

Interest in computer science is volatile

Reading an NYT article about the dearth of women in computer science, I was struck by this figure, which shows the percentage of college freshmen who say they might major in computer science.computer_science.png The article focuses on the fact, clearly visible from the figure, that women are increasingly underrepresented in computer science since the 1970's and early 1980's, when computer science really started taking off as a discipline.

What also struck me, however, was how volatile the baseline interest in the field has been. I was in college in the late-1990's, when majoring in CS was definitely viewed as a practical and lucrative thing to do, and I'm not surprised to see that interest has fallen off since then. But the fall-off shown here was much steeper than I would have imagined. Have enrollments declined at that rate as well?

Even more surprising to me was that there had been an earlier, equally dramatic boom-and-bust cycle. I knew from watching Triumph of the Nerds that PC sales really took off around that time, and I know about movies like Tron and WarGames, which came at the peak of the earlier wave shown here. But I didn't know there was such a steep drop-off in interest then either. Was that one because of the collapse of a tech bubble too?

Two more questions:

Does anyone want to chime in on why women are less and less represented in CS since the early 1980s? My thought was that professionalization of education in general, and hardening of ideas about who works in the IT profession, would be leading causes. There were a few theories in the NYT article (subtle messages from families, the rise of a very male gaming culture) but it seemed like there was a lot more to be said.

Do any other disciplines have enrollments this volatile?

Posted by Andy Eggers at 5:00 PM

10 November 2008

Conditions under Which Observational Studies Produce Comparable Causal Estimates

In the latest issue of Journal of Policy Analysis and Management, Thomas Cook, William Shadish and Vivian Wong wrote a paper proposing three conditions under which experiments and observational estimates are comparable based on their review of 12 recent within-study comparison studies. It is a little bit confusing, at least to me at first glance, to use "conditions" rather than "designs" here, as what the authors are really arguing is under three different types of research designs estimates from observational studies are comparable to causal estimates. More specifically, they suggest that:

1) regression discontinuity (RD) estimator produces similar effect estimates to experimental ones;

2) when intact group matching is used to minimize pre-test differences in at least outcome measures between the experiment and comparison populations, estimates from observational studies are trustworthy; and

3) when selection process into treatment is completely or plausibly known and could be properly measured, statistical procedures like propensity score matching can provide unbiased estimates.

As you can see, these three claims are based on selected published or to-be-published studies. But publication bias may lead them to overstate these claims, which in this case means observational studies with estimates comparable to experimental ones are disproportionally likely to be published than those without comparable estimates, and so how accurately or confidently we can rely on these claims to evaluate the comparability of estimates from observations studied remains ambiguous. In addition, this issue also relates to what standards we are using to judge comparability. If the standards are fuzzy, our judgment will be fuzzy to some extent as well. But overall, I appreciate the authors' enormous efforts on tracing recent literature on this topic and the resulted paper is full of wisdom.

When I am finishing this post, I realize that this paper was actually presented at our applied statistics workshop last October. But here comes the official version of the paper. And I think this is a very important topic that is worth a revisit.

Thomas Cook, William Shadish and Vivian Wong. 2008. "Three Conditions under Which Experiments and Observational Studies Produce Comparable Causal Estimates: Findings from Within-Study Comparisons", Journal of Policy Analysis and Management, Vol. 27, No 4, 724-750.

A previous paper distributed at the applied statistics workshop:

Posted by Weihua An at 11:23 AM

7 November 2008

Imai on "Identification and Inference in Causal Mediation Analysis"

Please join us this Wednesday, November 12th when Kosuke Imai will present "Identification and Inference in Causal Mediation Analysis". Kosuke is currently a professor in the Department of Politics at Princeton University and an alum of the Harvard Government Department. He has provided the following abstract for his talk:

Causal mediation analysis is routinely conducted by applied researchers in a variety of disciplines including communications, epidemiology, political science, psychology, and sociology. The goal of such an analysis is to investigate alternative causal mechanisms by examining the roles of intermediate variables that lie in the causal path between the treatment and outcome variables. In this paper, we first prove that under the assumption of sequential ignorability, the average causal mediation effects are nonparametrically identified. This identification result contrasts with previous studies which have concluded that the nonparametric identification of average causal mediation effects requires an additional assumption. Second, we show that under the same sequential ignorability assumption the average causal mediation effects can be identified in the linear structural equation model commonly used by applied researchers. Some practical implications of our identification result are also discussed. Third, we consider a simple
nonparametric estimator of the average causal mediation effects and derive its asymptotic variance. Fourth, we offer sensitivity analyses in both parametric and nonparametric settings so that researchers can examine the robustness of their empirical findings to the violation of the sequential ignorability assumption. Finally, we analyze a randomized experiment from political psychology to illustrate the proposed methods.

A paper for the talk is available here .

The applied statistics workshop meets at 12 noon in room K-354, CGIS-Knafel (1737 Cambridge St) with a light lunch. Presentations start at 1215 pm and usually end around 130 pm. As always, all are welcome and please email me ( with any questions

Posted by Justin Grimmer at 11:09 AM

6 November 2008

Would banning candy cigarettes reduce smoking prevalence? And would such a ban ever get past the courts?

It seems that people who have a ``lifetime history of candy cigarette use'' may be more likely to have ever smoked (Klein et al). Some countries like Canada, the UK, Ireland, Norway, Finland and Australia apparently believe that there is a causal link and already ban this type of candy. I think there are good reasons to believe candy cigarettes may have an influence on children and there are qualitative studies that suggest mechanisms like attitudes towards smoking. They certainly look like the real deal and might even build brand recognition. Check out this sample of German candies (DKFZ: 41). The middle one says "filter tipped", "king size" and features camels. Makes you wonder how they present sugar content in place of tar.


Anyway this makes me wonder what standards we must meet to make a plausible case for regulation. At least in the US there are strong barriers to regulating anything that may be construed as limiting ``commercial speech''. Sure enough historically institutions like the Federal Trade Commission had a hard time getting such policies past the courts, and providing sufficient evidence on causal links is a critical factor. For example in the case of regulating TV ads of high-sugar foods to children, establishing causality was one of the main barriers to implementation (the others were political and practical, as Mello et al write).

What are the hopes (fears) for a ban on candy cigarettes? To me it seems difficult to credibly argue that candy cigarette use leads to smoking later in life. If there is a causal link it will be hard to establish empirically (random draw of candy sticks, anyone?) and even harder to meet the high legal standard. I wonder how courts weigh quantitative versus qualitative arguments on such issues. Or will we only have regulations for issues where we can identify causal relations?

PS: Wikipedia says that North Dakota banned candy cigarettes from 1953-1967. Maybe we will see an empirical evaluation soon?


Klein et al (2007) "History of childhood candy cigarette use is associated with tobacco smoking by adults" Preventive Medicine 45(1): 26-30

Mello, M et al (2006) "Obesity -- The New Frontier of Public Health Law" N Engl J Med 354(24): 2601-2610

DKFZ (2008) "Rauchende Kinder und Jugendliche
in Deutschland - leichter Einstieg, schwerer Ausstieg
" [in German only, lists the countries that ban candy cigarettes. The candy cigarettes picture above appears on page 41.]

Posted by Sebastian Bauhoff at 5:00 PM

5 November 2008

R Animations for Teaching Statistics

Here is a neat R package that can be used to create animations for teaching a wide variety of topics in statistics: survey sampling, bootstrap, probability theory, just to name a few.

Also, for those not as interested in implementing the animations directly in R, there's also a web page with everything already done for you!

Posted by John Graves at 11:13 AM