6 January 2006
Alchemists' gold. The perpetual motion machine. One might also think of cold fusion and warm superconductors. These are some of the great mythical aims of the so-called "hard" sciences. A few of these concepts have also been compared to attempts at ecological inference, the search for accurate predictions about the internal cell counts of a set of contingency tables (such as one for each precinct) when only the row and column totals of table are observed. The fundamental problem of ecological inference is, of course, that radically different internal cell counts can lead to identical row and column totals, and because we only get to see the row and column totals, we cannot distinguish among these different sets of counts. Another way of saying this is that the problem is impossible to solve deterministically (since the relationship between the cell entries and row and column marginals is not one-to-one), causing some to label ecological inference an "ill-posed inverse problem". In fact, without making some statistical assumptions, the estimation problem would not be identified, although it would be bounded because some values for the cell entries are are ruled out for each precinct's contingency table by the observed column and row totals (these are called "the bounds").
Ecological inference arises in the legal setting in cases litigated under the Voting Rights Act. Section 2 of the VRA prohibits a state or municipality from depriving a citizen, on account of race or ethnicity, of an equal opportunity to participate in the political process and to elect candidates of his/her choice. The Delphic statute has been interpreted to disallow districting schemes that have the effect of diluting minority voting strength. In practice, to succeed in a vote dilution claim, a plaintiff must almost always prove that voting in the relevant jurisdiction is racially polarized, meaning that whites vote differently from blacks who vote differently from Hispanics. Because the secret ballot prevents direct observation of voting patterns, expert witnesses are forced to attempt the dangerous task of drawing inferences about racial voting patterns from precinct-level candidate support counts (column totals) and precinct-level racial voting-age-populations (row totals).
A large literature exists on the ecological inference problem. Bizarrely, one constituency has rarely if ever contributed to this debate: the lawyers and judges who consume a great deal of what the literature produces. I'll be attempting to start to fill this gap in subsequent entries.