15 February 2010
I recently read The History of Statistics: The Measurement of Uncertainty before 1900, Stephen Stigler's excellent recounting of the early development of statistical theory and methods. I highly recommend this book to readers of this blog. You are sure to enjoy Stigler's engaging prose, which recount the struggles and triumphs behind techniques we now employ regularly.*
Stigler's insights are of particular interest for social scientists, as a central puzzle in his book is why social scientists waited so long to adopt statistical methods (lagging behind astronomers by almost a century). A major stumbling block for 19th century social investigators was the combination of relatively weak theories of social behaviors with "the plethora of potentially influential factors," great inherent heterogeneity, and limited data. Social scientists were reluctant to classify as error those deviations representing true but unmeasured or poorly understood patterns in human activities. In contrast, astronomers had strong theories regarding planetary motions and were able to compare their predictions with observed facts, thus gaining confidence in the use of statistics and probability to estimate quantities of interest and quantify associated uncertainty. Given continued struggles to sort through the myriad causes and consequences of social behaviors and organizations, modern social scientists will read with great interest about how influential 19th century thinkers understood their problems and attempted to solve them.
* When describing Adrien Marie Legendre's "invention" of least squares, Stigler notes that "the word minimum [makes] five italicized appearances [in Legendre's paper], an emphasis reflecting his apparent excitement" and the reader cannot help but share in this excitement -- what a great idea it was to minimize the sum of squared errors!