data (non)sharing

Jeremy Freese, an RWJ Health Policy Scholar at IQSS this year, sent me this amazing abstract (below) from the front lines of the replication movement, in psychology. On the same topic, but different discipline, don't miss Jeremy's "Reproducibility Standards in Quantitative Social Science: Why Not Sociology?" (find the pdf at his homepage) forthcoming, Sociological Methods and Research, July 2006. (I've written some on this topic too).

"The Poor Availability of Psychological Research Data for Reanalysis"By Wicherts, Jelte M.; Borsboom, Denny; Kats, Judith; Molenaar, DylanAmerican Psychologist. 61(7), Oct 2006, 726-728.

Abstract

The origin of the present comment lies in a failed attempt to obtain,through e-mailed requests, data reported in 141 empirical articlesrecently published by the American Psychological Association (APA). Ouroriginal aim was to reanalyze these data sets to assess the robustnessof the research findings to outliers. We never got that far. In June2005, we contacted the corresponding author of every article thatappeared in the last two 2004 issues of four major APA journals. Becausetheir articles had been published in APA journals, we were certain thatall of the authors had signed the APA Certification of Compliance WithAPA Ethical Principles, which includes the principle on sharing data forreanalysis. Unfortunately, 6 months later, after writing more than 400e-mails--and sending some corresponding authors detailed descriptions ofour study aims, approvals of our ethical committee, signed assurancesnot to share data with others, and even our full resumes-we ended upwith a meager 38 positive reactions and the actual data sets from 64studies (25.7% of the total number of 249 data sets). This means that73% of the authors did not share their data.

Posted by Gary King at October 11, 2006 7:38 PM