Animal Cognition Researchers Take Crucial Step Toward Data Sharing

Researchers in animal cognition have just released a report on the steps they have taken over the past 20 months to move their field toward archiving and broader sharing of research data. Their pioneering work is all the more remarkable in that their willingness to be open in sharing data comes despite well-grounded concern among these scientists regarding the security of their labs and animals and the safety of their data. The researchers are hoping that their recognition of the need to overcome obstacles to data sharing will prove inspirational to other psychologists who have been reticent about data sharing despite facing much less risk.

Their report, “Data Archiving for Animal Cognition Research,” was recently published in Animal Learning and Behavior and is also available through the National Institute of Mental Health which partnered with scientists to produce the detailed framework for archiving and data sharing. (View the report at www.nimh.nih.gov/research/confsummaries.cfm or at www.brown.edu/psychology/anicog.) The report can easily serve as a model for others because it tackles most of the major issues surrounding data sharing, and takes account of state-of-the-art developments shaping data sharing internationally across all sciences. In fact, the report goes beyond consideration of electronic archiving of primary research data to consider the means to link – essentially seamlessly – a variety of research materials.

In the broad scientific debate over whether, when, what, and how to archive, some watershed issues have emerged. The position a field takes on each of them shapes the nature of scientific information sharing for that field. While those in other areas of psychology will need to develop their own stances with respect to each of these issues, the positions that those in animal cognition have taken now stand as a map of the crucial issues and as one well-reasoned pathway through those issues.

TO SHARE OR NOT TO SHARE
Acknowledging that researchers’ access to data from other investigators’ studies “is usually limited to the summaries provided in results and discussion sections of publications,” the animal researchers emerged (after passionate discussion pro and con) on the side of greatly expanded data sharing. Among the reasons cited for their support of increased data sharing are:

• increased opportunities for further analysis and interpretation of data;
• broader and potentially richer treatment of data including data that will not be published;
• stimulation of new theoretical insights;
• increased efficiency and quality of the research effort;
• enhanced tools for teaching, and more informed use of research in the formation of public policy.

They do not recommend that their own reasoning be unquestioningly adopted by other fields. Rather, they say that “[I]t is worthwhile for individual fields (and even sub-fields) to undertake organized efforts such as this that incorporate input from a significant number of the field’s members and other experts.” That approach, they say, “will ensure that archiving projects are designed in ways that satisfy each community’s needs and aims, will increase the likelihood that novel ideas about archiving will gain a hearing in some forum, and will enable fields to draw from the experiences of one another as their efforts progress.”

STANDARDIZATION IS KEY
The burden of arranging data in a format that will make it easily understandable to scientists not involved with the original data collection has long been cited as a reason for not sharing data. Those in animal cognition take the position that the eventual archiving of data should now be planned as part of the initial research design. That effort, as well as ease in accessing archived data, can be aided by standardization.

A good deal of the standardization work has been done. An international set of standards was put in place by the Open Archives Initiative (for more on OAI’s standards, go to www.openarchives.org). Those standards are gaining ever wider use by scientists in every discipline. Adoption of the standards is recommended by the report.

One key to the success of the standards is the inclusion of supporting information that enables competent use of the data. Two types of information in addition to the data themselves are needed: metadata and documentation. Metadata are the indexing guide to the data. They tell future users such things as who produced the data, when, where, species used, experimental paradigm, and keywords for accessing the data. In addition, a useful data set will include documentation that tells how the data were collected and analyzed, how the data are formatted, and also unusual details of the study that are essential to know if the data are to be used properly.

Because of the sensitive nature of some animal research data, those in animal cognition say that their documentation should also contain detailed information about the lab or field conditions under which the data were collected so as to make it amply clear to users that the data were collected in adherence with ethical, health, and treatment standards.

DISTRIBUTED RATHER THAN CENTRALIZED DATABASES
One of the watershed issues in archiving is that of who should maintain the archive. The practice in the social sciences has been for single institutions to maintain data on behalf of those in the discipline. Thanks in part to the OAI standards (which did not exist when the social science databases were created) those in animal cognition prefer that there be many databases all conforming to the same standards. At the most basic level, the report calls for individual scientists to archive their own data and make it publicly accessible. One suggested way to make this feasible is to add archiving costs to grant applications. The report notes in the report that the federal government is increasingly encouraging data sharing by federal grantees. Thus federal support of archiving through the normal granting process seems a practical device for advancing the practice. Universities, scientific societies and journals also should support archiving, according to the report.

Personal archiving is a first step that would lead toward aggregation of data sets at perhaps the departmental or university level and through online indexes. The movement toward sharing is seen as an evolutionary process that should be fostered by academic and scientific organizations, rather than mandated. For example, personal archiving should be considered in promotion and tenure decisions. Graduate training should incorporate instruction in archiving as a normal part of training in research design. Scientific societies should publish journal articles on archiving processes and research that incorporates secondary data analysis, develop protocols for citing archival data sets and even consider hosting archives in their fields. The report acknowledges that journal policies toward copyright and prepublication would need to be adjusted to encourage rather than discourage personal posting of data sets and articles when an article has been approved for publication. It is also suggested that scientific societies could foster expansion of data sharing by developing online listings of data sharing sites and their contents. The spirit of data sharing is captured in the follwing excerpt from the report: “[R]esearchers must recognize that science is a collective enterprise in which they have responsibilities both to fellow scientists and to the other segments of society that support and utilize their work. Therefore, the most basic product of research – data – should be made available to colleagues and to society at large. While researchers should have priority in access to and use of the data they worked hard to produce, they should not be able to control those data for an indefinite period.”

WHAT TO SHARE
Because the animal researchers’ preference is for individual responsibility in data sharing, they do not deem it realistic to reach extensively into the past for data. They suggest that the place to start archiving data is with current research. In general, they favor posting data sets when the article describing results has been approved for publication. They also point out that there has been no good outlet for data from research that is well designed but whose results fail to attain the level of significance required for publication. Such results may be as valuable in advancing knowledge as are studies with statistically significant results; therefore, individuals should post potentially useful, if otherwise unpublishable, data.

In general, data should be made available as soon as possible; however, some data are part of an ongoing research project. Again, the animal researchers do not support compulsory posting of these data. When the researcher intends to make further use of the data, it is reasonable to delay making the data available until such time as the researcher deems it appropriate. And although top priority goes to posting new data, the researchers believe it is of value to post past data that have proven their lasting worth in advancing the field.

The researchers take the position that access to data should be free and open, and that the archives should be user friendly. The intent is to encourage use. For this reason, they also believe that self-identification should not be a prerequisite for use of a database. On the other hand, they emphasize that users have certain responsibilities too. At the least, there should be proper citation when a database is used in a publication. If possible, the user should consult with the data producer. Where appropriate and feasible, secondary analysts of data may wish to co-author publications with the original collector of the data. The success of data sharing is dependent on building a culture of sharing. And that is most likely to happen when producers and users of data respect each other’s rights and privileges.

Animal researchers are particularly concerned about the proper treatment of video recordings of animals behaving in field and laboratory settings. Such recordings are valuable sources of information. But experience has demonstrated how open to misinterpretation such recordings can be. The animal researchers admonish themselves to be especially thorough in their provision of documentation so that the likelihood of misinterpretation and misuse can be minimized.

MEASURING PROGRESS
The blueprint for data sharing that is represented by the report is an important step toward routine data sharing among those in animal cognition. The framers of the report have set some milestones that will help them gauge progress. They believe that if a critical mass of participation can be achieved, it will result in the adoption of data sharing as a field-wide practice. If in the next five years at least 10 major animal cognition laboratories engage in regular data sharing, using the OAI standards and the procedures recommended in the report, this should be sufficient to drive permanent change.

A welcome hand in reaching the milestone has come from the libraries of several major universities that have taken it upon themselves to provide archiving services for faculty. MIT’s DSPACE (www.dspace.org) and the University of California’s eScholarship (http://escholarship.cdlib.org) are two examples. The movement toward university libraries handling the archiving function would seem to solve the problems of long-term preservation of data, of adaptation of data storage to new devices, and of distribution to new procedures. These are problems that individual researchers would have trouble handling on their own because they require constant adaptation to new media.

If more libraries take on the archiving function, it should help researchers across all the sciences begin to view archiving as a normal part of the research process. The Association of Research Libraries and its Scholarly Publishing and Academic Resources Coalition are trying to organize the research library community to do just that, and are taking other actions that are aimed at making data and the research literature more accessible. A white paper released by ARL/SPARC last summer (www.arl.org/sparc/IR/ir.html) has helped orient the research library community in that direction. The white paper has been followed up with a workshop that took place in October and with continuing coverage of archiving developments in the ARL newsletter.

Those in animal cognition are expecting to see substantial growth over the next five years not only in archiving, but also the number of published articles that are based on secondary data analysis. They are particularly interested in seeing research articles published by those at institutions that lack extensive animal research facilities, because one goal of archiving is to extend the opportunity to do animal cognition research to scientists who do not have direct access to animal research facilities. They also hope to see the publication of what might be termed crossover research – that is, the use of cognitive data from one animal species to inform research on animals from other species. The publication of such studies will be a strong indication that archiving is helping to expand the scope and depth of animal cognition research.

THANKS WHERE IT IS DUE
Movements within a field do not occur magically; they happen because individuals make the effort to move the field forward. And that is the case here as well. APS Fellow and Charter Member Russell Church of Brown University, who is well known among fellow psychologists not just for his research on time perception but also for his leadership in the Psychonomic Society and the Eastern Psychological Association, came to Howard Kurtzman, Chief of the Cognitive Science Program at the National Institute of Mental Health with a proposal to bring together leading researchers in animal cognition to explore the possibility of extending the role of data sharing in cognition research.

Their collaboration resulted in a two-day workshop last summer that was transformational. The workshop brought together not only researchers but also experts in archiving, publishing, policy, and law. While some researchers came with mildly favorable to neutral attitudes toward data sharing, others were adamantly opposed. The two days of intense deliberation resulted in a conviction shared by all that animal cognition needs data sharing regardless of the risks that might come from making research data publicly and freely accessible. Not one to lead with ideas alone, Church has been doing for some time what he hopes will become field-wide practice in the near future. Readers can get a look at this future by going to www.brown.edu/Research/Timelab.