•  
  •  
 

Publication Date

9-8-2017

Article Type

Full-Length Paper

Abstract

Abstract

Objective: To ensure that resources designed to teach skills and best practices for scientific research data sharing and management are useful, the maintainers of those materials need to evaluate and update them to ensure their accuracy, currency, and quality. This paper advances the use and process of outside peer review for community resources in addressing ongoing accuracy, quality, and currency issues. It further describes the next step of moving the updated materials to an online collaborative community platform for future iterative review in order to build upon mechanisms for open science, ongoing iteration, participation, and transparent community engagement.

Setting: Research data management resources were developed in support of the DataONE (Data Observation Network for Earth) project, which has deployed a sustainable, long-term network to ensure the preservation and access to multi-scale, multi-discipline, and multi-national environmental and biological science data (Michener et al. 2012). Created by members of the Community Engagement and Education (CEE) Working Group in 2011-2012, the freely available Educational Modules included three complementary components (slides, handouts, and exercises) that were designed to be adaptable for use in classrooms as well as for research data management training.

Methods: Because the modules were initially created and launched in 2011-2012, the current members of the (renamed) Community Engagement and Outreach (CEO) Working Group were concerned that the materials could be and / or quickly become outdated and should be reviewed for accuracy, currency, and quality. In November 2015, the Working Group developed an evaluation rubric for use by outside reviewers. Review criteria were developed based on surveys and usage scenarios from previous DataONE projects. Peer reviewers were selected from the DataONE community network for their expertise in the areas covered by one of the 11 educational modules. Reviewers were contacted in March 2016, and were asked to volunteer to complete their evaluations online within one month of the request, by using a customized Google form.

Results: For the 11 modules, 22 completed reviews were received by April 2016 from outside experts. Comments on all three components of each module (slides, handouts, and exercises) were compiled and evaluated by the postdoctoral fellow attached to the CEO Working Group. These reviews contributed to the full evaluation and revision by members of the Working Group of all educational modules in September 2016. This review process, as well as the potential lack of funding for ongoing maintenance by Working Group members or paid staff, provoked the group to transform the modules to a more stable, non-proprietary format, and move them to an online open repository hosting platform, GitHub. These decisions were made to foster sustainability, community engagement, version control, and transparency.

Conclusion: Outside peer review of the modules by experts in the field was beneficial for highlighting areas of weakness or overlap in the education modules. The modules were initially created in 2011-2012 by an earlier iteration of the Working Group, and updates were needed due to the constant evolving practices in the field. Because the review process was lengthy (approximately one year) comparative to the rate of innovations in data management practices, the Working Group discussed other options that would allow community members to make updates available more quickly. The intent of migrating the modules to an online collaborative platform (GitHub) is to allow for iterative updates and ongoing outside review, and to provide further transparency about accuracy, currency, and quality in the spirit of open science and collaboration. Documentation about this project may be useful for others trying to develop and maintain educational resources for engagement and outreach, particularly in communities and spaces where information changes quickly, and open platforms are already in common use.

Keywords

research data management, GitHub, peer review, academic libraries

Article Correction History

The HTML and PDF versions of this article were corrected on September 11, 2017 to change the Creative Commons license from CC BY-NC-SA to CC 1.0 Public Domain Dedication at the authors' request.

Acknowledgments

The DataONE educational modules were developed by a team of members from the CEE Working Group in 2011-2012. The peer review process was completed in 2015-2016 with support and feedback from past and present members of the DataONE CEO Working Group, as well as others from the DataONE team.

Funding Statement: DataONE is supported by US National Science Foundation Awards 08- 30944 and 14-30508, William Michener, Principal Investigator; Matthew Jones, Patricia Cruse, David Vieglais, and Suzanne Allard, Co-Principal Investigators.

Disclosure: The authors report no conflict of interest.

Corresponding Author

Heather Soyka, Kent State University, P.O. Box 5190, Kent, OH 44242, hsoyka@kent.edu

Rights and Permissions

Copyright Soyka et al. © 2017

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons 1.0 Public Domain Dedication.

Appendix1_1114.pdf (94 kB)
Appendix 1. Email request to individual reviewers

Appendix2_1114.pdf (119 kB)
Appendix 2. Rubric for reading outside reviewer comments

Share

COinS
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.