Guidelines for reviewers: How to review a data paper?

Data papers aim at describing a dataset made available to a larger community. Data papers are a scientific valuable production and should provide all required information for a large use of the data. The data paper should be completed by a metadata file that describes the dataset, and by the dataset itself, made available in an open repository.

Reviewers will carefully consider: (a) the quality of the manuscript, (b) the quality, completeness and reusability of the dataset, and (c) the relevance of the dataset and its potential contribution to the progress of science.

They will not review the whole data set, whose quality is under the responsibility of the authors (see our related blog post “Guidelines for authors: how to share your datasets?”).

Quality of the manuscript

Usual criteria for assessing manuscript quality including style, consistency, clarity. The review will address following questions:

• Is a DOI provided for the database and is the dataset accessible via the given identifier?
• Is the metadata template filled out as recommended?
• Do title and key message accurately reflect the content of the data paper?
• Is the Data Paper internally consistent, suitably organized and written in proper English?
• Are relevant non-textual media (e.g. tables, figures) appropriate?
• Have abbreviations and symbols been properly defined?
• Is the context of prior research properly described, citing relevant articles and datasets?

Quality of the dataset

Although publication of a data paper does not guarantee the overall quality of the dataset, there is a need to check whether suitable and reproducible methods have been used to obtain the data, and whether the data are displayed in a sensible way. The dataset must be a long-term resource, stable, complete, permanent and of good quality. The review will address following questions:

• Are the data logically and consistently organised? Are they easily readable and usable?
• Is anything missing in the manuscript or the data resource itself that would prevent replication of the measurements, reproduction of the figures or other representations?
• Are the methods used to generate the data (including calibration, code and suitable controls) described in sufficient detail and suitable to maintain of integrity of the dataset?
• Have possible sources of error (including methods, calculation and interpretation) been appropriately addressed in the protocols and/ or the paper?
• Are the data consistent internally and described using applicable standards (e.g. in terms of file formats, file names, file size, units and metadata)?
• Does the manuscript provide an accurate description of the data? And how to access them (e.g. link and/or data access policy)?
• Are the methods used to process and analyse the raw data appropriate? Are they sufficiently well documented that they could be repeated by third parties? Accepted formats are: 1) datasets, deposited additionally to scientific datasets in the repository; 2) links to online published papers; 3) a section in the body of the manuscript dedicated to material and method.
• Are the data files complete and match the description in the Metadata?

Utility and contribution of data set

• Does the data resource cover a scientifically important region(s), time period(s) and/or group(s) of taxa to be worthy of a separate publication?
• Is the dataset sufficiently original to merit publication as a data paper?
• Is there any potential of the data being useful in the future?
• Are the use cases described in the data paper consistent with the data presented? Would other possible use cases merit comments in the paper?
• Are all conclusions made in the data paper substantiated by the underlying data?
• Are the depth, coverage, size, and/or completeness of the data sufficient for the types of application or research questions outlined by the authors?

Contact us for more information at annforsci@inra.fr

Read the related posts on our blogs:

Annals of Forest Science promotes Open Science by publishing data papers

Guidelines for authors: how to share your datasets?

Publishing data papers in Annals of forest science: detailed guidelines for a smooth preparation and submission