Experimental Papers

Information about experimental procedures is relevant to the decision of whether or not to publish a paper reporting results from laboratory and field experiments and researcher conducted surveys. Detailed information of this type is also valuable to scholars who subsequently do related work or attempt to replicate results. Therefore, author(s) of such manuscripts should include with their submission sufficient material on the procedures to enable review. If accepted, additional material to facilitate replication and follow up analysis will be expected, and will be made available through the Quantitative Economics website. During the submission process, all this material will be password protected and available only to the editor and referees evaluating the manuscript with the understanding that the material will be used for the sole purpose of evaluating the submitted paper (and not, e.g., for research purposes). Examples of material to be included upon submission and upon acceptance are described below. Any impediments to providing such material at the acceptance stage should be indicated upon the initial submission. For manuscripts that are rejected, supplementary material will be removed if the author requests it.

Important procedural aspects should be explained at the submission stage (either in the body of the paper or an appendix as is appropriate for the particular paper and aspects). We leave the decision on what details to initially include to the author. If during the review process the editor or referees feel additional information is needed, requests for that material will be made, and may naturally cause delay in processing, hence we encourage as complete a submission as feasible.

If any accompanying materials, such as experimental instructions, are not written in English, then a translation should be provided.

Suggested information to provide for the review process:

1. The subject pool and recruiting procedures.
2. The experimental technology – when and where the experiments were conducted; by computer or manually; online, and so forth.
3. Any procedures to test for comprehension before running the experiment, including the use of practice trials and quizzes.
4. Matching procedures, especially for game theory experiments.
5. Subject payments, including whether artificial currency was used, the exchange rate, show-up fees, average earnings, lotteries and/or grades.
6. The number of subjects used in each session and, where relevant, their experience.
7. Timing, such as how long a typical session lasted, and how much of that time was instructional.
8. Any use of deception and/or any instructional inaccuracies.

Additional information for publication on the supplementary website.

A detailed appendix is required to facilitate replications and data analysis and should consist of whatever can be provided for these purposes. This will include further detail on the items above, and provision of original materials. The following are items that are desirable and typically expected, but further detail about what is needed in each case can be obtained from the coeditor handling the paper.

1. Detailed statement of protocols.
2. Samples of permission forms and record sheets.
3. Copies of instructions and slides and/or transparencies used to present instructions.
4. Source code for computer programs used to conduct the experiment and to analyze the data. This does not include compilers (such as ztree) that are publicly available.
5. Screen shots showing how the programs are used.
6. The experimental data together with adequate documentation of the format.

Reasonable judgment should be used. For example, if instructions for different sessions differ only slightly, then one sample of the instructions suffices, with the differences noted in a short accompanying document. These rules should be understood also to apply to surveys conducted by the authors. When the authors are not the primary source of the data, we require only the data, a statement of where it came from and the programs used to process it – detailed documentation of the procedures used by the original data providers about how it was collected is not required.

We understand that there may be a need for exceptions to the policy for confidentiality or other reasons. When this is the case, it must be clearly stated at the time of submission that certain data or other appendix materials cannot or will not be made available, and an explanation provided. Such exceptions require prior approval by the editors.