A good author response should help the reviewers understand the work, as submitted. In practice, it is rare that a response will successfully win a philosophical argument with the reviewers (e.g., if the reviewers simply do not find a result interesting). A more useful goal for a response is to increase confidence that the results presented in the paper are sound. For instance, a reviewer may be concerned about the correctness of a design or experiment because the paper simply omits a relevant detail, perhaps for space. A helpful response would present the missing information needed for the reviewer to evaluate correctness of the design or experiment. Please be succinct in writing your response and comply with the length limit of 1000 words. Going over this limit will harm the chance the paper gets accepted.
Some key items to consider include:
- Focus first on the questions raised in the comment by the reviewers collectively (if present), followed by specific questions in individual reviews, and any major factual misunderstandings. The reviewers will be asked to identify the questions that have the most bearing on their decision. You are not expected to respond to every point in the reviews.
- Factual errors: Although uncommon, reviewers do occasionally misunderstand a fact critical to a paper. For instance, a reviewer might be mistaken about the configuration of a test system that has a first-order impact on the experimental results. In this case, a good response would politely clarify this detail, and, if present, remind the reviewers where in the submission this detail was presented (or apologize for the omission). From there, you may wish to summarize the implications of this detail on the understanding of the results.
- Be "prefix optimal": put the most important clarifications first (generally what is explicitly requested in the reviews or from the most reviewers), and more minor points, or points that seem important to only one reviewer, toward the end.
- Keep the response within the length limit and be succinct: It is critical to answer the key questions in 1000 words.It can be helpful to write short, bullet-style responses to minor questions that can be answered briefly (and easily searched by reviewers). Make the response easily searchable, with reviewer IDs and keywords: Reviewers may wish to search (i.e., Ctrl+F) for a specific point during discussion. Help the reviewers advocate for your work.
- New data: It is ok to include new data in your response if the data directly addresses a concern raised in the reviews and if the length of the response does not exceed the length limit of 1000 words. For instance, in a log-structured file system, a reviewer may be concerned about how long the log cleaner ran during the experiments. It would be ok to report this data for experiments already in the paper, or perhaps include an additional stress test.
- The reason most response instructions prohibit new data is to avoid changing the scope of the submission at the end of the review process.