SoCG'17 – Guidelines for the evaluation of submissions by sub-reviewers

Relevant topics

See the call for papers for a non-exhaustive list of topics.

SoCG’17 web site: http://socg2017.smp.uq.edu.au/socg.html

EasyChair SoCG’17 site: https://easychair.org/conferences/?conf=socg17

 

Criteria


When evaluating a paper, one should address the following (interrelated) issues. A paper with a high score should in general score high on several of them. These criteria are general and apply to all types of papers: theoretical, applied, or experimental.


 

Relevance. In what respect is the paper relevant to computational geometry? Is it directly relevant for the design, use, analysis, or implementation of geometric algorithms? Does it have indirect implications for the development or the theory of geometric algorithms? Does it contribute to the mathematical foundations of discrete or combinatorial geometry and/or algebraic geometry and/or computational topology?

 

Foundational/conceptual contribution. Does the paper introduce a new model, new notion, new definition, new approach, novel implementation, novel application? Note the significance and reasons for this novelty (and note the absence of such a novelty, if applicable).

 

Technical development. Does the paper involve

 

Relation to open problems.  Does the paper solve completely/partially an open question?  How important is this question (central/important/interesting/legitimate/stupid)?  How much effort has been invested previously in solving it and by whom?

 

Social interest in paper.  Is it potentially interesting to the whole community of computational geometry, to a major field, to everyone in a restricted area, or interesting only to the authors?

 

How will it contribute?  Does it have the potential to influence or affect future work? Does it have the potential to have an impact on application domains? Is it/can it be important in other fields/subjects or have a wider influence?

 

Type of contribution. Is it a

 

 

Some thoughts on how to evaluate papers of different types

When evaluating a paper, it is important to keep in mind that there are different types of papers and that the criteria by which papers of one type is evaluated may be different from those used for papers of another type. Moreover, the importance of a common criterion may vary from type to type. 

 

Among the paper types that we expect to encounter are papers focusing on:

·         mathematical foundations,

·         algorithmic design and complexity and/or lower bounds,

·         experimental & implementation issues, and

·         applications.

Hybrid papers, which consist, for example, of both an algorithmic design and analysis part and an experimental part are of course also common.

 

Below is an attempt to characterize these paper types and to specify the main criteria by which they should be evaluated.

Mathematical Foundations. A typical paper will contain theorems and proofs describing new results in discrete or combinatorial geometry, and/or in algebraic geometry, and/or computational topology. The paper will primarily be evaluated on its technical depth, the importance of the results, the elegance of the solution, the connection of the problem studied to computational geometry and topology, and the potential future impact on algorithm development.                   

Algorithmic Complexity.
A typical paper will contain algorithms, theorems, proofs and/or lower bounds describing new results on computational geometry problems. The paper will primarily be evaluated on the (mathematical or computational) relevance and importance of the problem studied, its technical depth, the elegance of the solution, and the potential future impact of the results and/or the proposed new methods and techniques.                                                    

Experimental & Implementation.
A typical paper will make a clear contribution to the implementation and evaluation of geometric algorithms, such as exact, approximate, and/or algebraic computation, algorithms engineering, and/or the experimental evaluation of competing algorithmic approaches. The paper will primarily be evaluated on the completeness and the expected impact of the proposed implementation, the soundness of the experiments, the quality and quantity of testing, and on the general amount of knowledge gained.                                       

Application.
A typical paper will describe the modeling and algorithmic choices made when developing or adapting computational geometry techniques for an application area. The paper will be primarily evaluated on the soundness of the modeling decisions, the ingenuity of the solution, the effectiveness of the proposed method, and the expected impact in the application area.  One might also consider the lesson learned regarding the applicability or suitability of computational geometry tools to the specific area.

 

Score ratings and their meaning

 

Score

Interpretation

Strong accept (+3)

An enthusiastic accept. An excellent paper — advances the field in an important way — well written and makes it easy to understand the significance of the results. People should definitely attend the talk. This should be among the top third of the accepted papers. I would fight strongly for this paper.

Accept (+2)

A solid contribution. I feel I learned something worthwhile from this paper. I would want to go to the talk. Should be in the middle third of accepted papers.

Weak accept (+1)

A weak vote for acceptance. A reasonable contribution to an interesting problem  — or maybe the contribution is good but the authors don't express it well — or maybe it's a good paper, but the subject area is marginal for the conference. Not a stellar result, but worth accepting. Should be in the bottom third of accepted papers. 

Borderline (0)

Ambivalent. Probably publishable as a journal paper in a medium journal, but a bit too specialized or too incremental for SoCG. Or perhaps it has nice ideas but is too preliminary, or too poorly written. Please try to refrain from giving this score.

Weak reject (-1)

A competent paper, but not of sufficient interest/depth for SoCG. A weak to moderate vote for rejection, but I concede that other people see some merits in the paper.

Reject (-2)

Too preliminary / badly-written / making-such-a-minor-improvement-on-such-an-esoteric-topic. I would fight to have this paper rejected from the conference.

Strong reject (-3)

A poor paper, unsuitable for any journal. Trivial and/or non-novel and/or incorrect and/or out of scope.

Confidence ratings and their meaning

Grade

Interpretation

Expert (5)

Expert. Consider me an "expert" on this paper. I understand it in detail. I know the field, and I am perfectly sure about my judgement; I have checked and understood all proofs.

High (4)

High. I am fairly familiar with the area of this paper, and have read the paper closely enough to be reasonably confident of my judgment.

Medium (3)

Medium. I have read the paper carefully and understood the main ideas, but I'm not very confident of my judgment on it.

Low (2)

Low. I am not an expert. My evaluation is that of an informed outsider. I have some idea of what this paper is about, but I'm not all that confident of my judgment on it.

None (1)

Null. (to be avoided...!)  Please do not use this except in extreme circumstances.

 

The appendix

From the CFP: All details needed to verify the results must be provided. Supporting materials, including proofs of theoretical claims and experimental details that do not fit in the 15-page limit should be found in the appendix. The appendix will be read by the reviewers at their discretion and will not be published as part of the proceedings. Thus the paper without the appendix should be able to stand on its own.

 

Ethical Issues

Submitted papers are confidential. We must not distribute them, or use them for our research. Similarly, your reviews, grades, and confidence scores must be kept confidential.  Submissions should be judged solely on the basis of the submitted extended abstract.  You may have a personal bias on some papers. The reasons are many — personal/professional ties to authors, you or your student are just working on the same problem, etc. Only you can judge such a bias, and decide if you don't feel comfortable reviewing the paper. In this case, mention a conflict of interest to the PC.

 

Boris Aronov and Matya Katz, SoCG’17 co-chairs

socg17@easychair.org

 

Acknowledgment: We thank Günter Rote and Monique Teillaud for sharing their SoCG'05 and SoCG’08 instructions, and Marc van Kreveld and Bettina Speckmann for their suggestions regarding papers of different types.