DISCLAIMER: As of today, it is widely believed that in general, human-expert, computer-aided protein structure prediction can be more powerful than fully automated predictions. Similarly, we believe that human-expert assessment can be significantly more accurate than automatic evaluation.

All CAFASP-2 evaluations were produced using fully automated methods. Consequently, there are a number of limitations and shortcomings in the evaluations presented below, including the lack of an expert interpretation of the numerical data. However, this was one of the goals of CAFASP: to not only evaluate fully automated predictions, but also carry out the evaluation using fully automated methods. These methods were described in advance before the experiment began, and participation in CAFASP implied that participants agreed with these methods.

One of the advantages of running CAFASP in parallel with CASP, is that CAFASP participants can also benefit from the human-expert assessment provided by the casp assessors. We believe that as of today, the human-expert assessment will in most cases be a better indicator of the servers' performance than the one presented below.

Although all evaluations in CAFASP-2 were performed by Fully Automated programs, the full responsibility of running the programs and present the results lied entirely on each of the categories' coordinators . The procedures applied by these programs were delineated in advance . Participation in CAFASP-2 implied acceptance of these procedures and rules. The results of the automatic evaluation are presented here and are independent of any other human assessment that may be carried out.

In addition to the automated evaluation of CAFASP-2 predictions, two comparative analyses of the automated results filed to CAFASP-2 with those filed at CASP-4 may be carried out. These are described at the bottom of this page.

CAFASP-2 EVALUATION RESULTS:

• Fold Recognition Evaluation Results (Fischer)
• Secondary Structure Evaluation Results (Rost).
• Homology Modeling Evaluation Rules and Results (Dunbrack). Preliminary raw data only.
• Ab initio Evaluation Rules and Results (Ortiz). Preliminary raw data only.
• Contacts Prediction Evaluation and Results (Valencia).

CAFASP-CASP PERFORMANCE COMPARISONS:

The two comparative analyses of the automated results filed to CAFASP-2 with those filed at CASP-4 that will be carried out are:

• 1. CAFASP-CASP automated evaluation comparison. After the CASP-4 predictions become available to all, we may apply to them the exact same procedures as those described above.
• 2. CAFASP-CASP human assessor comparison. The CASP-4 assessor of each category may apply his/her assessing procedures to both the CASP-4 and CAFASP-2 predictions.

It is obvious that a CAFASP-CASP performance comparison is not 100% fair for several reasons:

• The CAFASP-2 predictions are filed months before the CASP-4 predictions are filed. Thus, it is likely that for some cases, the availability of more sequence or structure entries in the databases may make a particular target easier to predict.
• The manual casp4 predictions can make use of the CAFASP results (but not vice-versa).