Critical Assessment of Fully Automated Structure Prediction


The goal of MR-CAFASP, is to evaluate the potential of fully automatic 3D protein structure prediction servers to provide valuable models during the structure determination process. MR-CAFASP is not another competition! MR-CAFASP is aimed at providing a set of blind-predictions in order to assess their utility in the experimental determination process. The focus will be on targets with no close homologue of known structure, where MR techniques can't easily be applied, either because a parent of known structure is not easily identified, or because the sequence-structure alignment is not reliable.


During previous LiveBench experiments it was discovered that at least in two cases, highly confident "in-silico" models significantly differed from the experimental structure. In both cases, the experimental structure was subsequently removed from the PDB, and in one case, the replacement entry contained corrected models very similar to the original "in-silico" prediction (see e.g. Bujnicki et al, "Fold-Recognition Detects an Error in the Protein Data Bank.", Bioinformatics, 18, 1391-1395, 2002 and Bujnicki et al. "Errors in the D.radiodurans large ribosomal subunit structure detected by protein fold-recognition and structure validation tools", FEBS letters, 525, 174-175, 2002 , and the crystallographers reply in the same issue). This lead to the question of whether in-silico models may be of help during the structure determination process - a very important issue for Structural Genomics.

This idea appeared in a previous CASP experiment, where predicted models were solicited for one target that could not be solved. In addition, D. Jones has also recently suggested that distant homology fold-recognition models may be used as Molecular Replacement phasing models (Jones, D. "Evaluating the potential of using fold-recognition models for molecular replacement.", Acta. Crys. D57, 1428-1434, 2001). D. Baker and colleagues have also described a method to generate structures using limited NMR data combined with in-silico procedures (Bowers, Strauss and Baker. J. Biomol. NMR, 18:311-318, 2000).


  • Would a predicted model be of help to better fit the chain into a low-resolution, hard to interpret, electron-density-map? Can a predicted model help detect shifts and errors in the initial tracing of an electron-density-map?
  • Can a predicted model be used as a phasing-model?
  • How can NMR benefit from an accurate predicted model?
  • Because many predicted models may not be accurate enough, is it worthwhile for the experimentalist to spend some time verifying this? That is, because models can be obtained automatically, they require no effort. But how much time does an experimentalist require to verify whether the model is useful or not? Is it worthwhile to spend this time, even if only very few predicted models are accurate enough?

Highly confident fully automatic predictions will be selected from the targets of PDB-CAFASP , before the experimental structure is released. From these, participating labs will be able to carry out a number of tests vis-a-vis the experimental data, when the latter becomes publicly available (e.g. structure factors) or with the collaboration of the experimentalists, if they are interested.


Because the experimental structures of many PDB-CAFASP targets will not be available in the near future, we have selected a number of targets from LiveBench which have (i) strong predictions obtained by the servers (i.e. scores well-above confidence thresholds) and (ii) Structure Factors available in the PDB.

Preliminary tests are being carried out on a number of these targets.


Experimentalists and predictors wishing to be involved in this experiment are invited to participate in its various components.

If you are interested in trying MR for a particular target in our list of selected LB targets, using the servers' predicted models and the available structure factors (but without cheating with the actual released coordinates), please let me (Daniel) know, so you can get the info.

If you have data for a protein, but no phases, and you would like to attempt distant-homology MR, please send me your protein sequence, to see if the servers can provide valuable predictions to be tried as phasing models.


  • Daniel Fischer
  • Leszek Rychlewski


  • Boaz Shaanan
  • David Jones
  • Kenji Mizuguchi
  • Chen Keasar
  • Janusz Bujnicki
  • David Baker


If you want to enquire about MR-CAFASP, email us: or