CASP16

Message boards : Rosetta@home Science : CASP16

To post messages, you must log in.

AuthorMessage
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109015 - Posted: 18 Mar 2024, 8:35:25 UTC
Last modified: 18 Mar 2024, 8:36:03 UTC

CASP (Critical Assessment of Structure Prediction) experiments are held every two years. Recent rounds have seen dramatic increases in modeling accuracy, resulting from the introduction of deep learning methods: In 2018, for the first time, the folds of most proteins were correctly computed [1]; in 2020, the accuracy of many computed protein structures rivaled that of the corresponding experimental ones [2]; in 2022, there was an enormous increase in the accuracy of protein complexes [3].

We have seen the beginning of what deep learning methods may achieve in structural biology. In addition to further increases in the accuracy of protein complexes, methods are being developed for RNA structures, organic ligand-protein complexes, and for moving beyond single macromolecular structures to compute conformational ensembles. Accurate computational methods together with experimental data also offer the prospect of probing previously inaccessible biological systems. CASP has expanded its scope to provide critical assessment in all these areas.

CASP is only possible with the generous participation of the experimental structural biology community in providing suitable targets: A total of over 1100 targets have been obtained over the previous CASP rounds. We are now requesting targets for the 2024 CASP16 experiment.

The timeline for the 2024 CASP requires that targets are submitted starting now and until July 1. We would like to hear from you as soon as possible if you may have something suitable or have suggestions about other target sources. In order to maintain rigor, the experimental data for a target must not be publicly available until after computed structures have been collected. For assessment, CASP requires the experimental data by August 15, but the data can remain confidential after that. Target providers are invited to contribute to papers [11-15] for a special CASP issue of the journal Proteins.

ID: 109015 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109017 - Posted: 18 Mar 2024, 8:52:19 UTC

Timetable

April 2, 2024 - Start of the registration for CASP16 prediction experiment.
April 16, 2024 - Start of the testing of server connectivity ("dry run" for server predictors).
May 1, 2024 - Release of the first CASP16 modeling targets.
June/July 2024 - Early bird registration for the December CASP16 conference.
July 31, 2024 - Last date for releasing targets.
August 31, 2024 - End of the modeling season.
Early September 2024 - Collection of abstracts describing the methods used in CASP16.
August-October 2024 - Evaluation of predictions.
November 2024 - Invitations to groups with the most accurate models and the most interesting methods to give talks at the CASP16 conference.
November 2024 - Program of the conference finalized.
December 2024 - CASP16 Conference (tentatively Nov 30 - Dec 3).

ID: 109017 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109018 - Posted: 18 Mar 2024, 8:57:03 UTC
Last modified: 18 Mar 2024, 8:59:30 UTC

Organizing committee
John Moult, CASP chair and founder; IBBR, University of Maryland, USA
Krzysztof Fidelis, founder, University of California, Davis, USA
Andriy Kryshtafovych, University of California, Davis, USA
Torsten Schwede, University of Basel, Switzerland
Maya Topf, Centre for Structural Systems Biology, Hamburg, Germany

Scientific advisory board
Minkyung Baek, Seoul University, South Korea
David Baker, University of Washington, USA
Charlotte Dean, University of Oxford, UK
Nick Grishin, University of Texas, USA
Andrzej Joachimiak, Argonne National Lab, USA
David Jones, University College, London, UK
John Jumper, Google Deepmind, London, UK

ID: 109018 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109113 - Posted: 14 Apr 2024, 19:24:25 UTC
Last modified: 14 Apr 2024, 19:25:41 UTC

CASP16 registration is in full swing, with 83 groups already registered. The first CASP16 target will be released on Wednesday, May 1 at 9 am Pacific Time.

We will start a 'dry run' on checking format compliance and server connectivity on Tuesday, April 16. If you plan to enroll a server in CASP16, please do so as soon as possible so that we can test your server in the dry run and ensure flawless data transfer during the CASP16 prediction season. If you registered a human-expert group, you could also test the format of your model during the dry run by submitting a test prediction when the appropriate server test target is released.

The dry run will be conducted in four stages.

In the first stage (Tuesday-Thursday), we will test servers registered for protein and RNA tertiary structure prediction. We will release two test targets from the previous CASP - T1104 (protein) and R1117 (RNA) - and send their sequences to the registered servers. We expect the servers to return predictions in the requested TS format (https://predictioncenter.org/casp16/index.cgi?page=format#TS). Please check the Server Queries link from the main CASP16 web page (which will be available on Tuesday) to see the status of queries to your server. In case you notice any problems, please check your registration settings and contact us if in doubt. If you do not see your prediction(s) as accepted in our server status page (link Server Prediction Status) or through the Model Viewer /My CASP16 Profile pages, then either your prediction did not reach us, or was rejected due to an unregistered submission email or format issues.

On Thursday, we will start the second stage with testing automated predictors of protein and RNA oligomers. We will send you a query for a protein homodimer (T1124), protein heterodimer (H1106), and an RNA homodimer (R1107).

We will continue the dry run on Monday, April 22, with testing EMA servers on the released oligomeric targets.

And finally, starting Wednesday, April 24 we will be testing format for RNA oligomers (R1107) and protein-RNA(DNA) complexes (M1189). We would assume that your server can do RNA oligomers if it is registered for RNA prediction, and can do protein-RNA modeling if it is registered for both categories. Please write to us if this is not the case.

ID: 109113 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109151 - Posted: 23 Apr 2024, 7:02:32 UTC

Dear CASPers,

0. A reminder that CASP15 season will start in less than 2 weeks. We have already 125 groups registered. Last week we started testing servers for the connectivity and format compliance. The first two stages of the testing are currently running (tertiary structure and assembly for proteins and RNA). Next week we will continue with the EMA servers and servers for hybrid targets (e.g., protein-RNA complexes).

1. Two-stage oligomeric prediction.
This prediction season we are introducing a two-stage oligomeric prediction. First, we will release a target without stoichiometry information, and then with it. This will be a pilot experiment on a real-life modeling without knowing the protein aggregation state in advance. We will allow five models in each stage, but as it is common in CASP, the assessment will concentrate on the top-ranked prediction (Model 1). This experiment will be conducted on selected targets where we have enough prediction time. The whole prediction time for this kind of target tandems will be four weeks, and we would like to hear predictors' opinion on how to split this time between the two stages (default: two weeks each). Please discuss this and other CASP-related issues on the CASP16 Discord channels: https://discord.gg/mhnQCTVE.

2. ColabFold models and MSAs.
In CASP15, AlphaFold2 models generated by Arne Elofsson /Claudio Mirabello and ColabFold served as a very useful starting modeling point for predictors and the reference point for assessors. For CASP16, CASP organizers approached ColabFold group, and they graciously agreed to make their models and MSAs of CASP16 protein targets publicly available after closing the server prediction window (typically 3 days after the target release). The data will be provided for targets with known stoichiometry at http://casp16.colabfold.com .

3. Model 6 (not Tesla).
It has been suggested that it might help the assessors to untangle MSA effects from method features if there was one additional model from each participating group, generated using a reference MSA. Exploiting the availability of ColabFold MSA, we ask human-expert groups with MSA-based approaches to generate a special 'Model 6' based on this reference MSA. This model should be submitted alongside the five regular models per target, and it will receive a special treatment in the assessment.

4. Collecting MSAs.
In CASP16 we will be asking predictors with MSA-based methods to submit MSAs corresponding to their CASP16 models through a separate gateway. These will be used by assessors for additional tests /analyses. So please have your MSAs handy.

5. Re-release of difficult targets for SRA-based modeling (time allowing).
In late June - early July, when big portion of CASP modeling will be done, we will ask Martin Steinegger's group to generate MSAs for selected targets on the huge Sequence Read Archive (https://cshperspectives.cshlp.org/content/early/2024/02/05/cshperspect.a041465.abstract). Once generated, we will ask predictors to build models using those MSAs similarly to what we asked for the ColabFold MSAs (item 3 above).

6. MassiveFold for the EMA stage_2 experiment.
Björn Wallner's massive sampling approach proved to be one of the most successful methods in CASP15. Guillaume Brysbaert from the University of Lille implemented this approach in MassiveFold (https://github.com/GBLille/MassiveFold), however it remains out of reach for many groups due to high GPU requirements. For CASP16, Guillaume offered to generate 6000-9000 predictions per target (except for bigger ones) and make these available to predictors. We plan to use these models in the second stage of the EMA experiment by asking accuracy estimators to pick the best 5 models from the dataset and send their IDs as the stage_2 EMA predictions. More details to follow.

7. Protein-RNA-DNA complexes.
We have been collecting potential targets for the future round of CASP for a couple of months now, and already have a good starting set of targets. Among these, we already have two protein-RNA-DNA complexes, and two more have been promised. We have announced before that we may have this type of complexes in CASP16, and now we know that we will. With this, we ask predictors who have methods that are able to handle protein-RNA-DNA complexes to be ready for this kind of targets.

8. Ligand targets.
In addition to CASP targets with incidental ligands, we have secured four sets of ligand targets from pharma companies. Since each of these pharma 'super-targets' includes multiple ligand-receptor systems (e.g., one of the sets contains 200+ ligands for the same receptor), we will allot much more time for their prediction than we do for regular targets. We plan on releasing these four ligand super-targets in the first week of May, and these will be available for prediction until some time in July. Please check our format page for the latest LG format details: https://predictioncenter.org/casp16/index.cgi?page=format#LG

ID: 109151 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 16 Jun 07
Posts: 23
Credit: 2,750,225
RAC: 13,916
Message 109156 - Posted: 23 Apr 2024, 16:04:44 UTC - in response to Message 109151.  

Dear CASPers,

0. A reminder that CASP15 season will start in less than 2 weeks. We have already 125 groups registered.


I tried but could not find the participants list. Is Rosetta@home participating? If so, on what dates should we put all efforts to the project? Would be fantastic if you provide your suggestions on this matter as you follow the process. Thank you!
i crunch for Ukraine. Join our team forums about Rosetta@home
ID: 109156 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109166 - Posted: 24 Apr 2024, 12:45:01 UTC - in response to Message 109156.  
Last modified: 24 Apr 2024, 12:47:06 UTC

I tried but could not find the participants list. Is Rosetta@home participating? If so, on what dates should we put all efforts to the project? Would be fantastic if you provide your suggestions on this matter as you follow the process. Thank you!


The list will be public at the beginning of competition.

If you see the list of organizators there is David Baker, the dad of Rosetta@home/IPD.
Usually IPD partecipate with 2 teams (automatic and manual simulations)
Sometimes, in the past, they distributed some works on R@H (with specific names, like "_casp_")
But it's years they do not

Obviously they, during the competition, use R@H protocols (and others)
ID: 109166 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 16 Jun 07
Posts: 23
Credit: 2,750,225
RAC: 13,916
Message 109213 - Posted: 1 May 2024, 21:48:16 UTC
Last modified: 1 May 2024, 21:48:26 UTC

There are already some results uploaded and we can see 40 participant names https://predictioncenter.org/casp16/servers_distribution.cgi?server=-1&target=51&register=Submit&action=process&view=table

I do not know under which name Rosetta team will be participating yet
i crunch for Ukraine. Join our team forums about Rosetta@home
ID: 109213 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109219 - Posted: 2 May 2024, 15:19:36 UTC - in response to Message 109213.  

I do not know under which name Rosetta team will be participating yet


Maybe this edition they decide to not partecipate.

But seems strange
ID: 109219 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 16 Jun 07
Posts: 23
Credit: 2,750,225
RAC: 13,916
Message 109238 - Posted: 9 May 2024, 1:26:03 UTC - in response to Message 109219.  
Last modified: 9 May 2024, 1:28:19 UTC

This looks like a list of participants and current results https://predictioncenter.org/casp16/status_n.cgi

And here https://predictioncenter.org/casp16/numbers.cgi it says number of groups registered 198
i crunch for Ukraine. Join our team forums about Rosetta@home
ID: 109238 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109262 - Posted: 20 May 2024, 15:14:43 UTC - in response to Message 109238.  

And here https://predictioncenter.org/casp16/numbers.cgi it says number of groups registered 198


The number of groups increased to 210
And the simulations program seems to be in full mode

CASP16: week 4 (May 20-26) (2024-5-19)
We plan to release 11 targets: 7 new(*), 3 returning Phase1(+), and one Phase2(++) /QA:QMODE3.

Monday: 3 targets
* protein heterooligomer and its subunit 1
+ T1206 (CAPRI)
++ T2207

Tuesday: 2 targets
* prot-rna-rna complex and two RNA subunits
+ H1208 (CAPRI)

Wednesday: 2 targets
* protein heterooligomer :: assembly only
+ H1215 (CAPRI)

Thursday: 2 targets
* protein heterooligomer :: assembly only
* prot-rna-rna complex and two RNA subunits

Friday: 2 targets
* protein heterooligomer :: assembly only
* protein monomer

CASP organizers

ID: 109262 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109386 - Posted: 15 Jun 2024, 15:27:59 UTC - in response to Message 109262.  

MassiveFold: H1217 has been posted; T1218 file is 120Gb

1. A note from Guillaume Brysbaert on H1217:
The number of recycles was reduced to 3 for all conditions. The afm_dropout_full_woTemplates_r3 condition was removed as it became redundant with the afm_dropout_full_woTemplates. The number of predictions per neural network model was reduced to 5, which should have resulted in 7x15x5=525 predictions. However, some predictions crashed resulting in 395 models in total.

2. Because of the delay with the MassiveFold data on H1217, we will allow an extra prediction day on this target. The new QA deadline is June 17.

3. Please note that the tarball for today's target T1218 is very large (120 GB) and it may take some time to pull it off the shared storage.

ID: 109386 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109395 - Posted: 20 Jun 2024, 15:37:52 UTC

week 8 (Jun 17-23) (2024-6-17)
This week we are releasing 20 targets.

Monday: 5 targets
* protein heterooligomer
* RNA (stage 0: predict stoichiometry)
* RNA (stage 0)
+,CAPRI H1233
++,CAPRI H2220

Tuesday: 4 targets
* monomer
+,CAPRI T1234
+,CAPRI T1235
+,CAPRI H1236

Wednesday: 4 targets
* protein homooligomer
* RNA with ligand
+,CAPRI T1237
++,CAPRI H2222

Thursday: 4 targets
* hybrid complex (protein-RNA-DNA)
* RNA with ligand
+,CAPRI T1238
++,CAPRI H2223

Friday: 3 targets
* protein single-sequence target (filament)
+,CAPRI T1240
++,CAPRI H2225

ID: 109395 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109438 - Posted: 8 Jul 2024, 13:49:56 UTC

Clarification about the RNA multi-stage targets (2024-6-28)
Dear RNA predictors,

In CASP16 we implemented a multi-stage prediction procedure for select targets. According to this procedure, we first ask predictors to generate models without knowing the actual stoichiometry (stage 0); and then - with this information provided (stage 1). For details on the target naming scheme you can revisit our post https://predictioncenter.org/news.cgi#478. Until recently, only protein targets were subject to this procedure. Now that we have oligomeric RNA targets, we stage release of these targets similarly to what we have been doing for proteins. In brief, in stage 0 (R0xxx targets) you can submit models in any stoichiometry state and report the predicted oligomeric state in the STOICH field of the model (see CASP16 format page). Then, in stage 1 (R1xxx targets) you should provide models in the specific oligomeric state reported to predictors in the Stoichiom. field of the Target List page. Please make sure that in stage 0 at least one chain of your model corresponds to that of the structural template of the target, and in stage 1 all chains are as provided in the structural template.

ID: 109438 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109439 - Posted: 8 Jul 2024, 13:50:39 UTC

CASP16: plans for week 11 (Jul 8-14) (2024-7-7)
1. Next week we will conduct the first format check of ligand predictions on L-targets. By the end of the week those who have already submitted their models should expect an email from us with the information about the compliance. These checks are non-obligatory courtesy format checks in the middle of prediction window to give predictors more lead time to fix their models, if needed. Deadlines for submitting predictions on L-targets remain unchanged: Aug 1-4.

2. Next week we are releasing 17 targets, and after that week we will be slowly throttling down target release. As it stands now, by the end of CASP16 we should reach our goal of 100 different targets in a CASP cycle.

Monday: 3 targets
* protein monomer
+,CAPRI T1270
++,CAPRI H2244

Tuesday: 3 targets
* RNA target
++,CAPRI H2245
++ T2246

Wednesday: 3 targets
* RNA target
* protein monomer
* protein filament

Thursday: 4 targets
*,CAPRI hybrid complex
* protein monomer
++ T2257
++ H2258

Friday: 4 targets
* hybrid complex
+ R1250, stage 1 RNA
+ R1251, stage 1 RNA
++,CAPRI T2259

ID: 109439 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109480 - Posted: 24 Jul 2024, 14:16:14 UTC

CASP16: week 13 (Jul 22-28)
1. Next week we are releasing 11 targets.

Monday: 2 targets
* RNA
++,CAPRI T2227

Tuesday: 2 targets
* hybrid protein-RNA complex
* homomultimer

Wednesday: 3 targets
+ RNA R1281
++ T2278
++ T2279

Thursday: 2 targets
* homomultimer (alternative conformations)
++ T2280

Friday: 2 targets
* homomultimer
++,CAPRI M1287 (protein-DNA)

2. We also plan to release two targets where predictors will be invited to submit an ensemble of models describing the position and relative orientation of domains connected by a flexible linker in a two-domain protein. Deadline for these targets will be set in the end of August. It is also possible that we will have another ensembles target this CASP. The announcements will go out next week.

3. The last week of July will be the last week of CASP16 target release. As of now, we have 8 targets lined up, two each day from Monday, July 29 through Thursday, August 1. Some last minute changes are possible but there will be no new targets after August 1.

ID: 109480 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109591 - Posted: 17 Aug 2024, 15:10:36 UTC - in response to Message 109480.  

Dear CASP16 Participants,

We are happy to say we are near to finalizing the location of the CASP16 conference in Punta Cana, Dominican Republic. The dates are as announced earlier: November 30 (evening) to December 3 (lunch time), 2024. We expect to open registration shortly.
The sessions will take place at a newly-opened conference venue.
Punta Cana is known for its wonderful beaches.

See you in Punta Cana this November 30!

ID: 109591 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1990
Credit: 9,472,254
RAC: 12,952
Message 109698 - Posted: 1 Sep 2024, 15:21:03 UTC

Dear Ligand Predictors,

As we are getting more experimental structures for evaluation, we have noticed that some targets were released with inaccurate or incorrect SMILES information. To address this, we will be re-releasing the targets with corrected ligand data tomorrow (four RNA and one protein). We understand that some CASP16 predictors may not be ready to return to their modeling desks immediately, so we have extended the deadline for these targets as much as possible (September 18). This extension will still allow for the evaluation of predictions and their analysis by the assessors before the assessors-organizers meeting at the end of September.

We apologize for the inconvenience.

CASP organizers

ID: 109698 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Rosetta@home Science : CASP16



©2024 University of Washington
https://www.bakerlab.org