This is a really interesting discussion and you have very insightfully hit on the major issues.

We are running benchmarks on a set of test proteins once every two weeks or so to test new methods we are continually developing. We could post the results, but as Mod sense suggests progress would not appear to be fast (the problem is hard, so most new ideas don't end up being big steps forward. as suggested below, when we do make a big advance, this is documented in a scientific paper, but I appreciate these are probably not very accessible to a general audience.

here is a possibility: in preparation for casp next summer, we are trying to assemble all of our new methods into a structure prediction pipeline that we are testing on a set taken from previous casp experiments. once this is in place, i can report to you how the performance of the new combined method does relative to the casp8 approach, and try to break this down to identify what the main advances were

---

This is a really interesting discussion and you have very insightfully hit on the major issues.

We are running benchmarks on a set of test proteins once every two weeks or so to test new methods we are continually developing. We could post the results, but as Mod sense suggests progress would not appear to be fast (the problem is hard, so most new ideas don't end up being big steps forward. as suggested below, when we do make a big advance, this is documented in a scientific paper, but I appreciate these are probably not very accessible to a general audience.

here is a possibility: in preparation for casp next summer, we are trying to assemble all of our new methods into a structure prediction pipeline that we are testing on a set taken from previous casp experiments. once this is in place, i can report to you how the performance of the new combined method does relative to the casp8 approach, and try to break this down to identify what the main advances were

Sounds great...but maybe you could add something about how much and what we are trying to do along with that. I am not a Chemist or Scientist or anything like that, I was a firefighter and am now a computer inventory guy who loves computers. That last part means I also do ALOT of 'other duties as assigned' stuff! Where I am going is that I have no idea what "casp" or "casp8 approach" means or how it affects us crunchers. Maybe you could get an intern or something to break this down into non academic language and along with that provide some stats that say something like...10,000 combinations were tried last month with 150 of them sent on for future testing, the rest were sent back for a refinement of the process. Of the 150, 23 were in the area of flu research(inserting whatever is correct here), etc, etc. I do realize that if you told us that all of our crunching came out negative it might hurt your future of getting more people to come crunch, but I do not believe that to be correct. I think people want to help in some way to make this World a better place. WCG has projects like Nutritious Rice, Clean Energy and they have hundreds of thousands of people crunching for them. I just think a bit of self promotion by this site might help to bring more people here. Thanks for listening, love the idea of Rosetta!!!!

Hey, thanks all for the comments. 

As ever the hard bit is to encourage (seduce?) people with “slow but steady wins the race” but that appears to be most research (at least as far as I can tell.)

I like the fact the news thread on the front page gives details on the improvements made as they are applied (for instance the new energy function optimizations for 2.00 of minirosetta.) I’m not sure if there is but I didn’t spot it, if there is a place that lists, effectively the ‘change log’ of the various projects. Thinking like mikey said about this being about self promotion it could be quite fun/interesting if there was a page (as part of a ‘status’ page, it doesn’t all have to be numbers....) with the changes so someone looking at it for the first time thinks:-

“Ooooow, new protein-protein docking protocol in 1.80 and support for Disulfides in 1.82” (don’t even know what that is.... but it is in Wikipedia so assuming it is accurate maybe a link to it...)
“wow this project rocks with all these new features and long words.”

Ok so that was what I was thinking when I looked at those threads ;) and to be fair I got them from clicking on the news archive which anyone else ‘could’ do. However I knew they were in those thread and first time users probably won’t realise.

Details of the improvements with casp would be very kewl as it’s very much a real world link in the sense it’s interacting with other projects and the research community at large. Plus the chance to see some details on how/why it’s improved can only be a good thing!