Interesting discussion. I have only crunched a few of the CEP2 units, about 30 so far, but have yet to see an error. Only my quads and a dual processor Xeon have gotten the units. They run Vista, Win7 and Linux Mint. They run in a mixed environment. The uploads are 24 megs. Tough project, but I will let it run.

Cheers

Dear Dataman,


I wholeheartedly support this - the collaboration with and support by the IBM team is awesome - both for crunchers and us scientists. There is A LOT of work going into a project like CEP2. Our thanks to all crunchers goes without saying.


Yes, I guess we haven't explained this topic properly: The decision to buy our own servers and storage arrays was due to the limited capacity of the WCG servers. Since our results are bigger than usual, WCG could have only accommodated an upload of 400 results per day which would have rendered the whole project pointless. Our setup is at least in part modeled after the one of WCG and our research computing department is in close contact with our IBM friends on this. To give you some numbers: our servers have a load capacity of 200MB/sec (with some more in reserve) and the actual load is currently at about 10MB/sec (with the 40% throttle). So we are wide open for business .
A positive side effect of uploading directly here to Harvard is that we save the additional traffic of the detour via IBM.


I am not quite sure whether this analogy is useful but this is what I would reply: We have a project of a certain nature which we offer for your kind consideration. We hope to attract as many users as possible, but if a user finds our project too demanding and prefers to crunch on a different one, then that's ok, too. We don't compete with the other projects as they are equally worthwhile. If people like the challenge and the goal of CEP2, they will hopefully keep supporting us. We obviously don't want to upset anybody and are hence committed to fixing all problems that are in our hands - but we cannot do anything about the inherent cost of these calculations. The 'opt-in' is a tool to spare users with low-end machines a predictably frustrating CEP2 experience. We don't want to sell our project to people who will be unhappy with it.


I apologize if you found this point offending. I do computational science for a living and have done so for a while now - and I learn new stuff on a daily basis. Considering that this forum is also used by many newbies, some important general comments didn'€™t seem out of place.


Most definitely - I do however doubt that you will find many crunchers that have more experience as computational quantum chemists than our team. And it requires this knowledge of the code and of the science to put together a useful project.


I was more referring to people like my labmate who couldn't get CEP2 workunits. But yes, MrKermit's sudden data-flood (which brought our original server to its knees one night) helped us to significantly improve our system, which is great. We live and learn .


Excellent - we are in agreement then. No need to bury CEP2 after all .


Yes, the result size is indeed unfortunate. We went to some lengths to cutting it as much as possible (including reducing the main binary array from double to single precision). What is left is pretty much bare bone. If we were to dump the remaining key results there would be no point in generating them in the first place. The throttle has actually little to do with the upload behaviour...


Correct - hence the default setting and 'opt-in'.


Well, we've been tailoring Q-Chem for nearly a year before the launch to make things as smooth as possible on the grid. Q-Chem by itself is a high-performance program package with 135MB of source code which has been developed, expanded, and optimized for 20 years by professional developers and legions of academic contributors from the finest universities around the world... Believe it or not, but many smart people have put some serious thought into this code .

Bottomline from a result oriented perspective (which one could argue is a relevant one in WCG) is that CEP2 works! It produces many results every day, and we make progress with our research - despite all the bumps in the road! And for that we are very grateful to the WCG crunchers!

Best wishes

The CEP Team

Thank you very much for your timely, honest and informative response, cleanenergy.

OK, you have made a convert. When BOINC@Australia returns from its current crunching assault at Spinhenge, I will put CEP2 back on my crunch list and run it at a minimum level in rotation with the other projects I support.

You really should work on the file size/bandwith problem though. It is a killer.

Cheers and good luck ...

I have two servers with 8 cores each. When more than about 5 or so cores are running and a new CEP2 work units starts it creates some sort of an issue and all the 8 currently running processes are restarted from their last check point. This costs only a few seconds or minutes on most projects, but the CEP2 project can lose several hours, apparently.

This was happening so much that I was losing sometimes 20 CPU hours per day, so I had to turn on the restriction to only have one CEP2 work unit at a time. It didn't seem right to waste so much valuable CPU time when it could be used on the other projects.

This is a shame. I like the idea behind this project.

Dear steveleg,

absolutely - all uploads are useful to us and our research.

In the first case the timeout prevents that you have to crunch on a workunit for too long and cuts the more time intensive parts of the calcs short - the results up to that point however are perfectly fine.

The second case indicates a problem in the numerics, which is very common in computational chemistry. Everything before that is perfectly fine and it also tells us that the molecule in that workunit is a toughie.

Best

Your CEP team