I have looked over the result page and it seems as thou you are geting credit for the work completed not how long it took you to run it.

I am running fdupes to minimize some of the kernel buffers wasted by CEP2 among other things, such as saving 70% disk space.

please see other thread. cep2 provides hyperthreading opportunities by being sucha different animal, at least that's my thoery, and there's a lot of i7 9xx cores who can help assess/refine the slots, i think there's some potential in reducing page faults for CEP2 and the rest of the projects on large core counts.


http://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,30419

Dear Crunchers,

Thanks for your criticisms and suggestions (and of course for participating in our project!). We'd like to address some of the issues and concerns that were raised in this thread:


At present this is not unexpected for the following reasons:
1) The CEP2 load share is currently throttled to 40% of other projects. It will be increased incrementally to 100% over the next week or two. We opted for this warm-up strategy to safely test the stability of our servers, as this is the first time a WCG project uses its own hardware setup to collect results.
2) CEP2 is an 'opt-in' project due to its demanding nature and will hence exclude a certain segment of users.
3) For the same reason, every client machine is by default limited to one CEP2 instance at a time (which can be increased manually), since too many concurrent jobs can impact the overall performance (i.e., running out of RAM or simultaneous I/O can clog up the system and degrade its efficiency; the sweet-spot for many setups is 2-5 CEP2 jobs, but never more than the number of cores).
We clearly aim for a bigger load share but at this time we are where we should be.


We are actually preparing a big, multistage drive to rally new participants (in particular in our research community which has a lot of computers at its disposal), but this is scheduled for the time after the throttled warm-up. We already heard of people who cannot get as many CEP2 workunits as they'd like, so there is no point in increasing demand just now.


We prefer to think of it as 'challenging' .


CEP2 is undoubtedly demanding on the host hardware (that's why it's 'opt-in'), and we take your criticism seriously - but gb009761's comment hits the mark. The high demand reflects the nature of the research we are conducting! First-principles quantum chemistry is hard because it is a sophisticated method capable of capturing intricate physical phenomena.
You may argue that such a project should then not be on WCG. We disagree on this since already at this early stage CEP2 has been a success for us, producing more data than we could have obtained using our cluster. If we want to do this nontrivial, high quality numerical study on a large scale, then the WCG is the only way. That's what makes the WCG exciting for researchers: it allows us to do science which could otherwise not be done. We clearly think CEP2 is worthwhile, otherwise we wouldn't invest our time and money on it.

A few more concrete points:
1) Our servers run stable, have plenty of capacity, and an analysis indicates that it is not a bottleneck for uploads.
2) While the uploads are unfortunately bigger than in other WCG projects, we currently average at 22.7MB per result, which may not exactly qualify as 'huge'.
3) The RAM use per job is limited to 512MB.
4) Instead of running many CEP2 jobs simultaneously, we recommend to mix CEP2 with jobs of a less hard-core project (this is incidentally a common strategy in high-performance computing). Never run more jobs than you have cores - this completely kills your computer's performance!
5) Our error rate is well below the limit set by IBM, but we understand the frustration about any problems.
6) While Q-Chem is a commercial code, we have the source, can modify it for CEP2, and are in close contact with the developers. Our IBM friends and we have been tweaking things for the WCG and continue to do so, but the inherent complexity of the methods will not go away. Program packages like Q-Chem are constantly improved and expanded by the professional developers and academic contributors, so it is already an optimized high-performance code - we merely tailor certain aspects of it for the specific situation of the grid.


CEP2 is decidedly not a short project and will be around for a while! We do have estimates but as they depend on many variables it's not really worth setting in stone.


Again, thanks for all your feedback and input - the interaction with the WCG community is really great! We hope you find our explanations useful and decide to stick with CEP2. We are also glad that people like our research log .

Cheers,


The CEP Team

The credit issue has surely an impact on the attractiveness. But in my case it is again the fact that on many of my devices CEP2 just fails and all WU's get errored. I loose too much time to test and select. And also it may happen that after some successfull WU's they all get errored after. To much hassle in the end. So I do crunch on a few of them but I surely could have done more. I am sure that many crunchers that get errors and have one device or two would just avoid it.
HFCC, FAAH, HCC and HCMD2 run on all my devices without problems.
But for whatever reason CEP2, C4CW, HPF2 fail on many of my devices.
So to avoid the Sapphire badge runtime limit syndrom I decided to crunch the problematic projects to 5 years minimum and all the other up to 25 if they run long enough.
DDDT2 is a special case. I get one WU per month so it is not an issue.

CEP2 normally would be my favorite project because of what it represents. however, I have pretty much decided to limit it as much as possible. I don't totally mind that it is not 100% efficient and that there is some CPU wasted. It is the nature of the project. However, I can't get over the fact that I continue to get upload problems (multiple systems), so that is just a big waste of CPU time/electricity.
The limitation of 1 or a million WU isn't very good either, but that can be worked on as time permits (it is the holidays, techs go on vacations)

edit: true the error rate is below the requirement. however, does WCG track the time wasted due to the errors? I mean for most other projects, WUs error out right away. However, for CEP2 they can error out in the middle or after a successful WU is done, wasting hrs of crunch time.