| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: To Project Scientists: Clean Energy Project Phase 2 - Limited Cruncher Participation |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 84
|
|
| Author |
|
|
Dataman
Ace Cruncher Joined: Nov 16, 2004 Post Count: 4865 Status: Offline Project Badges: 
|
Let the stats speak for themselves, 20% 40% 80%, it's obvious that those who wanted a fair share in the mix profile multi-core devise did no buy into the golden goose being funnel fed method and AFAICD, this worked like a boomerang... it did for me... contribution dropped to 74% of 1 core. Pretty plz WCG techs, why not a little tête-à -tête with cleanenergy to review and set the weight back down. Those who have no reservations and selected CEP2 as their preferred project will have done so weeks ago and those who want a regular stream and not running for their May-West, will have closed the dam gauge to a 1 core trickle... and I'm sure that was not your objective... but maybe it was ;>) Thanks to everyone for all the interesting posts.  Sek makes a good point. However, the issues with CEP2 has caused me to reflect on other DC projects. More and more I find projects that have limited the number of participants by limiting enrollment or have limited the number of wu's and how they are distributed. In doing so they can control their capacity and environment issues. For the last half dozen years I seem to have had an assumption that everyone wants the work done as quickly as possible so that research can contine. On recent reflection I find this may not be true for various reasons I will not discuss here. So if a project is happy with how things are going then why shouldn't I? My choice is whether or not I want to crunch it or crunch it it's current operational state. There is so much good science going on right now in so very many good DC projects. There is no shortage of crunching. Just a shortage of crunchers. I hope the CEP2 discussion continues here.  "When you have the answer Change your perspective For everything you know Is wrong."    |
||
|
|
deltavee
Ace Cruncher Texas Hill Country Joined: Nov 17, 2004 Post Count: 4884 Status: Offline Project Badges: 
|
The Clean Energy Project Scientist has mentioned several times the default setting and 'opt-in' nature of this project. I suspect they would be content if everyone ran like that.
----------------------------------------
4858
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
to cleanenergy (scientists):
can you provide a list of files which are unsafe to hardlink to identical neighbors, e.g. checkpoints an output files ? i can submit simple shell scripts for review to act upon the redundance of large CEP2 thread counts with safety features to not interfere with results file inodes. this is feasable for posix clients with posix specific function of filesystem hard-links. e.g. everything that runs boinc except NTFS and windows. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I also have (mostly happily) settled into running with NO set on the overrides, as this seems to run smoothly, where I did seem to have more problems when running all the cores on this project. Another tweak to the settings on this project that I would like to see, however, is a slightly shorter report deadline, so that I could set that one Clean Energy task to always run with priority. Might this be possible?
|
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges: 
|
At this time the only additional change we are looking at is changing the 'open the floodgates' or 'one at a time' setting to become a setting that lets you pick how many a computer can have at once. The setting choices would be:
Default 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Default would check the bandwidth settings. For 2-16, the bandwidth limit would be ignored. This change would let most people be able to customize their contribution to the project to their liking. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I like that idea give me 2 at a time...
|
||
|
|
deltavee
Ace Cruncher Texas Hill Country Joined: Nov 17, 2004 Post Count: 4884 Status: Offline Project Badges:
|
Thanks for letting us know. The sooner, the better.
----------------------------------------
4858
|
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
We have given this project an equal weighting with the other projects as of last Friday. However, it is will be difficult for it to achieve parity in terms of runtime with other projects due to the opt-in, result limiting and bandwidth checking that it has.
Some people have asked about why people who opt-in to cep2 get more work for it than other projects. The reason is as follows: In the scheduler, there is a in-memory cache of results available to send to clients that has each slot assigned to different apps so that they are interleaved for all the applications. The number of slots for each app is proportional to the weight of the project in the database. Currently when a client is requesting work, it starts at a random spot in the in-memory cache and starts looking for results that can be sent to the client. If a result cannot be sent to the client for any reason (due to memory requirements, bandwidth requirements, user not approving the application, etc) then the result is marked as 'infeasible for some hosts'. This designation is important, because the scan of the in-memory cache actually occurs multiple times and does so in the following way: 1st pass checks for results that need to be sent to a reliable host 2nd pass checks for beta work that needs to be sent 3rd pass checks for results that have been marked infeasible 4th pass checks for any result in the 4 cases above, only results matching the approved projects for the host will be selected. 5th pass, if the user has opted to receive results from other project if none are available for their preferred projects, check for any result from any project Since CEP2 will have a lot of results marked infeasible due to being an opt-in project and due to the bandwidth check, they wind being selected at a greater weight in pass 3 than most of the other projects. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
to cleanenergy (scientists): can you provide a list of files which are unsafe to hardlink to identical neighbors, e.g. checkpoints an output files ? i can submit simple shell scripts for review to act upon the redundance of large CEP2 thread counts with safety features to not interfere with results file inodes. this is feasable for posix clients with posix specific function of filesystem hard-links. e.g. everything that runs boinc except NTFS and windows. Can you explain a little further what you are proposing and what problem you are looking to solve with this? The checkpoints, are unique, the only repeated data is the unzip of the input files that occurs at the start of each workunit. However, the BOINC architecture intends to provide isolation between each slot directory. |
||
|
|
KWSN - A Shrubbery
Master Cruncher Joined: Jan 8, 2006 Post Count: 1585 Status: Offline |
Knreed,
----------------------------------------I'm glad to hear you're looking into set numbers of workunits. I always believed that an all or nothing approach was far too extreme. On my I7 with HT, I would routinely end up crunching 7 or 8 tasks at once with a mix of three projects. Once these new options are introduced, I may opt back into the project. The only suggestion I would have made is a n(cores)/2 option as Sekerob mentioned. I know I have a wide range of cores per computer and only so many device profiles to go around.  Distributed computing volunteer since September 27, 2000 |
||
|
|
|