| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: Scheduling Optimizations |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 10
|
|
| Author |
|
|
enels
Senior Cruncher Joined: Apr 25, 2008 Post Count: 286 Status: Offline Project Badges: 
|
I was wondering what kind of scheduling optimization takes place.
There are the constraints of the projects and the constraints of the users, but within those constraints there are optimizations that seem possible. I'm thinking of user M who wants to run projects X and Y on her hyperthreading computer C but doesn't care how much of either. Maybe it is always better to run X and Y together. Maybe it is better for that processor to only run X. Now M buys another computer D and wants to run another project Z. Now maybe it makes sense to run Y and Z on C and X on D. Add to the mix user N who also wants to run X and Z on his computers E, F and G and is willing to optimize with M. M and N could be considered to be implicitly willing to optimize their computers by sharing projects. The constraints of users and their hosts are known. The RAC and the last result returned. The processor and memory configuration is typically known. The user settings and profile. I'm not sure if the CPU statistics provide enough information, however, for the optimizations I alluded to above. What might help is a benchmark that reported back on how different projects did in combination with each other on a particular host. The constraints of the projects aren't known to me and don't concern me. But there doesn't seem to be any optimization at the host level that I have seen. People will tell you, of course, an exponential increase in efficiency is dangerous. But with great risk comes great omg what just happened. Most of them are lunatics anyway. Not this one fellow though. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
This is an interesting question. You mention an exponential increase - that can't happen. It's not dangerous, it just isn't there to be got. The best you can hope for is to eliminate some of the conflicts that cause small performance losses when identical processes are competing for the same resource.
|
||
|
|
JmBoullier
Former Community Advisor Normandy - France Joined: Jan 26, 2007 Post Count: 3715 Status: Offline Project Badges: 
|
The constraints of users and their hosts are known. Strictly speaking, - it is known that some users have some constraints on some computers - even for them it is not easy to express them strictly and precisely - these constraints can change from one day to the next based on software (OS, Boinc, science apps) and hardware updates - it is not clear if similar environments show the same constraints for all users - for most users either there are no constraints, or they are completely unknown and last (for now) but not least - given the extreme diversity of environments (HW + SW) it seems that attempting to integrate all these parameters in the distribution schedulers may not be a reasonably achievable objective. All the above is not to reject the idea (I have no power for that anyway) but to show that we are still far from having a clear view of the (possible) problem. Cheers. Jean. ---------------------------------------- [Edit 1 times, last edit by JmBoullier at Apr 9, 2009 5:42:21 AM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
If it was inferred, a scenario: I select HFCC+CEP as the preferred projects, with "If there is no work..." ticked. WCG determines that my box is not so optimal at CEP so only sends HFCC and if that's short in supply, alternate. Would I get Peed Off with these patronizing BOFHs using my computer cycles differently than the spirit of why I volunteered CPU time?
----------------------------------------The future of BOINC will bring: - Dynamic work sizing based on performance of a device, since WCG knows hourly claims and flop throughput, so e.g. slower devices will receive smaller tasks. That will go with an overriding option for those who still like smaller or bigger (it's only on a drawing board somewhere in a corner for future development). - There is planned for development the automatic limiting of work units for specific projects if a device frequently fails at them. The current limiting is general i.e. work supply can sink to 1 a day to test if the fail condition was removed. Volunteers are bound to react to that if they see their logs with red lined messages to tell them something specific is wrong. - There has been a request, multiple times to allow the limiting of specific sciences who tend to bottleneck resources, or run the CPU extra hot (latter not here). Nothing new, and sure if desired can be re-tabled at the Berkeley developers forum (which they don't read). I think a number of members probably would appreciate that one. 2 Euro cents, printed in a different country of the Community.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Apr 9, 2009 7:22:55 AM] |
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1672 Status: Offline Project Badges: 
|
Hi,
----------------------------------------I am fascinated by all the suggestions provided by (experienced) users. I am wondering if it would be possible to manage a little bit all these suggestions and requirements in order to enable a consistent feedback to the Berkeley development team (maybe they can read  ).It is a spontaneous idea, I did not think at this time about technical solution, but perhaps a wiki could be an appropriate platform for gathering and structuring requirements ? ... Cheers, Yves |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Hi Yves,
----------------------------------------BOINC has the Trac system. Can be used by anyone with a good idea or feature suggestion. e.g. the techs use this system to inject what WCG likes changed, fixed or added. It works and does not, for there are items that keep being pushed out in the infinite. e.g. the BOINC Manager position of the window when restarted got always lost. Finally Finally it is fixed in the 6.6 client, but it took a few years. cheers
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Well, if you try to code this directly into bionics or the WCG servers, it would be pretty difficult (I am assuming as I don't really know how WCG's algorithms are set up...) but IMHO, I believe it would be very concievable to, since points are supposed to reflect helpful work, see which projects score the highest points per hour for each computer and have that computer favor taking those work units. This would maximize efficentcy and I believe that is enels' goal.
----------------------------------------I recently wrote something to this affect here, but never got a response. Happy Crunching!  [Edit 1 times, last edit by Former Member at Apr 10, 2009 1:59:16 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Otis11, I believe your goal would conflict with other things the scheduler does. In short, I think it would lead to an unsolvable equation.
I haven't worked out all the details. As for why the suggested optimisation has never been done, there are two reasons I know of: 1) lack of developer time, and 2) the time cost in serving scheduler requests. The former is unlikely to be available soon, and the latter is largely unknown. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Well, if you try to code this directly into bionics or the WCG servers, it would be pretty difficult (I am assuming as I don't really know how WCG's algorithms are set up...) but IMHO, I believe it would be very concievable to, since points are supposed to reflect helpful work, see which projects score the highest points per hour for each computer and have that computer favor taking those work units. This would maximize efficentcy and I believe that is enels' goal. I recently wrote something to this affect here, but never got a response. Happy Crunching! Would you please amend link in your previous post Otis11 as it is recursive. As for points, still let it be the members decision to go for the not optimal project if it appeals to them... see another option coming "send work with greatest performance yield". ;>)
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Apr 9, 2009 12:58:35 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Would you please amend link in your previous post Otis11 as it is recursive. fixed! see another option coming "send work with greatest performance yield". This is going to be an option!? [Edit 1 times, last edit by Former Member at Apr 10, 2009 2:03:45 PM] |
||
|
|
|