Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 93
Posts: 93   Pages: 10   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 169699 times and has 92 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: c-types arrived

Hello WCG.

Attention: Ingleside
Reference: Your [Mar 23, 2010 6:31:36 PM] post

Greetings,

First off, I have opted to defer participating in crunching DDDT2 WUs until the kinks - which are expected of any newly-launched endeavor for that matter -- have been worked out. So, I will not talk about anything particular to crunching DDDT2 WUs. Instead, what I want to discuss is something that is common for all WUs, DDDT2 included. In particular -- the DCF and estimatedRuntime -- to the extent that these and related matters affects workloads given to WCG crunchers.

So, please define the problem that you are referring to in your:
"Hmm, it seems you're overlooking the real problem... "

The way I see it, you have shown us the symptoms of what you may find as the problem. Needless to say, there can be potentially many problems hiding behind a given symptom. I think I have some idea of what the problem may be, but I would like to hear it from you.

Good day.
;
[Mar 24, 2010 6:52:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: c-types arrived

So, please define the problem that you are referring to in your:
"Hmm, it seems you're overlooking the real problem... "

The way I see it, you have shown us the symptoms of what you may find as the problem. Needless to say, there can be potentially many problems hiding behind a given symptom. I think I have some idea of what the problem may be, but I would like to hear it from you.

The "best" is if the estimates is very close to the "real" times, so DCF is close to 1, but it's still an advantage the estimates is a little larger than reality, so DCF will be 0.9xx. This way new users not too familiar with deadlines and so on won't risk becoming overcommitted at the start, even if they do increase their cache-size.

As for tasks having estimates much larger than reality, like the C-tasks, if you're only running C-tasks with similarly "wrong" estimates it's not a problem for majority of users, as long as they've got a permanent connection. For anyone that aren't permanently connected it will be a problem initially, since based on the estimates they've got maybe 2 days cached, but reality it's only 6 hours or something, and this can lead to idle computer until can re-connect and get more work.

Still, all work will be returned by the deadline in this instance.

If you're only running one type of tasks, too small estimate is worse than too large, since you risk becoming seriously overcommitted, downloading too much work than has any hope of returning by deadline. Afterall, it's not everyone that runs with a 0.01 days cache-setting. wink Still, appart for the 1st. batch of work, due to how DCF is immediately increased if too low, you'll quickly be "stable" in this instance.


The biggest problem is if you're mixing both too large-estimates and too small estimates. In this instance you can get a low DCF after running a serie of too-high-estimates, for so afterwards getting a serie of work with too small estimate. For one thing you'll risk becoming hopelessly overcommitted. Also, when you've finally worked-through the small estimate-tasks, and the DCF is high, if you now gets some too high-estimates again, you risk getting too little work.

For anyone not permanently connected, they'll basically half of the time be overcommitted, and half of the time run out of work before can re-connect, if don't get more or less continuous stream of CXCXCXCX so there only will be "small" variations. Even C-type is in limited supply, getting series like CCCCCCCXXXXXXXCCCCCCCCXXXXXCXXXXXCCCCCC is possible.

For anyone not permanently connected, I would rate a 2-day cache-size as fairly "normal", atleast this would be my choise if forced to use dialup, since connecting more than once a day would be a bad idea. With a 2-day cache-size, even a 5x difference in DCF could be a big problem...


Now, if you've got a permanent connection, and if you only runs WCG, and if you also uses a small cache-size like 0.01 days or 0.1 days or something, the variations in DCF doesn't really matter, since you won't run-out of work, and you won't get overcommitted with too much work than can return before the deadline.

For the 67% of WCG-computers with > 2 days average turnaround-time on the other hand, it can be more problematic.

Also, for the 10 - 20% of WCG-computers running multiple BOINC-projects, even with a 0.01-days cache-setting you can run into problems with so large variations in DCF, example missed deadlines on WCG-work.
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
[Mar 24, 2010 9:31:17 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: c-types arrived

Hello WCG.

Attention: Ingleside

Greetings:

It took me a while to respond as I've been running the content in your posts through my head. My understanding is reflected in the questions below:

What would be your suggestion..
1] In dealing with the matter of mixing WUs having varying estimatedRuntimes so that cruncher's risk of over/under-commitment is minimized?
2] For changes, if any, on websiteSettings? Do you think project-specific settings would help control over/under-commitment risk?

Note:
Maybe this discussion should be spun off to a separate thread; you may want to respond to this on a separate thread.

Good day
;
[Mar 26, 2010 7:18:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 93   Pages: 10   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 ]
[ Jump to Last Post ]
Post new Thread