Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 12
Posts: 12   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3634 times and has 11 replies Next Thread
sepmsk
Cruncher
Russia
Joined: Dec 31, 2007
Post Count: 24
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Are we going to expand the deadlines for Type A units?

Hello everyone,

I've got a type A workunit from the DDD Phase2 project. I believe the WU's deadline is too tight and I can miss it given my usual PC usage (in spite of this WU is calculated on a Core i7 machine!)

Probably I'm not the only person who can miss the deadline just because it is too tight. If so, we risk to waste some workunits, don't we? Do you think we should expand the deadlines for type A workunits?

Thank you.

P.S. By the way, I have never experienced any deadline issues with other projects on the WCG.
----------------------------------------
[Edit 1 times, last edit by sepmsk at Mar 27, 2010 10:12:36 PM]
[Mar 27, 2010 10:02:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3715
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

In short, the answer to the question in the title is No. smile

The return delay of type A WUs has been intentionally shortened and their distribution has been limited to usually fast returners** because these WUs condition the distribution of type B and type C behind.

From the type A WUs which have been already processed such a WU should need about 40 hours of process on your machine. If 40 hours in 8 days is OK for you and you are worried only because BOINC estimated a much longer duration, then ignore the estimate and let it go. If your usage constraints make it really impossible for your machine to do it, simply abort the WU as early as you can and don't worry. Most machines which will receive type A WUs are able to do the job in two days and there are more candidates than WUs.

Cheers. Jean.

** Fast returners are machines which have an average return delay below 2 days.
We are aware that most machines can be a fast returner for other projects (they only need to have a small enough work cache) but the slowest ones or those which are not running 24/7 may not be able to return the huge type A that fast.
This fact has been taken in account by the techs when the current distribution scheme has been decided.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Mar 27, 2010 11:09:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sepmsk
Cruncher
Russia
Joined: Dec 31, 2007
Post Count: 24
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

Thank you Jean.

As to the runtime required - I see that only ~1,64% of the work is being completed per calendar hour (not the run hours shown in Boinc) on this PC. The PC is a Core i7-860@2,80GHz with Linux Fedora 12, HyperThreading is on, CPU Frequency is On-demand, oscillating between 1,2 and 2,8 GHz. My settings limit CPU usage by 70% for Boinc. Given these numbers, approx. 60 calendar hours needed to complete a Type A unit.

BTW, if a deadline for DDD Phase 2 is missed and the result is returned after the deadline - how long the result will still be useful for the project after the deadline?
[Mar 28, 2010 6:31:46 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

A client that is known to have a harder time returning results is actually granted some extra time.

Useful? Well when a result is not meeting deadline and comes back before the backup, it's used, but if the backup is started before the message arrives that is has become redundant it's kind of a waste. A coming client is a little smarter. It will self abort too late results not yet started. Maybe it will even abort un-started tasks if it sees it's not going to make deadline. With exception of CPDN who have no hard deadline, it's something I would code into the client scheduler.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Mar 28, 2010 6:39:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

A coming client is a little smarter. It will self abort too late results not yet started. Maybe it will even abort un-started tasks if it sees it's not going to make deadline. With exception of CPDN who have no hard deadline, it's something I would code into the client scheduler.

Last time this was discussed, it was decided to not just auto-abort unstarted tasks, even the estimates indicates can't meet the deadline.

Instead, the BOINC API includes a function that tells how long until the deadline, and it's up to the individual project if they wants to use this by adding something to their applications. This can example be a check on application-start, example for Rice:
if time_to_deadline < 7 hours, don't run the task but immediately abort it.

Another use would be something like:
if time_to_deadline < 1 hour, stop task and upload current progress.


BTW, while most users wouldn't hit this problem, one of the rules of the Scheduling-server is to always assign a single task to a computer that doesn't have any work in project, except if it's hard_app, even if the computer based on the various estimates has no hope of returning it by deadline. So, if client just auto-aborts any tasks that can't meet the deadline, these computers will basically download X tasks until hitting their daily quota, and continues this for each continuous day, and never doing any work.

If a project makes an "oopsie" and incorrectly sets the wu-estimates much too high, with client auto-aborting, fairly quickly all wu with the wrong estimates will error-out, and it's a good chance most of the computers on project is sitting with daily-quota of 1, and with upto a 24-hour deferral until can ask for more work... (more than one non-WCG-project has made this oopsie).

By not auto-aborting, client will over time learn the estimate is bogus, and the DCF drops to 0.01 or something...
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
----------------------------------------
[Edit 1 times, last edit by Ingleside at Mar 28, 2010 9:24:44 PM]
[Mar 28, 2010 9:16:07 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sepmsk
Cruncher
Russia
Joined: Dec 31, 2007
Post Count: 24
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

Thanks to all for the answers.

A question to Sekerob :-) You wrote that a client that is known to have a harder time returning results is actually granted some extra time.

How does the system know that a client has a harder time? And how much extra time does it grant? As to the thread topic - I aborted the task execution a day before. Obviously, the root of the issue was the PC usage pattern, not the performance itself (it would be more than enough performance to complete the task on time if the PC worked, say, at least 12h per day). This is a home PC; it works on evenings usually. And it would be just not enough evenings to complete the task on time.

So the situation is - a task is calculated, a consistent progress is achieved and the result would be returned, but just not quickly enough to meet the deadline. Does the system addresses such situations somehow?
----------------------------------------
[Edit 2 times, last edit by sepmsk at Mar 29, 2010 8:55:41 PM]
[Mar 29, 2010 8:52:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3715
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

The PC is a Core i7-860@2,80GHz with Linux Fedora 12, HyperThreading is on, CPU Frequency is On-demand, oscillating between 1,2 and 2,8 GHz.
sepmsk,
If this is what you want, fine.
But in case you don't know, with this power management setting your processor is slowing down when it has "nothing to do for you". The problem is that BOINC with its lowest priority is considered by the system as a background task, not "something for you", and if you leave your machine switched on running BOINC only, for example while you have dinner, then the speed will drop to 1.2 GHz. Combined with your 70 % setting and hyperthreading that makes every thread behave as a very slow machine, hence your estimated runtime for an A-type.

If this is not what you want and if you would like to be able to process an A-type within its deadline you might start with setting your power management to Performance (or something similar) and raise BOINC to 100 % (with such a processor and the way BOINC works you should never be disturbed by BOINC).

And if this is not enough you might limit BOINC to 4 threads only ("use at most 50 % of the processors") while the A-type is running. You would still benefit from hyperthreading for your own activity but BOINC would process each of its 4 tasks more quickly.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Mar 30, 2010 4:53:53 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

Thanks to all for the answers.

A question to Sekerob :-) You wrote that a client that is known to have a harder time returning results is actually granted some extra time.

How does the system know that a client has a harder time? And how much extra time does it grant?

Haven't got the exact formula and unfortunately did not look when I got that 9 day deadline A-Type if that extended time was also granted to the wingman. Essentially WCG tracks return times, credit second, fpops capacity in amongst. Even slow devices can reach reliability state long as they keep returning shorter tasks within 48 hours on a regular basis. If then a task is assigned with very high fpops, a sum is made to see if more time is needed for that device. As from memory that number could get incremented up to 2x normal/rush deadline. I can picture faces that have a dynamite I860 who get paired with a N270 atom if they have enough memory, that used to do the rice tasks, 7 hours fixed week after week, :biggrin:, but as it is WCG tries to pair faster with faster already and medium with medium, if a wingman reports in too in time. There are 3 speed groups off late, but if push comes to shuff, the shared memory will not hold such a task too long for a speed match as else the distributor goes choking.

Sort of the broadline understanding, the nitty gritty not important for us.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Mar 30, 2010 8:42:38 AM]
[Mar 30, 2010 8:41:49 AM]   Link   Report threatening or abusive post: please login first  Go to top 
X-Files 27
Senior Cruncher
Canada
Joined: May 21, 2007
Post Count: 391
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

Has the deadline been extended? I thought Type As has 8 days but its not.

ts01_ c441_ ps0000_ 1-- 612 Pending Validation 24/03/10 10:51:10 27/03/10 22:43:45 36.68 945.0 / 0.0 <- mine
ts01_ c441_ ps0000_ 0-- - In Progress 24/03/10 10:51:07 08/04/10 16:20:53 0.00 0.0 / 0.0

ts01_ c450_ ps0000_ 1-- 612 Pending Validation 24/03/10 10:51:37 31/03/10 20:13:56 33.41 982.3 / 0.0 <- mine
ts01_ c450_ ps0000_ 0-- - In Progress 24/03/10 10:51:36 05/04/10 06:18:24 0.00 0.0 / 0.0

I remember my deadlines were 1apr2010.

Or is the deadline is based on the capability of the machine?
----------------------------------------

[Apr 3, 2010 2:06:43 AM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3715
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are we going to expand the deadlines for Type A units?

ts01_ c441_ ps0000_ 0-- - In Progress 24/03/10 10:51:07 08/04/10 16:20:53 0.00 0.0 / 0.0
ts01_ c450_ ps0000_ 0-- - In Progress 24/03/10 10:51:36 05/04/10 06:18:24 0.00 0.0 / 0.0

The only explanation I see for these strange 15-day and 12-day return delays for type A DDDT2 WUs is the attempt by the techs for allowing more time to slower devices.
However, if this is the case, either the correction routine is flawed, or if not maybe the distribution routines should exclude these devices from type A distribution.
The purpose of distributing type A WUs to fast returners only is to speed up the typa A process, and if normal correction routines can lead to allowing 15 days instead of 8 this is completely defeating the pupose.

Let's wait for what the techs will say about these cases.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Apr 3, 2010 2:14:59 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 12   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread