Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 3263
Posts: 3263   Pages: 327   [ Previous Page | 151 152 153 154 155 156 157 158 159 160 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3047244 times and has 3262 replies Next Thread
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

In my section of the report that says:

Number of units that are being sent as reliable (i.e. Accelerated) or are having an extra result sent out (Extreme acceleration):

number_units max_generation type        
------------ -------------- -----------
30 088 Extreme
9998 104 Accelerated
25581 112 Normal


This is drawn directly from the database (i.e. not me speculating).

However, a specific workunit only gets marked as being "Extreme", "Accelerated" or "Normal" while it is transitioning from one generation to the next. Thus if we generation 105 beomes the " Accelerated" generation, then workunits that have already been created as generation 105 will not suddenly be marked as "Accelerated". Only those workunits that transition into that generation (or less) get marked as "Accelerated". Thus you can receive jobs from a generation that is now the "Accelerated" generation that are not themselves "Accelerated" because they were created before that generation became "Accelerated".

An additional comment, is that I use my report for tracking progress on a daily basis - so I have written the SQL query in a way that it gives me the data as it exists at 12:00 UTC on the current date.
----------------------------------------
[Edit 1 times, last edit by knreed at Nov 11, 2021 5:23:36 PM]
[Nov 11, 2021 5:20:55 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

11/10/2021 40:201:19:13:39 71,213,037 18,347
11/09/2021 41:009:19:04:18 71,150,646 18,505

It is interesting that results returned has dropped lately. we are in the 18k range when the system can handle 21-22k results before we have to wait for tasks.


There is one user in particular that operates a very large number of computer that has an outside influence on the pace of the project. However, running WCG is only done when those systems are not being used for their primary purpose so you will see it go up and done based on how much contribution this user is providing.

The reason that they have this outsized influence is that run their computers at 100%, they are on all the time, they have 0 cache and report results immediately. Thus they can accelerate the overall pace of the project because they turnaround work so quickly.
[Nov 11, 2021 5:35:10 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12391
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Thank you, Kevin.

That explains the situation with regard to generation 087/088. I will ignore the latest generation in my stats. as many of them will have been issued before the promotion.

The same will go for the 103/104, but it is pushing the limits, but not impossible, to explain why 104's were still being issued without priority some 18 hours later. The cache at WCG must be higher than previously.

I will adjust my end date forecast by half a day to allow for the noon dump instead of midnight.

Kind regards

Mike
[Nov 11, 2021 7:49:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
MJH333
Senior Cruncher
England
Joined: Apr 3, 2021
Post Count: 267
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Mike,
The same will go for the 103/104, but it is pushing the limits, but not impossible, to explain why 104's were still being issued without priority some 18 hours later.

I've just received another 104 with an 8-day deadline (ARP1_0001085_104_0). This was at 19:45 UTC today.

However, I also received a resend 104 with an 8-day deadline (ARP1_0032453_104_2) at 19:24 UTC today.

I had thought that resends in the "Normal" generation all had shorter deadlines, but I must have misunderstood. (I see that I have some 106 and 107 resends with 8 day deadlines too.)

Cheers,
Mark
[Nov 11, 2021 8:33:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

The same will go for the 103/104, but it is pushing the limits, but not impossible, to explain why 104's were still being issued without priority some 18 hours later. The cache at WCG must be higher than previously.


Ok - there was an issue with my query and it did falsely report 104 has been accelerated. Here is the corrected data:

number_units max_generation type        
------------ -------------- -----------
31 088 Extreme
10628 103 Accelerated
24950 112 Normal

[Nov 11, 2021 9:03:55 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Work Available

I had thought that resends in the "Normal" generation all had shorter deadlines, but I must have misunderstood. (I see that I have some 106 and 107 resends with 8 day deadlines too.)

If I'm not mistaken, this behavior goes way back to the CEP / HCC days when members were first beginning to use AWS instances that could be "yanked" away if higher priority paid Amazon work needed the resources. When those instances were removed, a large volume of work became orphaned and then became high priority resend work that "gummed up" the queues while waiting for reliable machines to become available. Many members were using these instances and the backlog became severe. I believe the techs made those types of resends non-priority so they wouldn't have as great an impact on the processing queues. So, yes, there are certain resends that get the standard due dates.
[Nov 11, 2021 9:18:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12391
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Mark

From my observations, resends do not get priority status unless they are already past their original half-way point.

Mike
[Nov 11, 2021 9:39:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12391
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Thank you, Kevin.

Those figures look much better.

My next report will take account of the refinements that have been advised including the extra half day for the end date.

Mike
[Nov 11, 2021 9:53:42 PM]   Link   Report threatening or abusive post: please login first  Go to top 
MJH333
Senior Cruncher
England
Joined: Apr 3, 2021
Post Count: 267
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

entity and Mike,

Thank you for taking the time to explain this to me. As a newbie cruncher, I've got a lot to learn!

The examples I noticed of normal generation resends with the standard deadline seem to have occurred in cases where one of the original tasks errored out more or less immediately, so that would be consistent with Mike's observation.

Cheers,
Mark
[Nov 11, 2021 11:30:48 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Unixchick
Veteran Cruncher
Joined: Apr 16, 2020
Post Count: 970
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

ARP1_0027940_104_1 In Progress 2021-11-13 19:03:13 UTC 2021-11-18 07:03:13 UTC

Got a 104 with a shorter deadline
[Nov 13, 2021 7:06:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 3263   Pages: 327   [ Previous Page | 151 152 153 154 155 156 157 158 159 160 | Next Page ]
[ Jump to Last Post ]
Post new Thread