Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 3315
Posts: 3315   Pages: 332   [ Previous Page | 117 118 119 120 121 122 123 124 125 126 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3306564 times and has 3314 replies Next Thread
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1323
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Generation 077 is already treated as straggler. Local deadline 3.5 days (+ server grace period 1 day)
ARP1_ 0023073_ 077_ 1-- 	In Progress	29-7-21 05:14:07	2-8-21 17:14:07	0,00	0,0 / 0,0
ARP1_ 0023073_ 077_ 0-- In Progress 29-7-21 05:12:28 2-8-21 17:12:28 0,00 0,0 / 0,0

[Jul 29, 2021 7:10:01 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12435
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

According to knread, they are treating 5 generations as 'normal'. As we are now into 082, that would be generations 078 - 082. 077 is therefore now 'priority'.

I haven't seen an 081 or 082 yet.

Mike
[Jul 29, 2021 11:22:33 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Started getting these messages now...
Anyone else getting these?

Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | No tasks are available for Africa Rainfall Project

I see that every day too. I think that is WCG's strange way of telling you that you don't need any.
[Jul 29, 2021 4:29:38 PM]   Link   Report threatening or abusive post: please login first  Go to top 
kwolff88
Cruncher
Joined: Dec 31, 2004
Post Count: 19
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

I'm crunching an 049 right now. Specifically, ARP1_0010193_049.
[Jul 29, 2021 4:37:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Acibant
Advanced Cruncher
USA
Joined: Apr 15, 2020
Post Count: 126
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | No tasks are available for Africa Rainfall Project
I see that every day too. I think that is WCG's strange way of telling you that you don't need any.
You will see that if your computer is already assigned the maximum number of ARP tasks you've set in your preferences, for sure. They do need a different response for WCG's purposes with its different projects. If you cap the number of work tasks in the online preferences to get fine-grained control over the makeup of your processing tasks and backlog (especially in combination with maximum concurrent tasks in app_config.xml), you can end up getting tasks from projects you don't want unless you uncheck the option to allow work from projects you haven't opted into if no work is available.

In my case this meant I had been getting OPN tasks if my client cache ever was perceived to be too low compared to the settings in the online preferences where I try to split half-MCM and half-ARP (accounting for different completion times). I could lower the overall work cache setting further but then I wouldn't get much of a buffer. (15 max concurrent for ARP/MCM to use 30 threads total, 20 max downloaded tasks for ARP, 32 max downloaded tasks for MCM, and 0.15 day cache). I do this because otherwise the client can end up getting lopsided with mostly MCM if I don't limit maximum concurrent/downloaded for that or it can get buggy and download an inordinate amount of work units (see entity's 990 tasks after switching).
----------------------------------------

[Jul 29, 2021 6:32:50 PM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 981
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Just noticed that I was sent ARP1_0001951_083_1 at 18:30 UTC on 29th July...

Cheers - Al.
[Jul 30, 2021 1:42:24 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12435
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Thank you, Al.

An even earlier one and the results being returned are nearly 18,000 per day. At that rate it would take a complete generation 4 days.

083 indicates we are at about 45.4%, but for this month I am assuming 2 generations behind to allow for the stragglers, so 44.3%.

The latest interval is just 21 hours and the 10-interval average is down to 3.81579 days. The end date forecast is now August 2022, however, if this rate were to continue, the forecast would be March 2022, but I doubt it.

I would expect the next generation to start about 2 August but it could even be today.

Mike
[Jul 30, 2021 2:40:50 AM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

A few updates. First the latest distribution of the units:


count generation
-------- ----------
6 001
...
3 035
2 036
0 037
0 038
1 039
4 040
7 041
5 042
4 043
7 044
7 045
5 046
6 047
6 048
4 049
8 050
7 051
6 052
7 053
12 054
11 055
6 056
8 057
13 058
11 059
10 060
9 061
15 062
7 063
9 064
14 065
18 066
19 067
13 068
25 069
15 070
22 071
17 072
36 073
63 074
186 075
798 076
3067 077
7002 078
9564 079
7895 080
4665 081
1745 082
236 083
3 084


As noted on Monday we had about 2500 results (~1250 units) that were waiting to be assigned. We were able to recruit a little more firepower and by mid-day yesterday the backlog dropped to running between 0 to 200. As a result, for the first time in a long time we were able to have all the work assigned and in progress which is an important part of making the project run as quickly as possible.

I've been reworking a few things on the backend and from the time that a unit is validated, it will take about 15 minutes before then next generation is out running so the delays are also minimized on the server side (previous when we had a backlog, this turnaround time didn't matter as much so it was as long as 6 hours, but now getting the next generation out as quick as possible is worthwhile).

We are averaging about 8000 units/day right now (we the last two days being above that average). If this pace continues, then we will be completing a generation about every 4.5 days.

Thanks to everyone contributing and we really appreciate the help in getting this running quickly.
[Jul 30, 2021 6:01:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Dayle Diamond
Senior Cruncher
Joined: Jan 31, 2013
Post Count: 452
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Thanks Knreed.

With the growing spread of these workunits, I'm reminded of the last batches of FAH2. Each work unit was supposed to run 150 times, but some work ran another 30+ generations while the stragglers caught up.

Imagine running 83 more cycles just for those six lagging work units!

I'm hoping that was a one-time issue, and after generating a year's worth of weather data, the generations cease or are manually aborted.
[Jul 30, 2021 8:38:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
pwhidden
Cruncher
USA
Joined: Nov 17, 2004
Post Count: 32
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Kevin,

Thanks very much for the update. I hope you will be able to re-post the "generation" table every once in a while. It gives us incentive.

I too have been watching Mike's posts on the "end date". Of course that isn't really an end date but a date when the last generation gets sent out and we can assume declining project resource need shortly after that.

I am curious if you why those 6 are still on generation 1. The sequential aspect to this project on any one piece of land mean there are limits to how fast we can get the last work unit for the project completed.

These laggards are only given to "reliable hosts" which I believe is defined as returning results within 2.0 days. We also know that the average high priority WU turnaround time is 1.7 days. Since two valid results need to be submitted before advancing to the next generation, I am assuming the "laggard interval time" will be about 2 days + your 15 minutes to set up the next generation. Assuming there are no "invalid" or no responses from the clients assigned these laggards (I doubt it, but let's assume so) then the earliest the last work unit for the project will be completed is late July 2022.

Other than those 6 stuck on one, we moved the end group from generation 32 to generation 35 in 6 days.... or 2 days per generation. Small sample size, I know, but it would match my assumed "laggard interval time" above.

Paul
----------------------------------------

[Jul 30, 2021 9:18:47 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 3315   Pages: 332   [ Previous Page | 117 118 119 120 121 122 123 124 125 126 | Next Page ]
[ Jump to Last Post ]
Post new Thread