Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
![]() |
World Community Grid Forums
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Member(s) browsing this thread: Unixchick |
Thread Status: Active Total posts in this thread: 3321
|
![]() |
Author |
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
You will find that a 12/0/0 strategy will crunch ARP units faster. That would speed up your ARP production but reduce your total production.
The other projects are worthwhile so you would be depriving them of your assistance. A useful survey would be to start with 24/0/0 and gradually reduce to 12/0/0 recording average unit times at each stage. Then increase gradually to 12/12/0 and then transfer one at a time until 12/0/12. Mike |
||
|
leloft
Cruncher Joined: Jun 8, 2017 Post Count: 23 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
The other projects are worthwhile so you would be depriving them of your assistance. Not completely: I am deploying a 0/2/2 strategy on two other machines, a 0/0/2 on a third and a 4/2/2 on a fourth.A useful survey would be to start with 24/0/0 and gradually reduce to 12/0/0 recording average unit times at each stage. Then increase gradually to 12/12/0 and then transfer one at a time until 12/0/12. Preliminary indications are that the 24/0/0 strategy results in ARP workunits taking >65h each; an 18/0/0 strategy reduces that to about 40-50h and a 12//0/0 to about 30-40. A 12/6/6 pushes that back up to >>50h. My plan was to run a strategy for 1 week and calculate a mean number of ARP points per day from the daily device totals for days 3 to 7, change strategy at the end of day 7 and repeat.However, your strategy is an attractive one: using a stepwise change in strategy of 3 units each time, it would take 10 weeks to do the suggested cycle. It would certainly make for a good pilot. |
||
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Does that mean you have one machine with 8 threads, 2 with 4 threads and 1 with 2 threads?
As you are crunching all projects, I would have thought that you would want to maximise your total output and not just ARP. 24/0/0 seems to outputting 8.86 units per day, whereas 18/0/0 is 9.6 per day and 12/0/0 is 8.23 per day, based on your figures. That would seem to indicate that 18/0/0 is the best of those 3 options and that the I/O problems with ARP do not become a problem until above 18 ARP units. However, If you were to compare points and include the points from OPN/MCM that should produce the best overall. Mike |
||
|
leloft
Cruncher Joined: Jun 8, 2017 Post Count: 23 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Does that mean you have one machine with 8 threads, 2 with 4 threads and 1 with 2 threads? Yes: 1 x 2P, 2 x 4P, 1x 4P4V, 1 x 12P12VAs you are crunching all projects, I would have thought that you would want to maximise your total output and not just ARP. I am trying to maximise my useful output. I assume that (i) the fastest machines (quickest turnaround) are assigned the stragglers, i.e. to shorten the backlog faster in the tail than it is growing at the head; and (ii) 12 units every ~30h is of more use than 24 every ~60h. 24/0/0 seems to outputting 8.86 units per day, whereas 18/0/0 is 9.6 per day and 12/0/0 is 8.23 per day, based on your figures. That would seem to indicate that 18/0/0 is the best of those 3 options and that the I/O problems with ARP do not become a problem until above 18 ARP units. It's far to early to tell: those figures are not 'pure' i.e. the longer a unit takes, the more likely it is that the processing time is a result of a hybrid (exploratory) strategy. Hence the calculation of the mean from days 3-7 of a single strategy, after the previous strategy has been flushed out.One thing that I am unclear about: do all the ARP units contain roughly the same amount of work (as measured by 'points')? I ask because a 4/0/0 strategy on the 4P4V machine can turnaround ~400 point units in ~15h (~108 points per hour); a 6/0/0 turns ~500 point units in ~20h (~150pph) and an 8/0/0 ~550 point units in ~28h(~160 pph). This roughly translates to a 4/0/0 strategy processing ~6.4 units per day; a 6/0/0 processing ~7.2 and a 8/0/0 processing ~6.9. So are the 6.4 '15h' units more valuable than the 7.2 '20h' or the 6.9 '28h' ones? My idea is that they would be if they were 'backlog' units; if backlog units could be targeted to machines running a confirmed optimal turnaround strategy (perhaps the above 6/0/0 or 18/0/0) then that would satisfy my aim to maximise useful output by chipping away at the backlog, albeit at the 'cost' of overall points. In effect, satisfying work as opposed to best-paid work. |
||
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I am all in favour of maximising output. but each machine should be evaluated on its own.
The best way is to determine the total points generated per day across all 3 projects (I am ignoring HST for lack of availability). I think the best combination for your big machine will be about half ARP and the rest OPN/MCM, but I might be wrong. Every machine is different> The main problem with ARP is the amount of input/output required, especially at checkpoints and uploading. More RAM does help. Check out the specifications required. Your smaller machines may not be ok for ARP. Mike |
||
|
Vester
Senior Cruncher USA Joined: Nov 18, 2004 Post Count: 325 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I have an Intel Core i9-10850K @4.9GHz (ten cores/20 threads) that is currently running 19 ARP tasks. The computer is set to run all projects.
----------------------------------------![]() [Edit 1 times, last edit by Vester at Aug 17, 2021 12:36:50 PM] |
||
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Vester
It is advisable to cut your ARP back to 10 units at a time using app_config.xml Ask here if you would like a copy and instructions. The ARP units require a grest deal of capacity, especially at each checkpoint (each 12.5%) and uploading. Run OPN & MCM on the remainder of your threads. You should set Project Limits in your Profile to 1 more each over what is in app_config.xml That way you should get maximum output with minimum queue length and your machine will be considered 'reliable'. Mike |
||
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Latest stats:
Average Generation: 83.6 Pace (average time to complete a generation): 4.0 (7-day average) first_indexed generation num_units_currently_on_generation num_units_completed_last_day |
||
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Thank you, Kevin.
As 080 is the latest generation I have been able to identify as labelled 'priority'. I will base this response on that. If anyone knows of a later batch please advise and I will adjust my figures. 081 should be a priority by now, but the only ones that I have seen have been resends. There have been 30,382 units validated in generations up to and including 080 in the last 5 days, out of 95,685 returned. Allowing for 2 results being needed for each unit, that corresponds to 63.5% of work done. There are now 13,840 units remaining to be crunched in those generations out of a total of 226,584 up to generation 089 (just 6.1%). The stragglers are catching up, but the total is moving up. However, those generation 001 's are still stuck. I reckon that they would take at least 3 months to catch up. Maybe even a year. Mike |
||
|
leloft
Cruncher Joined: Jun 8, 2017 Post Count: 23 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I am trying to understand the structure of these various generations and workunit priority; I am reading this thread from the beginning and trying to answer the following questions:
(i) What is a 'generation' of workunits? (ii) I read that a unit is a 'plot of land'; What sort of sizes are they, typically (and atypically)? (iii) How does a generation (i.e. 001) get stuck? I'd greatly appreciate any pointers to articles that could answer these (and other) questions. Many thanks |
||
|
|
![]() |