Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
Member(s) browsing this thread: Unixchick
Thread Status: Active
Total posts in this thread: 3321
Posts: 3321   Pages: 333   [ Previous Page | 126 127 128 129 130 131 132 133 134 135 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3319423 times and has 3320 replies Next Thread
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

You will find that a 12/0/0 strategy will crunch ARP units faster. That would speed up your ARP production but reduce your total production.

The other projects are worthwhile so you would be depriving them of your assistance.

A useful survey would be to start with 24/0/0 and gradually reduce to 12/0/0 recording average unit times at each stage. Then increase gradually to 12/12/0 and then transfer one at a time until 12/0/12.

Mike
[Aug 16, 2021 4:36:06 PM]   Link   Report threatening or abusive post: please login first  Go to top 
leloft
Cruncher
Joined: Jun 8, 2017
Post Count: 23
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

The other projects are worthwhile so you would be depriving them of your assistance.
Not completely: I am deploying a 0/2/2 strategy on two other machines, a 0/0/2 on a third and a 4/2/2 on a fourth.
A useful survey would be to start with 24/0/0 and gradually reduce to 12/0/0 recording average unit times at each stage. Then increase gradually to 12/12/0 and then transfer one at a time until 12/0/12.
Preliminary indications are that the 24/0/0 strategy results in ARP workunits taking >65h each; an 18/0/0 strategy reduces that to about 40-50h and a 12//0/0 to about 30-40. A 12/6/6 pushes that back up to >>50h. My plan was to run a strategy for 1 week and calculate a mean number of ARP points per day from the daily device totals for days 3 to 7, change strategy at the end of day 7 and repeat.
However, your strategy is an attractive one: using a stepwise change in strategy of 3 units each time, it would take 10 weeks to do the suggested cycle. It would certainly make for a good pilot.
[Aug 16, 2021 9:41:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Does that mean you have one machine with 8 threads, 2 with 4 threads and 1 with 2 threads?

As you are crunching all projects, I would have thought that you would want to maximise your total output and not just ARP.

24/0/0 seems to outputting 8.86 units per day, whereas 18/0/0 is 9.6 per day and 12/0/0 is 8.23 per day, based on your figures. That would seem to indicate that 18/0/0 is the best of those 3 options and that the I/O problems with ARP do not become a problem until above 18 ARP units.

However, If you were to compare points and include the points from OPN/MCM that should produce the best overall.

Mike
[Aug 16, 2021 10:33:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
leloft
Cruncher
Joined: Jun 8, 2017
Post Count: 23
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Does that mean you have one machine with 8 threads, 2 with 4 threads and 1 with 2 threads?
Yes: 1 x 2P, 2 x 4P, 1x 4P4V, 1 x 12P12V
As you are crunching all projects, I would have thought that you would want to maximise your total output and not just ARP.
I am trying to maximise my useful output. I assume that
(i) the fastest machines (quickest turnaround) are assigned the stragglers, i.e. to shorten the backlog faster in the tail than it is growing at the head; and
(ii) 12 units every ~30h is of more use than 24 every ~60h.
24/0/0 seems to outputting 8.86 units per day, whereas 18/0/0 is 9.6 per day and 12/0/0 is 8.23 per day, based on your figures. That would seem to indicate that 18/0/0 is the best of those 3 options and that the I/O problems with ARP do not become a problem until above 18 ARP units.
It's far to early to tell: those figures are not 'pure' i.e. the longer a unit takes, the more likely it is that the processing time is a result of a hybrid (exploratory) strategy. Hence the calculation of the mean from days 3-7 of a single strategy, after the previous strategy has been flushed out.
One thing that I am unclear about: do all the ARP units contain roughly the same amount of work (as measured by 'points')? I ask because a 4/0/0 strategy on the 4P4V machine can turnaround ~400 point units in ~15h (~108 points per hour); a 6/0/0 turns ~500 point units in ~20h (~150pph) and an 8/0/0 ~550 point units in ~28h(~160 pph). This roughly translates to a 4/0/0 strategy processing ~6.4 units per day; a 6/0/0 processing ~7.2 and a 8/0/0 processing ~6.9.
So are the 6.4 '15h' units more valuable than the 7.2 '20h' or the 6.9 '28h' ones? My idea is that they would be if they were 'backlog' units; if backlog units could be targeted to machines running a confirmed optimal turnaround strategy (perhaps the above 6/0/0 or 18/0/0) then that would satisfy my aim to maximise useful output by chipping away at the backlog, albeit at the 'cost' of overall points. In effect, satisfying work as opposed to best-paid work.
[Aug 17, 2021 9:45:51 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

I am all in favour of maximising output. but each machine should be evaluated on its own.

The best way is to determine the total points generated per day across all 3 projects (I am ignoring HST for lack of availability).

I think the best combination for your big machine will be about half ARP and the rest OPN/MCM, but I might be wrong. Every machine is different> The main problem with ARP is the amount of input/output required, especially at checkpoints and uploading. More RAM does help.

Check out the specifications required. Your smaller machines may not be ok for ARP.

Mike
[Aug 17, 2021 11:58:26 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Vester
Senior Cruncher
USA
Joined: Nov 18, 2004
Post Count: 325
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

I have an Intel Core i9-10850K @4.9GHz (ten cores/20 threads) that is currently running 19 ARP tasks. The computer is set to run all projects.
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by Vester at Aug 17, 2021 12:36:50 PM]
[Aug 17, 2021 12:26:39 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Vester

It is advisable to cut your ARP back to 10 units at a time using app_config.xml Ask here if you would like a copy and instructions. The ARP units require a grest deal of capacity, especially at each checkpoint (each 12.5%) and uploading.

Run OPN & MCM on the remainder of your threads.

You should set Project Limits in your Profile to 1 more each over what is in app_config.xml

That way you should get maximum output with minimum queue length and your machine will be considered 'reliable'.

Mike
[Aug 17, 2021 1:04:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Latest stats:

Average Generation: 83.6
Pace (average time to complete a generation): 4.0 (7-day average)
first_indexed       generation num_units_currently_on_generation num_units_completed_last_day 
------------------- ---------- --------------------------------- ----------------------------
2019-10-01 22:26:53 000
2019-10-30 18:58:54 001 6
2019-12-08 11:56:25 002
2020-01-12 02:02:34 003
2020-02-08 03:43:00 004
2020-02-24 06:27:42 005
2020-03-09 17:38:25 006
2020-03-17 08:44:19 007
2020-03-23 20:52:24 008
2020-04-01 14:39:46 009
2020-04-12 08:29:32 010
2020-04-21 02:41:36 011
2020-05-02 03:16:28 012
2020-05-10 13:29:40 013
2020-05-22 10:46:51 014
2020-06-02 21:07:48 015
2020-06-20 20:53:08 016
2020-07-01 12:31:12 017
2020-07-09 18:39:23 018
2020-07-18 16:08:31 019
2020-07-26 16:32:08 020
2020-08-08 15:15:22 021
2020-08-19 00:49:10 022
2020-08-24 07:02:09 023
2020-08-30 05:56:33 024
2020-09-04 11:35:58 025
2020-09-09 17:27:07 026
2020-09-15 06:25:11 027
2020-09-20 10:01:14 028
2020-09-25 22:07:49 029
2020-10-02 07:08:22 030
2020-10-07 17:55:57 031
2020-10-14 16:25:19 032
2020-10-18 20:05:40 033
2020-10-25 15:34:22 034
2020-10-31 22:55:26 035
2020-11-04 06:29:28 036
2020-11-12 06:33:47 037
2020-11-17 09:21:26 038
2020-11-24 13:47:28 039
2020-11-30 07:44:02 040
2020-12-07 20:20:00 041
2020-12-13 18:26:56 042
2020-12-20 00:33:11 043 2
2020-12-25 22:27:11 044 3
2021-01-01 07:57:34 045 1 4
2021-01-07 18:08:33 046 4 1
2021-01-15 02:41:00 047 1 2
2021-01-22 20:25:40 048 4
2021-01-28 10:53:04 049 6 1
2021-02-03 14:32:54 050 4 2
2021-02-09 03:20:45 051 2
2021-02-16 14:14:47 052 4 4
2021-02-22 01:22:20 053 7 2
2021-02-28 10:29:30 054 5 3
2021-03-06 18:23:14 055 7 3
2021-03-12 10:16:29 056 7 2
2021-03-17 08:30:15 057 5 2
2021-03-23 06:08:46 058 6 2
2021-03-29 22:39:10 059 6 3
2021-04-05 05:01:38 060 9 2
2021-04-10 21:09:07 061 5 4
2021-04-16 23:20:59 062 9 7
2021-04-22 07:50:06 063 12 4
2021-04-28 23:02:38 064 9 5
2021-05-04 04:45:55 065 10 3
2021-05-09 14:11:18 066 7 3
2021-05-16 14:55:41 067 9 6
2021-05-23 15:02:08 068 10 8
2021-05-26 06:43:43 069 14 8
2021-05-29 18:38:55 070 12 8
2021-06-03 15:46:15 071 13 5
2021-06-11 23:13:21 072 15 11
2021-06-15 11:54:58 073 17 11
2021-06-22 00:30:34 074 21 5
2021-06-27 11:56:43 075 13 5
2021-07-02 15:06:05 076 18 2
2021-07-08 20:49:12 077 12 15
2021-07-14 07:30:06 078 32 18
2021-07-18 14:21:26 079 80 41
2021-07-20 23:37:16 080 729 456
2021-07-23 21:00:51 081 3103 1110
2021-07-27 02:27:09 082 6020 1690
2021-07-29 02:04:50 083 6787 1706
2021-07-30 14:32:48 084 6403 1593
2021-08-03 02:15:23 085 5159 1229
2021-08-05 08:06:31 086 3524 775
2021-08-08 05:34:18 087 1969 407
2021-08-11 02:09:16 088 1183 108
2021-08-14 02:27:13 089 298 29
2021-08-17 22:32:03 090 29

[Aug 18, 2021 12:26:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

Thank you, Kevin.

As 080 is the latest generation I have been able to identify as labelled 'priority'. I will base this response on that. If anyone knows of a later batch please advise and I will adjust my figures. 081 should be a priority by now, but the only ones that I have seen have been resends.

There have been 30,382 units validated in generations up to and including 080 in the last 5 days, out of 95,685 returned. Allowing for 2 results being needed for each unit, that corresponds to 63.5% of work done.

There are now 13,840 units remaining to be crunched in those generations out of a total of 226,584 up to generation 089 (just 6.1%).

The stragglers are catching up, but the total is moving up.

However, those generation 001 's are still stuck. I reckon that they would take at least 3 months to catch up. Maybe even a year.

Mike
[Aug 18, 2021 6:35:15 PM]   Link   Report threatening or abusive post: please login first  Go to top 
leloft
Cruncher
Joined: Jun 8, 2017
Post Count: 23
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available

I am trying to understand the structure of these various generations and workunit priority; I am reading this thread from the beginning and trying to answer the following questions:
(i) What is a 'generation' of workunits?
(ii) I read that a unit is a 'plot of land'; What sort of sizes are they, typically (and atypically)?
(iii) How does a generation (i.e. 001) get stuck?
I'd greatly appreciate any pointers to articles that could answer these (and other) questions.
Many thanks
[Aug 19, 2021 10:10:38 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 3321   Pages: 333   [ Previous Page | 126 127 128 129 130 131 132 133 134 135 | Next Page ]
[ Jump to Last Post ]
Post new Thread