| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: Help Conquer Cancer
Thread: [RENAMED] Some concerns regarding the HCC project (page fault and poor performance, in particular (but not only) by multi-core hosts; #cores>2) |
| No member browsing this thread |
|
Thread Status: Locked Total posts in this thread: 210
|
|
| Author |
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Nope. There are a number of posts by knreed explaining this, some in the "Member News" forum. For starters in the quorum 2 arrangement, WCG knows exactly the claim history of every device. It looks at how far apart the 2 claims are. IF close enough, they are averaged. If far apart, the device history is taken into account (award per second) and the closest to history is the machine determining the credit for the quorum. As the precision is at seconds level, it regularly happens that the lower claim is upped to match the higher claim.
----------------------------------------Samples of HCC (as a popular item this moment): X0000042500293200411221320_ 1-- Valid 11/23/2007 03:25:27 11/23/2007 17:30:38 5.87 45.3 / 66.6 < sample of underclaimer X0000042500293200411221320_ 0-- Valid 11/23/2007 03:25:03 11/26/2007 15:15:13 4.33 66.6 / 66.6 (Moi on 4-core) X0000045460648200502081420_ 2-- Valid 11/23/2007 21:57:07 11/24/2007 04:09:54 6.01 77.1 / 70.4 X0000045460648200502081420_ 1-- Valid 11/23/2007 21:54:15 11/26/2007 21:02:45 4.13 63.7 / 70.4 (Moi on 4-core) X0000045390996200501241358_ 0-- Valid 11/23/2007 20:31:20 11/26/2007 19:36:03 4.23 65.2 / 65.2 (Moi on 4-core) X0000045390996200501241358_ 1-- Valid 11/23/2007 20:31:08 11/26/2007 08:44:53 5.90 89.5 / 65.2 X0000042580363200412071750_ 1-- Valid 11/23/2007 05:22:44 11/26/2007 16:55:18 4.34 66.8 / 75.8 (Moi on 4-core) X0000042580363200412071750_ 0-- Valid 11/23/2007 05:22:17 11/24/2007 05:40:38 8.97 84.8 / 75.8 X0000042490643200412132150_ 2-- Valid 11/23/2007 06:05:27 11/23/2007 18:13:05 11.68 80.1 / 70.9 X0000042490643200412132150_ 0-- Valid 11/23/2007 02:42:24 11/26/2007 10:17:02 4.01 61.7 / 70.9 (Moi on 4-core) X0000040461297200410131753_ 0-- Valid 11/26/2007 00:15:57 11/26/2007 15:55:59 7.14 73.4 / 67.3 X0000040461297200410131753_ 1-- Valid 11/26/2007 00:15:07 11/27/2007 08:53:28 3.99 61.1 / 67.3 (Moi on 4-core) This machine (Moi) is not affected by the PF Delta issue clouding the run times and benchmark is always automatically run at night (once every 5 days), thus near optimal, at stock. added: the very extremes of KerSamson i have not seen on any of my results. Think some of those are cases of 2 meeting where one is somewhat effected by the PF delta issue and the other very badly.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Nov 27, 2007 9:05:44 AM] |
||
|
|
zombie67 [MM]
Senior Cruncher USA Joined: May 26, 2006 Post Count: 228 Status: Offline Project Badges: 
|
Wow. Those rules just cloud the real problem all the further.
----------------------------------------In any case, the differences you use as examples are not what we are talking about here. Those are tiny variances in comparison.  |
||
|
|
zombie67 [MM]
Senior Cruncher USA Joined: May 26, 2006 Post Count: 228 Status: Offline Project Badges:
|
Here are some real problems:
----------------------------------------X0000040361335200410081511_ 1-- Valid 11/25/2007 17:45:54 11/26/2007 11:14:05 3.93 87.6 / 159.3 X0000040361335200410081511_ 0-- Valid 11/25/2007 17:45:52 11/26/2007 03:05:47 8.94 159.3 / 159.3 X0000040361408200410081509_ 0-- Valid 11/25/2007 17:52:57 11/26/2007 06:32:43 9.29 161.9 / 109.4 X0000040361408200410081509_ 1-- Valid 11/25/2007 17:48:55 11/26/2007 11:12:49 5.80 109.4 / 109.4 X0000040361416200410081509_ 1-- Valid 11/25/2007 17:52:57 11/26/2007 07:30:50 9.92 172.1 / 172.1 X0000040361416200410081509_ 0-- Valid 11/25/2007 17:49:22 11/26/2007 01:33:38 3.66 56.8 / 172.1 X0000040200283200410222323_ 1-- Valid 11/25/2007 10:31:23 11/25/2007 21:35:05 10.08 174.8 / 174.8 X0000040200283200410222323_ 0-- Valid 11/25/2007 10:31:22 11/26/2007 05:09:48 8.24 79.7 / 174.8 X0000040200284200410222323_ 0-- Valid 11/25/2007 10:31:23 11/25/2007 21:15:10 10.38 180.0 / 88.1 X0000040200284200410222323_ 1-- Valid 11/25/2007 10:31:18 11/26/2007 09:26:29 8.28 88.1 / 88.1 X0000040200285200410222323_ 1-- Valid 11/25/2007 10:31:34 11/26/2007 14:03:31 9.24 75.9 / 75.9 X0000040200285200410222323_ 0-- Valid 11/25/2007 10:31:23 11/25/2007 21:36:45 10.12 175.5 / 75.9 |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Those rules were created for a good reason and your samples, though cant see who is who, wonderfully demonstrates that the 'closeness' rule works to satisfaction in both directions. The root here quite certainly is the inefficiency of a (un)certain group of machines displayed in processing HCC. Hope the techs will address that soon, but think getting HCC running on Linux is more pressing.
----------------------------------------PS, the present benchmark itself i think is good for the horse manure pit, but not entering into any elaboration as to how it could be done differently or better. Discussed ad nauseam.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Nov 27, 2007 10:29:58 AM] |
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1673 Status: Offline Project Badges: 
|
Hi you both,
----------------------------------------I share totally the observation and opinion of Zombie67. Mainly HCC WUs are computed by only two hosts. Indeed, sometime the granted points is the average of the two claimed point amounts (this behaviour meet the usual description of the points allocation mechanism), BUT sometime this granting process seems to run absolutely "randomized": - sometime the "less claimer" is dominant - sometime the "most claimer" is dominant. I cannot recognized any pattern for this behaviour. I don't have any information about the second host which computed the data (multi core or single core, 32 or 64 bit OS, ...). Finally, I can only confirm that it looks like that something is going wrong. Cheers, |
||
|
|
Movieman
Veteran Cruncher Joined: Sep 9, 2006 Post Count: 1042 Status: Offline |
Nope. There are a number of posts by knreed explaining this, some in the "Member News" forum. For starters in the quorum 2 arrangement, WCG knows exactly the claim history of every device. It looks at how far apart the 2 claims are. IF close enough, they are averaged. If far apart, the device history is taken into account (award per second) and the closest to history is the machine determining the credit for the quorum. As the precision is at seconds level, it regularly happens that the lower claim is upped to match the higher claim. Samples of HCC (as a popular item this moment): X0000042500293200411221320_ 1-- Valid 11/23/2007 03:25:27 11/23/2007 17:30:38 5.87 45.3 / 66.6 < sample of underclaimer X0000042500293200411221320_ 0-- Valid 11/23/2007 03:25:03 11/26/2007 15:15:13 4.33 66.6 / 66.6 (Moi on 4-core) X0000045460648200502081420_ 2-- Valid 11/23/2007 21:57:07 11/24/2007 04:09:54 6.01 77.1 / 70.4 X0000045460648200502081420_ 1-- Valid 11/23/2007 21:54:15 11/26/2007 21:02:45 4.13 63.7 / 70.4 (Moi on 4-core) X0000045390996200501241358_ 0-- Valid 11/23/2007 20:31:20 11/26/2007 19:36:03 4.23 65.2 / 65.2 (Moi on 4-core) X0000045390996200501241358_ 1-- Valid 11/23/2007 20:31:08 11/26/2007 08:44:53 5.90 89.5 / 65.2 X0000042580363200412071750_ 1-- Valid 11/23/2007 05:22:44 11/26/2007 16:55:18 4.34 66.8 / 75.8 (Moi on 4-core) X0000042580363200412071750_ 0-- Valid 11/23/2007 05:22:17 11/24/2007 05:40:38 8.97 84.8 / 75.8 X0000042490643200412132150_ 2-- Valid 11/23/2007 06:05:27 11/23/2007 18:13:05 11.68 80.1 / 70.9 X0000042490643200412132150_ 0-- Valid 11/23/2007 02:42:24 11/26/2007 10:17:02 4.01 61.7 / 70.9 (Moi on 4-core) X0000040461297200410131753_ 0-- Valid 11/26/2007 00:15:57 11/26/2007 15:55:59 7.14 73.4 / 67.3 X0000040461297200410131753_ 1-- Valid 11/26/2007 00:15:07 11/27/2007 08:53:28 3.99 61.1 / 67.3 (Moi on 4-core) This machine (Moi) is not affected by the PF Delta issue clouding the run times and benchmark is always automatically run at night (once every 5 days), thus near optimal, at stock. added: the very extremes of KerSamson i have not seen on any of my results. Think some of those are cases of 2 meeting where one is somewhat effected by the PF delta issue and the other very badly. I think the issue that's getting lost here is that the people with 8 core machines plus other machines are seeing the machines with 4 cores or less doing fairly well in the quorum while the 8 core machines are getting hammered at times depending on who they get matched up with. My 8 core clovers do essentially nothing except WCG and the benches stay pretty consistant: 9/24/2007 8:21:35 PM||Benchmark results: 9/24/2007 8:21:35 PM|| Number of CPUs: 8 9/24/2007 8:21:35 PM|| 2982 floating point MIPS (Whetstone) per CPU 9/24/2007 8:21:35 PM|| 6676 integer MIPS (Dhrystone) per CPU -------------------------------------------------------------------------------------- 11/14/2007 2:43:38 PM||Benchmark results: 11/14/2007 2:43:38 PM|| Number of CPUs: 8 11/14/2007 2:43:38 PM|| 2956 floating point MIPS (Whetstone) per CPU 11/14/2007 2:43:38 PM|| 6506 integer MIPS (Dhrystone) per CPU ----------------------------------------------------------------------------------- What I find hard to swallow to be blunt is that the majority of the machines that run WCG that also run other processes are turning in a more consistant bench than I am and thats why I'm getting the "lesser" award in these cases? Think about that for a few minutes. It's just not logical that a machine that's used for many things will bench more consistant than one that is limited to a very defined area of work. To zombies comment on the 8 way taking a longer timeframe: I can only comment on my machines. Mine run at over 3000mhz and my experience with the FAAH units is they do 8 at a time in the same or darned close to the same timeframe as any machine(C2D architecture) at the same speed. IE: I do 8 WU in the same timeframe that a Q6600 running at 3000mhz does 4 WU.  [Edit 1 times, last edit by Movieman at Nov 27, 2007 10:44:43 AM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Hi Movieman,
----------------------------------------Think that what is discussed in the Page Fault thread is the cause. Not sure, but think if you stick to just doing HCC exclusively the averages will work their way up. A very steady benchmark is what promotes how the algorithm handles claims and quorum. Regarding your line "It's just not logical that a machine that's used for many things will bench more consistent than one that is limited to a very defined area of work.". I know what you are saying, consistently too low (see next para)! I know from observation that the timing of that can cause huge sways in the values, particular discwriting, so one night got up and forced one.... its now always running at night on the 24-7 machine (Have to get up again after an upgrade  Heck am i a self confessed points addict or just motivated to help others get what drives their motivation  ).And this from the very secret Italian room.... set the BOINC.exe to the second highest priority. The program does very little until it does that once-per 5 day test and will run it with elevated attention. I repeat myself: I think it BS that the bench is impacted by runtime events. The credits are computed on CPU seconds and not wall-clock... think that suites your thinking or  End of the off-topic
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1673 Status: Offline Project Badges:
|
My observations (and concerns) apply for HCC project only.
----------------------------------------By all the other projects, I did not experience nor observe a similar behaviour and so a poor granting. The poor granting is logical if we consider that single or dual core hosts indeed do compute much faster HCC WUs than a quad core (or double quad core) hosts. I cannot not understand the reason for the slow computing of quad core systems with HCC. Yesterday, we noticed that very few memory space is used by computing HCC WUs in comparison to HFP2 or DDDT WUs. Again, because we don't have all the necessary information available, it is difficult to identify a pattern. Based on the feedbacks and comments provided by several crunchers, we can suppose that HCC has some problems for managing 64bit systems and/or quad core systems. At this place, I cannot investigate further as long I don't have more information about host configuration. Anyway, because less powerfull systems work well on HCC, I think that it is necessary, for this particular project and for the future as well, to investigate deeper and closer in order to avoid wasting time and resources. Cheers, |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Looks like it could be a combination of the really high Integer benchmark on the 8 core machines and the higher than expected page faults on HCC alone slowing them down that could be producing these silly numbers.
|
||
|
|
zombie67 [MM]
Senior Cruncher USA Joined: May 26, 2006 Post Count: 228 Status: Offline Project Badges:
|
Right. The problem with HCC is not the credits. That is just one symptom.
----------------------------------------The problem is that a E6600 (dual core, 2.4ghz) is 2-4 times faster than a dual Xeon 5355 (8-core 2.66ghz), both running XP. Both have the same core architecture. There is no reason the 8-way machines should be so much slower, other than a problem with the application. |
||
|
|
|