| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: BOINC Agent Support
Thread: Problem: Invalid Working Units in large numbers. Please help. |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 68
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Dear WCG administrators:
----------------------------------------In the past few weeks we have been observing a rather disturbing phenomenon. We are seeing valid results (at least as far as we can tell) submitted by our team members, being invalidated and resubmitted again for re computation. This is not limited to a specific installation. In fact it covers a wide spectrum: Boinc client: all standard clients, 5.4.9 and 5.4.11 Hardware spec: AMD, Intel, and Mac Operating system: W2K3, WinXP, XP64 Pro, Linux, OSX WCG project: faah, cancer, and also Betas We are concerned that this may be widespread not only among team members but also among WCG participants as a whole. If so, then considerable computer power may be continuously wasted. Individual participants may assume that such an occasional "Invalid" result is acceptable, but for a team that has been running these working units 24/7 non stop for the last three months on stable hardware, it is possible for us to spot this trend. It started two weeks ago and is still continuing to this day. We were hoping that this would be an occasional problem associated with the transfer to the new scoring system, but apparently it is not; the problem persists. We hope that you give this problem your utmost attention and advise on the action needed on our part (if any) to come to a quick resolution to this issue. Attached below, please find a (twenty or so) sample, just to illustrate the extent of the problem. If further info is needed, please do not hesitate to contact us. Best regards meshmesh On behalf of team XtremeSystems PS: to all in the USA, happy thanks giving day. Edit 28-Nov Removed some examples where Computer IDs could not be ascertained. Added a few new recent examples. There are hundreds of these in case more info is needed. I wonder whether some change in the validation procedure that occurred during the first week of November, around the same time the new crediting system was introduced, has contributed to this problem. Maybe a plot of the daily percentages of Invalid results re submitted for calculation (not the Invalid due to optimised clients, etc) would help pin point the problem. Please allow me to convey that the level of frustration with the wasted computer cycles is getting really high. Seeing a list of a dozen WCG website pages of Invalid results from a computer farm all running standard clients is disheartening. And it just keeps coming. I do understand that returning back from a long weekend would have a backlog of work, but I hope that this issue would be given some urgency, if possible, and would be highly appreciated. If further info is needed, please don't hesitate. Thank you very much. ======== Start of Quorum Samples ============= Legend: machine specs Workunit_Name Status Sent_DateTime Return_DateTime CPU_Time Claimed/Granted_Credit Conroe E6700 @1066 +46 with 2 gig DDR2 @ 4:5 Computer ID: 80793 faah0883_ bdb184_ mx2bpz_ dry_ 03 Valid 11/11/2006 21:38:37 11/12/2006 06:19:30 7.36 50/57 faah0883_ bdb184_ mx2bpz_ dry_ 03 Invalid 11/07/2006 05:24:48 11/11/2006 21:37:09 2.67 63/28 <== faah0883_ bdb184_ mx2bpz_ dry_ 03 Valid 11/07/2006 05:24:44 11/09/2006 13:17:45 6.44 47/57 faah0883_ bdb184_ mx2bpz_ dry_ 03 Valid 11/07/2006 05:16:39 11/09/2006 10:11:28 47.82 72/57 faah0882_ bdb088_ mx2bpz_ 03 Valid 11/13/2006 10:03:52 11/13/2006 22:18:42 8.53 60/49 faah0882_ bdb088_ mx2bpz_ 03 Valid 11/12/2006 21:49:25 11/13/2006 09:55:42 5.62 44/49 faah0882_ bdb088_ mx2bpz_ 03 No Reply 11/05/2006 21:48:16 11/12/2006 21:48:16 0.00 0/0 faah0882_ bdb088_ mx2bpz_ 03 Invalid 11/05/2006 21:48:08 11/09/2006 08:24:52 2.68 62/25 <== faah0882_ bdb088_ mx2bpz_ 03 Valid 11/05/2006 21:42:55 11/06/2006 05:25:06 4.78 44/49 faah0881_ bdb267_ mx2bpx_ dry_ 04 Valid 11/07/2006 22:28:39 11/08/2006 17:25:47 13.50 59/57 faah0881_ bdb267_ mx2bpx_ dry_ 04 Invalid 11/04/2006 23:27:24 11/07/2006 22:24:05 2.65 62/28 <== faah0881_ bdb267_ mx2bpx_ dry_ 04 Valid 11/04/2006 23:23:32 11/05/2006 11:15:00 7.76 53/57 faah0881_ bdb267_ mx2bpx_ dry_ 04 Valid 11/04/2006 23:22:35 11/05/2006 13:07:52 9.61 57/57 faah0881_ bdb241_ mx2bpy_ 00 Valid 11/08/2006 19:11:39 11/09/2006 05:27:12 5.79 44/50 faah0881_ bdb241_ mx2bpy_ 00 Valid 11/04/2006 23:26:28 11/08/2006 18:49:10 5.91 56/50 faah0881_ bdb241_ mx2bpy_ 00 Valid 11/04/2006 23:25:38 11/06/2006 11:24:25 7.07 80/50 faah0881_ bdb241_ mx2bpy_ 00 Invalid 11/04/2006 23:22:12 11/06/2006 22:35:42 2.62 62/25 <== faah0881_ bdb271_ mx2bpy_ 07 Valid 11/06/2006 22:38:46 11/07/2006 23:27:17 6.35 64/67 faah0881_ bdb271_ mx2bpy_ 07 Valid 11/04/2006 23:32:04 11/05/2006 21:04:12 7.40 67/67 faah0881_ bdb271_ mx2bpy_ 07 Valid 11/04/2006 23:29:52 11/05/2006 20:55:26 13.49 69/67 faah0881_ bdb271_ mx2bpy_ 07 Invalid 11/04/2006 23:22:12 11/06/2006 22:36:14 2.88 68/33 <== MacPro 2.66 Woodcrest 2GB FB-DIMM OSX Computer ID: 287228 B10233_ 0127_ CTMA4A-40-18-15 Valid 11/12/2006 13:55:39 11/12/2006 19:11:07 2.18 33/33 B10233_ 0127_ CTMA4A-40-18-15 Valid 11/12/2006 07:45:15 11/12/2006 10:34:26 1.91 32/33 B10233_ 0127_ CTMA4A-40-18-15 Valid 11/12/2006 07:40:28 11/12/2006 13:28:43 2.36 34/33 B10233_ 0127_ CTMA4A-40-18-15 Invalid 11/12/2006 07:37:56 11/12/2006 12:46:59 1.90 36/17 <== B05540_ 0019_ CTMA3B2-7-10-0-c2 Valid 11/20/2006 19:03:20 11/21/2006 05:29:54 2.38 28/26 B05540_ 0019_ CTMA3B2-7-10-0-c2 Invalid 11/20/2006 01:44:06 11/20/2006 07:29:14 1.23 23/13 <== B05540_ 0019_ CTMA3B2-7-10-0-c2 Valid 11/19/2006 23:49:53 11/20/2006 18:18:59 1.64 21/26 B05540_ 0019_ CTMA3B2-7-10-0-c2 Valid 11/19/2006 23:39:49 11/20/2006 11:17:45 3.16 29/26 Sempron 2800+ @2800Mhz, 1gb TCCD Computer ID: 71980 B05522_ 0133_ CTMA3A2-6-17-21-c2 Valid 11/20/2006 01:39:49 11/20/2006 05:54:33 1.19 17/20 B05522_ 0133_ CTMA3A2-6-17-21-c2 Invalid 11/18/2006 00:07:18 11/20/2006 01:24:47 1.45 21/10 <== B05522_ 0133_ CTMA3A2-6-17-21-c2 Valid 11/18/2006 00:05:12 11/18/2006 09:18:37 1.88 15/20 B05522_ 0133_ CTMA3A2-6-17-21-c2 Valid 11/18/2006 00:03:37 11/19/2006 18:58:00 2.18 27/20 B05530_ 0271_ CTMA3B1-7-7-21-c2 Valid 11/20/2006 13:38:19 11/20/2006 17:10:17 1.97 24/20 B05530_ 0271_ CTMA3B1-7-7-21-c2 Valid 11/18/2006 19:45:58 11/19/2006 03:24:08 4.04 15/20 B05530_ 0271_ CTMA3B1-7-7-21-c2 Invalid 11/18/2006 19:45:51 11/20/2006 12:54:02 1.53 22/10 <== B05530_ 0271_ CTMA3B1-7-7-21-c2 Valid 11/18/2006 19:41:15 11/20/2006 04:07:00 1.78 22/20 x2 4600+ @ 2.65 - 2x1gb ram - XP 64pro Boinc 5.4.11 Computer ID: 68354 B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/18/2006 22:18:00 11/19/2006 03:02:52 1.78 25/22 B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/17/2006 10:57:49 11/17/2006 15:57:52 3.06 17/22 B05517_ 0195_ CTMA3A1-6-22-15-c1 Invalid 11/17/2006 10:54:46 11/18/2006 03:59:19 2.41 36/11 <== B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/17/2006 10:50:24 11/18/2006 21:37:18 1.73 24/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/18/2006 04:11:09 11/20/2006 01:32:15 2.16 17/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/17/2006 12:20:27 11/17/2006 22:10:40 2.00 22/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Error 11/17/2006 09:17:29 11/17/2006 12:07:29 0.70 4/0 B05516_ 0345_ CTMA3C2-12-8-25-c2 Invalid 11/17/2006 09:15:08 11/18/2006 03:59:19 2.23 33/11 <== B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/17/2006 09:13:26 11/17/2006 14:08:15 1.73 25/22 iMac G5 1.8 PPC - 1gb ram - 10.4 Boinc 5.4.9 Computer ID: 62916 B05531_ 0104_ CTMA3B1-12-16-1-c1 Valid 11/19/2006 17:53:20 11/21/2006 02:08:34 10.56 37/38 B05531_ 0104_ CTMA3B1-12-16-1-c1 Valid 11/18/2006 22:38:18 11/19/2006 11:38:33 9.03 37/38 B05531_ 0104_ CTMA3B1-12-16-1-c1 Invalid 11/18/2006 22:33:46 11/19/2006 14:16:52 8.72 68/19 <== B05531_ 0104_ CTMA3B1-12-16-1-c1 Valid 11/18/2006 22:26:51 11/19/2006 15:55:31 4.25 38/38 x2 4600+ @ 2.65 - 2x1gb ram - XP 64pro Boinc 5.4.11 Computer ID 68354 B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/18/2006 22:18:00 11/19/2006 03:02:52 1.78 25/22 B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/17/2006 10:57:49 11/17/2006 15:57:52 3.06 17/22 B05517_ 0195_ CTMA3A1-6-22-15-c1 Invalid 11/17/2006 10:54:46 11/18/2006 03:59:19 2.41 36/11 <== B05517_ 0195_ CTMA3A1-6-22-15-c1 Valid 11/17/2006 10:50:24 11/18/2006 21:37:18 1.73 24/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/18/2006 04:11:09 11/20/2006 01:32:15 2.16 17/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/17/2006 12:20:27 11/17/2006 22:10:40 2.00 22/22 B05516_ 0345_ CTMA3C2-12-8-25-c2 Error 11/17/2006 09:17:29 11/17/2006 12:07:29 0.70 4/0 B05516_ 0345_ CTMA3C2-12-8-25-c2 Invalid 11/17/2006 09:15:08 11/18/2006 03:59:19 2.23 33/11 <== B05516_ 0345_ CTMA3C2-12-8-25-c2 Valid 11/17/2006 09:13:26 11/17/2006 14:08:15 1.73 25/22 iMac G5 1.8 - 1ghz Ram - OSx 10.4 Computer ID 62916 Note: This is a slow machine that overclaims, but this is another matter. B05501_ 0052_ YTMA10B2-5-10-17-c1 Valid 11/23/2006 08:49:45 11/23/2006 12:57:32 1.99 18/18 B05501_ 0052_ YTMA10B2-5-10-17-c1 Valid 11/22/2006 22:33:07 11/23/2006 08:09:21 2.01 18/18 B05501_ 0052_ YTMA10B2-5-10-17-c1 No Reply 11/15/2006 21:22:57 11/22/2006 21:22:57 0.00 0/0 B05501_ 0052_ YTMA10B2-5-10-17-c1 Valid 11/15/2006 21:15:12 11/16/2006 00:41:10 1.99 18/18 B05501_ 0052_ YTMA10B2-5-10-17-c1 Invalid 11/15/2006 21:11:47 11/19/2006 01:58:45 27.39 217/9 <== 2.0 P4 - 1 ghz ram - XP Pro 5.4.11 Computer ID 67624 B05554_ 0064_ CTMA4A-34-15-11-c2 Valid 11/23/2006 11:09:47 11/24/2006 18:39:51 1.52 22/19 B05554_ 0064_ CTMA4A-34-15-11-c2 Valid 11/21/2006 18:48:22 11/22/2006 14:46:22 1.94 21/19 B05554_ 0064_ CTMA4A-34-15-11-c2 Valid 11/21/2006 18:47:37 11/23/2006 10:55:00 2.38 14/19 B05554_ 0064_ CTMA4A-34-15-11-c2 Invalid 11/21/2006 18:46:06 11/22/2006 06:40:24 5.51 34/9 <=== 1ghz T-Bird - 2x256ram - XP Pro 5.4.11 Computer ID 73587 B05537_ 0287_ CTMA3B2-6-9-17-c1 Valid 11/23/2006 16:41:10 11/24/2006 05:55:35 2.53 18/18 B05537_ 0287_ CTMA3B2-6-9-17-c1 Invalid 11/19/2006 19:58:46 11/20/2006 05:55:38 2.80 33/9 <== B05537_ 0287_ CTMA3B2-6-9-17-c1 Valid 11/19/2006 19:58:18 11/23/2006 16:13:06 2.60 17/18 B05537_ 0287_ CTMA3B2-6-9-17-c1 Valid 11/19/2006 19:57:19 11/20/2006 01:16:17 2.88 18/18 x2 4400 @ 2.2 2x1ghz ram XP pro 5.4.11 Computer ID 65518 faah0955_ d076n065_ x1AJX_ 02 Valid 11/26/2006 17:47:02 11/27/2006 09:39:31 10.17 66/65 faah0955_ d076n065_ x1AJX_ 02 Invalid 11/26/2006 09:00:13 11/26/2006 17:41:49 5.35 67/32 <== faah0955_ d076n065_ x1AJX_ 02 Valid 11/19/2006 09:05:07 11/20/2006 06:09:27 10.51 68/65 faah0955_ d076n065_ x1AJX_ 02 Error 11/19/2006 08:51:53 11/19/2006 08:57:03 0.00 0/0 faah0955_ d076n065_ x1AJX_ 02 Valid 11/19/2006 08:51:51 11/19/2006 16:00:23 3.14 61/65 faah0955_ d076n065_ x1AJX_ 02 Invalid 11/19/2006 08:48:51 11/26/2006 15:14:01 1.52 19/19 B05535_ 0262_ CTMA3B1-14-3-18-c1 Valid 11/26/2006 18:58:05 11/27/2006 04:56:26 3.07 17/20 B05535_ 0262_ CTMA3B1-14-3-18-c1 Invalid 11/26/2006 12:15:49 11/26/2006 17:41:49 1.91 24/10 <== B05535_ 0262_ CTMA3B1-14-3-18-c1 Invalid 11/19/2006 11:44:15 11/27/2006 07:07:06 4.28 48/10 B05535_ 0262_ CTMA3B1-14-3-18-c1 Valid 11/19/2006 11:38:55 11/20/2006 17:54:15 2.23 20/20 B05535_ 0262_ CTMA3B1-14-3-18-c1 Valid 11/19/2006 11:37:58 11/19/2006 17:27:57 3.12 24/20 faah0959_ d085n266_ x1AJX_ 00 In Progress 11/28/2006 04:31:20 11/30/2006 11:57:44 0.00 0/0 faah0959_ d085n266_ x1AJX_ 00 Invalid 11/27/2006 21:48:54 11/28/2006 04:27:09 6.34 79/32 <== faah0959_ d085n266_ x1AJX_ 00 Valid 11/20/2006 22:01:37 11/22/2006 03:20:18 17.24 54/63 faah0959_ d085n266_ x1AJX_ 00 Error 11/20/2006 21:48:56 11/20/2006 21:54:05 0.00 0/0 faah0959_ d085n266_ x1AJX_ 00 Valid 11/20/2006 21:44:05 11/28/2006 13:35:10 3.57 73/63 faah0959_ d085n266_ x1AJX_ 00 Valid 11/20/2006 21:35:48 11/21/2006 10:09:01 6.96 62/63 ======== End of Quorum Samples ============= [Edit 2 times, last edit by Former Member at Nov 28, 2006 6:06:15 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hello meshmesh,
You need to add the Device ID for each machine as well as the work unit ID so that knreed can look at their results. Lawrence |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Meshmesh, your samples are interesting. Some of them are more interesting than the others. Take this one, 5th in your list .....
faah0881_ bdb241_ mx2bpy_ 00 Valid 11/08/2006 19:11:39 11/09/2006 05:27:12 5.79 44/50 faah0881_ bdb241_ mx2bpy_ 00 Valid 11/04/2006 23:26:28 11/08/2006 18:49:10 5.91 56/50 faah0881_ bdb241_ mx2bpy_ 00 Valid 11/04/2006 23:25:38 11/06/2006 11:24:25 7.07 80/50 faah0881_ bdb241_ mx2bpy_ 00 Invalid 11/04/2006 23:22:12 11/06/2006 22:35:42 2.62 62/25 <== The invalid result (the 4th result in the list) claimed 62 and was declared invalid and granted 25 because it was an outlier. But, but, but, but how can 62 be an outlier when the 3rd result claimed 80 and was declared valid? We have an obvious case of persecution of team XS here! Probably the fault of that scoundrel Dagorath. But wait a minute.... The mean of the 3 valid claims (44, 56 and 80) is 60 yet the credits granted indicate the mean is 50. Huh? How can that be?? Double check , 44 + 56 + 80 = 180. 180 divided by 3 = 60, not 50. 180 divided by 4 = 45, again not 50 It looks to me like the 80 claim is not what was actually claimed. I bet the third claim was in fact 50 and not 80. Just a typo? Yeah, right! Throw the bums out!!!! --- |
||
|
|
mike047
Senior Cruncher Joined: Aug 22, 2006 Post Count: 262 Status: Offline Project Badges: 
|
Lawrence,
----------------------------------------aren't the wu's unique and there fore would not need the device id? I am ignorant as to how these are inter-related, but unless the wu's are run repeatedly...there is only one set to review. The problems that I have with the invalids, seems to be TIME related. My work will report nearly the same credit but does the work in a significantly shorter period of time[one half sometimes]. It seems as though if paired with two slower boxes the faster takes the "hit" regardless of the points claim.
mike
----------------------------------------Crunch Hard, Crunch Often  [Edit 1 times, last edit by mike047 at Nov 23, 2006 1:23:29 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
meshmesh, it would be helpful if you could take a look at the stderr text for the failed work units.
We have experienced a few issues lately, but the ones reported so far do not fit your experience very well. In particular, we have had no FAAH issues on the BOINC platform recently. I think there is some issue with HDC, since we have had a few reports about that. And for the conspiracy theorists: the claimed/granted points and times seem perfectly consistent to me. When a work unit fails, it may easily behave differently to ones that complete normally. |
||
|
|
mike047
Senior Cruncher Joined: Aug 22, 2006 Post Count: 262 Status: Offline Project Badges:
|
What should I look for , in particular in the log. I certainly am not complaining , I am curious as to the reason for the invalidity of work on a consistant performing machine. Nearly all my boxes claim in the 70's and are on either 5.4.9/5.4.11 client. Sometimes the differnce in claimed points in the quorum is only about 10+/- points. What is the deciding factor in the choice of the invalid result??
----------------------------------------I run only faah. Just curious and understand that there is nothing I can do or that you can do. Edit example: faah0958_ d083n433_ x1AJX_ 01 Valid 11/22/2006 16:06:16 11/23/2006 09:12:22 11.82 76 / 77 faah0958_ d083n433_ x1AJX_ 01 Valid 11/20/2006 11:30:56 11/21/2006 02:56:31 11.60 76 / 77 faah0958_ d083n433_ x1AJX_ 01 Invalid 11/20/2006 11:30:20 11/22/2006 16:00:17 5.30 77 / 39 faah0958_ d083n433_ x1AJX_ 01 Valid 11/20/2006 11:23:21 11/21/2006 12:54:20 13.50 80 / 77 mine is the invalid:( device ID; 80817
mike
----------------------------------------Crunch Hard, Crunch Often [Edit 5 times, last edit by mike047 at Nov 23, 2006 3:23:13 PM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
meshmesh
----------------------------------------Overall, i had 1 (HDC) invalid on the past 130 or so WU's i processed in the last few weeks. The only change i know of for HDC and FAAH on BOiNC is the added graphics. Reports for FAAH continue to be very low. Do u have a % of 'invalid' WU's where a 4th copy was send out? mike047 You're right that each WU is unique and only so many copies are send out until quorum is achieved. The device ID just makes it easy to quickly identify and filter all 'invalid' cases out for a given machine in the database, rather than looking for a long complicated number, one by one. That allows for looking at commonalities in the errors on one machine. The number that we see in the Result Status Page is though not the entire number..... compare it with what u see in BOiNC message screen and note, that there are supplemental characters.... thus strictly the number shown in the Result Status page might not be all too unique. Like u always curious as to why something is failing... the mentioned graphics is the only change I'm aware of. cheers
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
mike047
Senior Cruncher Joined: Aug 22, 2006 Post Count: 262 Status: Offline Project Badges:
|
Device ID=Device name???
----------------------------------------I'll post the Id with the info if I know what it is. I have two invalids on the example machine within two day apart.
mike
Crunch Hard, Crunch Often |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
It's usually in the top part of the message log, when u start up BOiNC. Also u can see it in BOINCview for each device in the complete work tab... something like Computer ID...45678
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
mike047
Senior Cruncher Joined: Aug 22, 2006 Post Count: 262 Status: Offline Project Badges:
|
Cool, learned something new today, not bad for an OLD guy,eh??
----------------------------------------I'll edit the example to provide the information. THANKS
mike
Crunch Hard, Crunch Often |
||
|
|
|