| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Mapping Cancer Markers Forum
Thread: Too many Pending Validation |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 144
|
|
| Author |
|
|
cz50975
Advanced Cruncher Joined: Dec 9, 2004 Post Count: 95 Status: Offline Project Badges: 
|
there is problem with some MCM validators from approximately 2023-02-06 21:30:33 UTC
----------------------------------------Here is example - WU MCM1_0196018_0172 https://www.worldcommunitygrid.org/contribution/workunit/260080169 MCM1_0196018_0172_0 Win11 PenVal 2023-02-05 07:42:05 UTC 2023-02-06 21:30:33 UTC 2.67 / 2.84 93.9 / 0 MCM1_0196018_0172_1 Win10 PenVal 2023-02-05 07:42:12 UTC 2023-02-05 14:22:48 UTC 1.32 / 1.34 75.2 / 0 Now majority of uploaded MCM WUs waiting in queue for validation as example above. [Edit 2 times, last edit by cz50975 at Feb 7, 2023 9:50:10 PM] |
||
|
|
Aperture_Science_Innovators
Advanced Cruncher United States Joined: Jul 6, 2009 Post Count: 139 Status: Offline Project Badges: 
|
Adri, does your automation give you a way to quickly find the WUs where all devices have crunched & returned them?
---------------------------------------- |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2159 Status: Offline Project Badges: 
|
Adri, does your automation give you a way to quickly find the WUs where all devices have crunched & returned them? I'm not quite sure if I'm understanding you correctly, but if I do, then I do indeed have a list of WUs with OS and returned times for each task in each WU (and there were 356 of them, with an additional 2 already being Valid when I was compiling the list).  The list has been uploaded to pending-twofolds-triplets.html If you were asking me how I did it, then it is a bit more complex. In any case, I have been using wcgstats and Perl.  Adri [Edit 3 times, last edit by adriverhoef at Feb 7, 2023 11:13:58 PM] |
||
|
|
Aperture_Science_Innovators
Advanced Cruncher United States Joined: Jul 6, 2009 Post Count: 139 Status: Offline Project Badges:
|
Adri, does your automation give you a way to quickly find the WUs where all devices have crunched & returned them? I'm not quite sure if I'm understanding you correctly, but if I do, then I do indeed have a list of WUs with OS and returned times for each task in each WU (and there were 356 of them, with an additional 2 already being Valid when I was compiling the list). The list has been uploaded to pending-twofolds-triplets.html If you were asking me how I did it, then it is a bit more complex. In any case, I have been using wcgstats and Perl. Adri That's the question I meant to be asking, yeah. Just phrased inelegantly because I forgot what you'd called your tool :) Seems like a pretty straightforward thing to automate, have something that queries the API for all Pending Validation WUs, then filters it down to ones where no host still has it in progress. Perl perhaps not being the language I'd choose, but I'm sure it works well. I see we were wingmen for each other on at least one task (I had the _0)...guess your systems are much faster than mine... [Edit 1 times, last edit by Aperture_Science_Innovators at Feb 8, 2023 3:07:09 AM] |
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7664 Status: Offline Project Badges: 
|
I agree there is a problem with the MCM validator. I currently show 557 Pending validation where I usually show maybe 50 to 100, sometimes even less.
----------------------------------------I also took a look at the global totals for this project. There is usually 800,000 to 1,000,000 units returned per day. For Feb. 7, 2023 the total shows 558,373 units returned. So there is probably 300,000 to 500,000 units stuck in the validation stage. Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1951 Status: Offline Project Badges:
|
I see we were wingmen for each other on at least one task (I had the _0)...guess your systems are much faster than mine... Those 3 results for one WU is because the _1 result was not returned by its due time, so when that due time had past, the _2 was send out a couple minutes later, which was then returned successfully 1h15min after _1 in fact was successfully returned (though 3.5h past due time).Question now would be how long does it take for this to validate and if one of the results gets scrapped or all 3 get validated and points/runtime awarded... Ralf  |
||
|
|
thunder7
Senior Cruncher Netherlands Joined: Mar 6, 2013 Post Count: 232 Status: Offline Project Badges: 
|
The strange thing is that if you don´t read this, and just look at the 'my contribution' page, you'd think your computer(s) are sick and underperforming in the worst way. There should be a possibility to see this (something called a 'status page' hint hint) without having to rely on people debugging the system.
|
||
|
|
bfmorse
Senior Cruncher US Joined: Jul 26, 2009 Post Count: 297 Status: Offline Project Badges: 
|
My list of PENDING VALIDATION WU's goes back to Jan 30th. only because original wingman sat on the WU for six days before returning with ERROR (<core_client_version>7.16.11</core_client_version>)
Current wingman has been IN PROGRESS since Feb 5th 16:00:10 UTC Currently have 3457 with various stages of both wingman and me with PENDING VALIDATION and a few with wingman still IN PROGRESS. I just spot checked the WU's showing VALID, I have 1886 WU going back to Jan 29 05:51:39 UTC. where the _1 WU came back on Feb 6 with a <core_client_version>7.8.3</core_client_version> error. Is the system waiting for ALL wingmen to report their results? Guess not, I just found one (MCM1_0195698_7873) that the _0 WU was issued on 1-30 13:44:06 UTC and was returned on 2-06 13:39:51 UTC and shows PENDING VALIDATION with times of 4.27 / 6.96... Sorry, I tend to get frustrated by people that appear to sit on WU with NO REPLY and then return them with minimal times and unfortunately, I see that a lot. (I returned the _2 WU on 2-5 15:09:54 UTC with VALID; another Volunteer returned the _1 WU and that is also showing VALID) CYCLOPS: IF YOU FEEL THIS DOESN'T BELONG, JUST DELETE THE MESSAGE. |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2159 Status: Offline Project Badges:
|
If you were asking me how I did it, then it is a bit more complex. In any case, I have been using wcgstats and Perl. That's the question I meant to be asking, yeah. Just phrased inelegantly because I forgot what you'd called your tool :) That's funny, because wcgstats is part of the package that I called WCGtools  (… on Sourceforge).Seems like a pretty straightforward thing to automate, have something that queries the API for all Pending Validation WUs, then filters it down to ones where no host still has it in progress. You're right, it is pretty straightforward, wcgstats will indeed query the API and the last part ('then filters it down') was done using this oneliner: perl -w00ne 'next if /In Progress/;$s=0;while(s/Pending Validation//){$s++}; print "$s\n"' all-pendings |sort -n|uniq -cI see we were wingmen for each other on at least one task (I had the _0)...guess your systems are much faster than mine... With so many pending WUs the chances of having one or more in common is becoming greater and greater … and my slowest (and oldest) device is the Fairphone 1 from 2014. (I have a Fairphone 2 and a Fairphone 3, too, and my girlfriend has a Fairphone 4, but they're not BOINCing … the smartphones, that is.  )Adri |
||
|
|
Aperture_Science_Innovators
Advanced Cruncher United States Joined: Jul 6, 2009 Post Count: 139 Status: Offline Project Badges:
|
The strange thing is that if you don´t read this, and just look at the 'my contribution' page, you'd think your computer(s) are sick and underperforming in the worst way. There should be a possibility to see this (something called a 'status page' hint hint) without having to rely on people debugging the system. Indeed, my daily average runtime dropped by about 40% yesterday relative to normal, and looks like it's on track to drop to 50% of normal today. I'm sure there are plenty of us here with the skills to implement a status page, and seeing what Adri has done with wcgtools, the motivation too. @Cyclops, any change we could get the WCG team behind letting the community create a status page? |
||
|
|
|