Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 87
Posts: 87   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 10153 times and has 86 replies Next Thread
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change

I was thinking this 20% redundancy was reserved for errors and no replies and not for valid 'ínvalids'.

My reading of the 1.2 value was it being without qualification as in: WCG expects to send out 1.2 results average to volunteers to net 1 result that can be transferred to the Master Science Database via the Assimilator.

I'm not sure if this was a first for any BOINC life science project to put a mini WU test into the jobs. I expect for extreme prudence to be maintained, since the computing continues to be outwith the immediate control of the scientists, which is why quorum 2, 3 even 4 was run in the early days of BOINC to get absolute verification.

This is also why HCC is not suited for zero redundancy running. Each of the 90 million images needs 100.00000% perfect confirmation for the reference database to have any meaning as a source for further studies. No wet lab test to confirm the usefulness of a candidate compound that was identified out of the millions of tests.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Aug 25, 2008 10:23:15 AM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change

Over 11,000 workunits completed yesterday for Dengue. There was a redundancy factor of 1.228 for those workunits. The redundancy factor is the average number of results returned for the workunit including valid results, invalid results, errors, etc.

This value should be compared to the value of about 2.2 that we had prior to the implementation of the 'zero redundancy'.

This change has increased the number of workunits (and therefore science) completed by ~80% while maintaining essentially the same level of confidence in the results.

Of the 1.228 factor, 0.067 were error or invalid results while 1.16 were valid results.

We are unlikely to try push the redundancy factor any lower as it would only result in a small increase in the amount of science performed but it would have much larger impact on the confidence in the results.
----------------------------------------
[Edit 1 times, last edit by knreed at Aug 25, 2008 2:08:32 PM]
[Aug 25, 2008 2:07:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change

I had the same behaviour with at first an invalid result and afterwards it became valid because of the result of the second computer:

dddt0602m0770_ 100338_ 1-- In Progress 08/24/2008 10:37:55 08/26/2008 20:13:55 0.00 0.0 / 0.0
dddt0602m0770_ 100338_ 0-- Invalid 08/23/2008 11:51:38 08/24/2008 10:31:58 8.66 116.3 / 0.0


dddt0602m0770_ 100338_ 1-- Valid 08/24/2008 10:37:55 08/25/2008 01:20:56 5.04 96.5 / 106.4
dddt0602m0770_ 100338_ 0-- Valid 08/23/2008 11:51:38 08/24/2008 10:31:58 8.66 116.3 / 106.4



We have dug through the code to figure out what was going on with these and we found that we had a slight bug that was allowing results marked invalid to get a second chance at validation when additional results were returned. This is why results marked 'invalid' were turning into results that are 'valid'.

We have several checks in place that look at the results that are returned on their own. Some of these checks are used to immediately determine that the result is invalid. Others are used to determine that we are not sure about the result and a second copy needs to be sent (and thus the result is marked inconclusive). In the two cases above, the results failed one of our checks that marked the result immediately invalid. However, as it has been clearly shown (and we have manually confirmed) that this check is sometimes incorrect in determine a result to be invalid, we are changing that check to mark the result as inconclusive in the future. This will result in another copy being sent to confirm the answer.
[Aug 25, 2008 2:57:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change

Nice one Kevin! wink
[Aug 25, 2008 2:58:27 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1320
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
rose Re: DDDT Single Redundancy Change

Kevin, thanks for the detailed feedback applause !!
[Aug 25, 2008 5:51:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1320
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change


The next days I will only run DDDT besides ABC @ Home. If this behaviour happens more frequently I will report this here.


After the earlier in this thread reported 'invalid' (inconclusive) task I got 24 DDDT tasks.
They were all Quorum 1 and Replication 1.
So 0% redundancy. Absolute perfect!!
[Aug 28, 2008 6:57:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3715
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: DDDT Single Redundancy Change

Great!

Thank you for the feedback. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Aug 28, 2008 7:23:59 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 87   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 ]
[ Jump to Last Post ]
Post new Thread