| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Beta Testing
Forum: Beta Test Support Forum
Thread: Better spread of beta WUs among beta testers |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 18
|
|
| Author |
|
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges: 
|
Dear duanebong,
Thanks for your contribution! It's not possible to achieve a system that is viewed as "fair" by everyone. Fairness depends on each individual, and some could argue that "one Beta WU/user" is not fair at all. I agree with you. However, it's always possible to discuss the problem and try to make the system fairer, instead of leaving things as they are. It can be discussed (maybe a poll can be set up, with the feasible options) in order to find the fairest solution according to most users' opinion. Some users have server farms with 100s of high clockspeed cores. They run 24hrs/day, 7 days/week and can return beta WUs within hours. Others have a single low power core and run for a few hours on weekdays only - taking a week to return a single WU. Science output at WGC would be impaired if both are allowed 1 beta WU per user. This could be solved, for example, by giving a quick deadline (of some days) for beta WUs. If someone has "a single low power core and run for a few hours on weekdays only", one will unfortunately miss the deadline and the WU will be sent to another user. But this is just a quick example of solution; other possibilities may be thought of. Machines that run 24/7 complete more WUs (naturally!). So they contact the server more often to report completed work. In doing so, they have a higher chance of finding beta WUs. One can improve chances of getting beta WUs by running 24/7, at the same time WGC also gets more science done - win/win! As I've told before, I'm running 24/7 with a 0,10 day cache and haven't received one single WU for this last beta test. This is not a personal problem; my concern is about the current "beta distribution system" being unfair with lots of users, besides of diminishing the diversity of systems/machines in which beta WUs are tested. Thanks again for your contribution to this discussion!  |
||
|
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges:
|
Dear SekeRob*,
The concept of 'user' is peripheral... the function is only there to add up statistics of results produced by multiple machines registered under the same 'user'. There is zero interaction between clients, none whatsoever, in fact they're agnostic of each other when it comes down to communicating. WCG is an amazing result of intelligence used for the good. I'm sure intelligence can be used to improve the system. There is always a good solution; and WCG techs have been great (and very smart) along the years. Just to get 'fair-er' would be a major rewrite, just for exactly what? A badge? I'm not trying to raise awareness just for a badge. Badge hunting is fun, but my arguments so far have been: 1. a fairer system for users who are trying to help in beta tests; and 2. more diversity of machines and system configurations running beta WUs (more diversity = more information = better tests). Want more beta work, beat the system, start by subscribing to get an instant mail the moment something is posted in the BETA (Read Only) forum I've done that, I receive these e-mails, and still haven't received one single WU for this last beta "wave". As I told before, I'm crunching 24/7 with a 0,10 day cache. Besides, not everyone has the time (and technical knowledge) to make all the micromanagement (scripts etc.) needed to try to get some beta WUs. I have some time and knowledge and didn't get any WU. Then why not make the system fairer for everyone? beta goes out with absolute feeder priority, and right that is. "Right" in your opinion, and I respect that. Thanks for your contribution! |
||
|
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges:
|
Honestly, we just need to increas the volume of WU produced. Taking a look at 2 Betas I was able to snag this wave, they had Min. Replication of 1, and both were run on Windows 10. If I was doing it, I would have AT LEAST 1 Windows 10 use, 1 Win 8, 1 Win 7, 1 Linux and 1 Mac per WU (assumind Mac is supported). Afterall, this is BETA. It doesn't really matter if tests are redundant, the purpose is to make SURE that they are working. Dear imakuni, You have a good point too! Testing all WUs in all main OSs (or at least more than one OS) would also increase the information on diverse systems and configurations, resulting in a better test. It would also increase the volume of beta WUs, thus increasing the chance of more users participating. Thanks for your contribution! [Edit 1 times, last edit by F. Bravo at Jul 31, 2017 6:47:46 PM] |
||
|
|
imakuni
Advanced Cruncher Joined: Jun 11, 2009 Post Count: 103 Status: Offline Project Badges:
|
You have a good point too! Testing all WUs in all main OSs (or at least more than one OS) would also increase the information on diverse systems and configurations, resulting in a better test. I also forgot to mention, but make it double for 32 and 64 bit OS (if supported).  Want to have an image of yourself like this on? Check this thread: https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,29840 |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
So I wanted to chime in real quick with some information.
----------------------------------------We run more than enough work in beta to be able to meet our needs for testing different operating systems and variants of processors and platforms. By the time we complete the beta testing processes, we have pretty high confidence that application is in pretty good shape. This is not to say that further issues aren't found after we go live, but that is always a possibility. We try to avoid running excessive amounts of beta work since this work is not usually used by the researchers other then for the purpose of verifying that the application is running correctly (i.e. often they have run a set of work already on their in house grids and machines and they are verifying the results of beta against those in order to confirm things are working correctly). We set the deadlines for these tasks shorter because we need the data back faster in order to get things verified with the researchers. So overall, it is meeting our needs for beta testing our upcoming applications. Right now, there are beta jobs assigned to 497 users on 633 different machines with 1867 tasks in progress. There are a few of users were able to get more than 10. The vast majority have fewer than 10 tasks assigned. Only 9 users got more than 20 tasks during this beta. We certainly understand the desire to get to run the beta tasks (they are fun to see running and the beta badge is one of the hardest to get). However, we are not experiencing a problem in terms of being able to test the application on different platforms during beta testing as we are currently running it. [Edit 4 times, last edit by knreed at Jul 31, 2017 7:40:55 PM] |
||
|
|
TonyEllis
Senior Cruncher Australia Joined: Jul 9, 2008 Post Count: 261 Status: Recently Active Project Badges: 
|
I have received 36 Beta WUs so far during this present testing period spread over 8 machines, total of 33 cores, with varying CPU/OS/boinc versions running 24/7... somewhat surprised by knreed's append - didn't realize this was so 'lucky' - thought that was a 'normal' amount...
----------------------------------------But is it 'luck'? - like every thing that is in short supply one has to be in the right place at the right time properly prepared. I an running relatively short WUs (SCC/Zika) with a buffer of about 4.5 days, process approx 300 - 400 WUs per day depending on the mix (SCC is shorter) - so guess requesting more work quite often... With the buffer size here, when Beta units are downloaded the current WUs are immediately suspended, and the Beta WUs run. Nornal WUs resume as soon as the Beta WUs finish. Is it 'unfair' there are several machines involved? Is it 'unfair' that shorter WUs are part of the current targets here and hence greater number of 'more work' requests? Is it 'unfair' to be rewarded for a greater effort/contribution? You be the judge... In many ways this discussion is somewhat similar to some discussion that has taken place for 'Help Stop TB' where WUs are also difficult, but not impossible, to get in any larger number...
Run Time Stats https://grassmere-productions.no-ip.biz/
|
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7666 Status: Offline Project Badges: 
|
I usually have between 60 and 76 threads running at any one time. I received 22 beta units of which 13 errored because of the download problem. I consider myself fairly lucky to have gotten that many. I run with about a 1 day cache. Getting Beta units is just a matter of them being available when your machine(s) happen to be requesting work.I have more important things to do than babysit my machines to try to land Beta units or any other scarce units. If they come, fine and if they don't that is fine too.
----------------------------------------Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
keithhenry
Ace Cruncher Senile old farts of the world ....uh.....uh..... nevermind Joined: Nov 18, 2004 Post Count: 18665 Status: Offline Project Badges:
|
I agree Tony. Yes, it would be nice to be able to get more betas. A new shinny badge makes for a nice "at-a-boy!". That's not what betas are for though. It's a chance for WCG to decide that a new science app. (or new science app. version) is ready for production use and have reasonable confidence in that decision. It's not for contributing to the research. The sooner the new/updated science app can be put into production, the sooner we can start doing our part and contributing to the real research. The sooner we can do that (and the more of us that do), the sooner the researchers can start analyzing those results and hopefully make the discoveries that they hope for.
----------------------------------------And yes, having more machines/cores does improve your chances of getting betas. If you buy ten lottery tickets and your neighbor buys one, each ticket has the same chances of being a winner but your chances are better than your neighbor's simply because you have more tickets. Each time someone's machine sends a scheduler request to the servers, you have a chance of getting a beta WU - IF there happen to be any loaded into the feeder at that point in time. Compared to the other projects, there are extremely few beta WUs so the chances of one sitting there waiting to be sent out at any given point in time is extremely low (yep, it's that infamous supply and demand thing again). The more often you ask, the greater your chances of asking when one just happens to be sitting there waiting to be sent. Having more machines/cores is just like having more lottery tickets. Crunching short running WUs means you get more chances in a given period of time to ask for more work. If someone wants to get more betas, they want to be the driver of the charter bus full of kids constantly asking "are we there yet", not the parent in the car with the kid in the back seat who sleeps thru the journey only waking up once to ask along the way. |
||
|
|
|