Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 71
Posts: 71   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 8640 times and has 70 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: At this rate

So people that have all projects selected do more of the work you chose not to do; cancelling the effect of your choice on the project.
Just because you choose not to crunch a project does not mean that others should not. That is what load balancing does.
You choose what you do, but WCG works around it and does what the WCG wants disrespecting peoples choice.
That is what a policitian calls the big lie. If you choose not to crunch a project, then WCG does not force you to do so. Never has. Leave it alone.
This enforced system of False Choice is sneaky, deceptive, immoral and arrogant
Another big lie. The process is fully documented and technically sound. Just because you opt out of a project does not mean that WCG must also do so. Stop trying to tell me what I can crunch. The pledge is by WCG to the scientists to support their work and you are provided the opportunity to participate in any of the active projects.
If the WCG genuinely wanted choice, the people that have all projects selected would crunch equal amounts of those tasks. That way if half the crunchers wanted to do work on a project they think is particularly important it would finish faster. For the WCG to dictate that all projects are equal is despotic.
Again you are trying to DICTATE to WCG how it operates, demanding that WCG penalize any project that one cruncher chooses to ignore. Sounds like something a despot would say.
If the WCG ever accept a project that someone disagrees with, their only choice would be to stop crunching for WCG altogether.
Absolutely false. All any cruncher has to do is to opt out of the project.

Chill man, you are wired way too tight on this. There are 513,547 members as of yesterday and you do not get to design WCG nor implement policies. All you can do is crunch or move on. There is nothing inherently sinister or deceptive about the WCG design or processes. Saying there is does not make it so.
[Jun 9, 2010 1:39:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

sk ... I have not seen your above rant over at GPUGrid which I know you participate in and they give you absolutley no choice at all between their multiple subprojects. Do you have something in particular against WCG? You sound so bitter about WCG I wonder why do you even bother crunching here?
I crunch elsewhere because they also do different research that I like. GPUGrid'€™s research is unique. Their lack of choice is not an issue as they use one type of research method. Here it is more complicated; there are lots of projects that come and go and there is a badge system that depends on choice. I think load balancing belittles the badge system and our choice of project. Those that chose to crunch for one or two projects have their choice offset by other crunchers who attach to all projects. Their choice is being cancelled out. It is that type of €˜load balancing that I have the particular problem with. It is not that I dislike any present or past projects; it is that others might and I might in the future. As a result I and others might have a problem crunching here at all because of one project we don'€™t like. For example, if a prime number project was introduced, find deep sea oil, looking for ET, or a nuclear research project. Then through load balancing we would in effect be supporting these projects. I want my choices to mean something. If I chose to just crunch HCC then load balancing would make that choice meaningless. I expect that most people crunching for one or two projects don'€™t know that their choice is not speeding up those projects over the others.
Don't take my specific opinion about one perceived flaw in WCG to reflect my overall opinion, I am merely expressing my distaste and concern of the use of load balancing. I have said plenty of good things and have supported and promoted the WCG for several years. Just because it is a well run, reliable project does not mean it has no flaws or cannot improve.
----------------------------------------
[Edit 1 times, last edit by skgiven at Jun 9, 2010 5:54:17 PM]
[Jun 9, 2010 5:51:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

@skgiven

Please read this post I made late last year. It is still appropriate for the concerns you have expressed.

https://secure.worldcommunitygrid.org/forums/...ead,28107_offset,0#260714
[Jun 9, 2010 5:59:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: At this rate

The load balancer is not designed to provide the individual cruncher with the ability to penalize a project. It is designed to allow you to choose what project(s) to support. You do not get to tell me that I cannot support a specific project because you choose not to support it.

Also, there is no flaw, perceived or otherwise. Your perception is that it does not do what you WANT it to do and the response by the Techs is that it does exactly what it is designed to do. WCG got 672,852 WU done yesterday, every active project benefited and the project choices of every one of the 97,130 crunchers were respected.
[Jun 9, 2010 8:35:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

Astrolab, how would choosing to crunch penalize a project? We cant delete results. I did not tell you what to crunch.
A few too many personal insults coming from you!
[Jun 9, 2010 8:55:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
nasher
Veteran Cruncher
USA
Joined: Dec 2, 2005
Post Count: 1423
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

i think what astrolab was trying to say is if you choose not to work a project and other people choose to run ALL projects then why shouldn't the load balancer work each project equally. personally i may not choose to crunch something but WCG decided to run all of them its just YOU personally are not workin on those results.

or am i somehow missing your point entirly
----------------------------------------

[Jun 9, 2010 9:07:28 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: At this rate

Maybe I misunderstood you:
I think load balancing belittles the badge system and our choice of project. Those that chose to crunch for one or two projects have their choice offset by other crunchers who attach to all projects.
Your opinion that other projects should be held back because you may choose not to support a project. Maybe i misunderstood you, but you appear to be saying that since you choose a project that the same project should have fewer WU crunched by others.
I think load balancing belittles the badge system and our choice of project.
That appears to be an opinion that the badges (and therefore the projects) that I choose to work on should be affected by your choice.
If I chose to just crunch HCC then load balancing would make that choice meaningless[
Clearly you are stating that by choosing HCC you intend on having all other projects penalized.
I expect that most people crunching for one or two projects don'€™t know that their choice is not speeding up those projects over the others
Again another statement that says by choosing specific projects, you want those projects favoured and other projects penalized
I did not tell you what to crunch
But that IS exactly what you are saying: If you skip a project, they you want the system to not send me the WU you skipped

Just reading what you write. Of course it's personal, opinions ARE personal. It is an oxymoron to be impersonal when disagreeing with another's opinions.
[Jun 9, 2010 10:46:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

Maybe I misunderstood you
I hope so.
Your opinion that other projects should be held back because you may choose not to support a project. Maybe i misunderstood you, but you appear to be saying that since you choose a project that the same project should have fewer WU crunched by others.
I am saying the opposite. I am saying that if I decide to crunch for a project my choice to crunch should be reflected in the amount of work done by that project, to the extent of my contribution (perhaps 0.001% more). If I decide to only crunch HCC and you decide to only crunch FA@H then why should others have to crunch more tasks from other projects, unknowingly (all tasks)?
My decision to crunch for a project does not and should not detract from other projects – I am not un-crunching tasks!
If I crunch HCC it should not mean that you cannot, nor should it mean that your choice is impaired. What I am saying is that if everyone decided to crunch HCC on half of their machines then the masses should have their way; HCC should be crunched by 50% of all machines. I want more choice, not less.
The project practacalities may not allow for this, but it does not mean it should not be a goal.
[Jun 9, 2010 11:42:34 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7640
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

Good luck trying to get 50% of the people to agree on anything. Reductio ad absurdum: If 100% of the crunchers only decided to crunch one project and only one project to the exclusion of all others would the system accomodate this ? Frankly, I do not know, but I doubt it. It is possible there would be enough work units for everybody for some time, but would the researchers, the recipients of the results be prepared to get that much information that fast? Once again, I do not know what their capacity is for receiving information. I think that under some "emergency" situation perhaps the power of the entire grid may be concentrated on one project, but I do not know if that is technically possible or desirable.

These are all large projects. Most of them have a lot work for an extended period of time. I am even guessing that some projects may have been turned down by WCG because they were too large and not feasible with amount of processing power available today.

I think most of us choose to crunch because we believe we are doing something worthwhile for humanity. The badges are a nice added bonus, but just try buying a cup of coffee with one. I like Knreed's explanation on the load balancing. It makes sense. In the end crunch what YOU like and leave the rest for others. To each his own.

Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Jun 10, 2010 1:32:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: At this rate

I am saying the opposite. I am saying that if I decide to crunch for a project my choice to crunch should be reflected in the amount of work done by that project, to the extent of my contribution (perhaps 0.001% more). If I decide to only crunch HCC and you decide to only crunch FA@H then why should others have to crunch more tasks from other projects, unknowingly (all tasks)?

Just to make a simple example, let's say the Feeder has 1000 slots (I know in reality it's more, but as I said, I'll making it simple).
Let's also say the 4 currently active projects gets an equal share, all tasks takes the exact same amount of time regardless of project, and I'm not leaving any space for beta & DDDT2.
1000 slots / 4 projects means 250 slots/project.
Lastly, let's initially say the Feeder is only re-filled then empty...

What would this mean?
Well, starts with 250 HCC, 250 HCMD2, 250 HPF2 and 250 FAH.
Let's say 10 users in a row only has FAH selected, and each gets 20 wu's, this leaves 50 FAH.
Afterwards, 10 "all project"-users asks for work, and each gets 20 wu's. This should be 5 in each project, but it's not neccessarily true all of these 10 users gets exactly 5 HCC, 5 HCMD2, 5 HPF2 and 5 FAH. As an average on the other hand they should get 5 wu/project.

So, after 20 users has gotten work, it's currently left:
200 HCC, 200 HCMD2, 200 HPF2 and zero FAH.

Next 10 users selected HCC & HPF2 asks for work, each gets on average 10 HCC and 10 HPF2.
Left: 100 HCC, 200 HCMD2, 100 HPF2 and zero FAH.

Next 9 users selected HCMD2 asks for work, and each gets 20 wu's.
Left: 100 HCC, 20 HCMD2, 100 HPF2 and zero FAH.

Next, 11 all-project users asks for work, and the remaining wu's is given-out.
The 1st. of these gets maybe 7 HCC, 7 HCMD2 and 6 HPF2.
The 2nd. of these gets maybe 7 HCC, 6 HCMD2 and 7 HPF2.
The 3rd. of these gets maybe 6 HCC, 7 HCMD2 and 7 HPF2.
Number 4 to number 11 will each get 10 HCC and 10 HPF2 as average, since there aren't any HCC-work left, and there aren't any FAH-work left.

Since it's now no work left in the Feeder-array, the Feeder is re-filled with 250 wu's for each project.

As this example shows, for technical reasons, some users choosing to run only one of the sub-projects doesn't influence the total amount of work crunched in the different projects, each sub-project gets an equal share of the total.
And yes, for technical reasons, the "all users" gets more work in some of the projects than in other projects. If looks on single users, it can be very lop-sided, maybe only getting work in a single project...


But anyway, let's make a more advanced example, and a little closer to the real behaviour.
Now, each wu in each project takes a different amount of cpu-time to run, I don't remember the exact numbers for the various sub-projects currently running, so I'll not going to pair it with actual projects... Let's also say the Feeder is at 1085 slots.
Project A: 10 hour cpu-time.
Project B: 1 hour cpu-time
Project C: 4 hour cpu-time
Project D: 5 hour cpu-time.

Due to these differences in cpu-time, to get a roughly equal share, you'll need:
A: 70 slots. (700 hours of work).
B: 700 slots. (700 hours of work).
C: 175 slots. (700 hours of work).
D: 140 slots. (700 hours of work).
Total: 1085 slots, 2800 hours of work.

Now, individual work-requests will be very different from user to user, but to not make it hopelessly complicated, let's say each user wants 40 hours of work.
Also, let's put 2 limits up, the Scheduling-server can max send-out 20 wu's per second, and the Feeder can max add 20 new wu's, but it will only re-fill every 2 seconds.
If each scheduling-request takes one second, what would these limitations mean in practice?

Well, for A, C & D, these limitations has no effects, since neither limits will be exceeded.
For B on the other hand, both limits will influence things, since users wants more work than scheduling-server can give, and work is issued faster than Feeder can be re-filled.

So, let's say 2 "all project"-user asks for work, what would they get?
Let's say both gets 1 A, 12 B, 2 C and 2 D, equaling 40 hours of work.
As for how Feeder is re-filled... not sure, there's 3 possibilities, either it starts on a random project and re-fills as much as possible before takes the next project, or it randomly fills-up with work regardless of project, or it randomly starts on a project and fills 1 wu per project as needed...
Hmm, I'll do it simple, and use the last option, meaning re-fills with 2 A, 10B, 4 C and 4 D.
Left: 70 A, 696 B, 175 C, 140 D.

The two next users asks for only C, each gets 10 wu's for a total of 20, and 4 B and 16 C gets re-filled, leaving 70 A, 700 B, 171 C, 140 D.

Except if the 2 next users also asks for only C, the C will be re-filled back to normal, while B can possibly decrease again, if issues many of these...

But, let's look on 2 extremes, let's say B is a very popular project, partially due to the very short wu's, and maybe partially due to it being a new project, so many users wants to get their badge... What can this mean?

Let's say 50 users in a row wants only B...
Since each user can max get 20 wu's per second, and each wants 40 wu's, it will take 100 seconds to fill all these users requests...
For each second, 20 wu's is issued, and for each 2 seconds 20 new wu's is added.
With 700 wu's to start with, this works for 70 of the 100 seconds, but at this point the cache is empty, and only every 2nd. request can be re-filled... Meaning, either it can take much longer before every user gets their fill, adding the probability other users will also ask for work, or most will get work from other projects instead...


Another "extreme" is, all users runs with equal share...
This basically means, for each second, 2 less B is available in the cache, so after 350 seconds, roughly 10 minutes, some of the users won't get their equal share of B-work, and will instead get A, C and D.

A mix of these two would be every 2nd. user is only running B, while the others runs all project. In this scenario, 32 B will be issued every 2 seconds, together with 1 A, 2 B and 2 C, meaning 15 B will be re-filled, giving a deficiency of 17 B every 2 seconds, so after 82 seconds, B will be empty, and users will get work from other projects instead.


Does WCG have this limitation currently? I don't know, but it's definitely a possibility they do have, if majority of users chooses to run only the fastest-running project...

If I crunch HCC it should not mean that you cannot, nor should it mean that your choice is impaired. What I am saying is that if everyone decided to crunch HCC on half of their machines then the masses should have their way; HCC should be crunched by 50% of all machines. I want more choice, not less.
The project practacalities may not allow for this, but it does not mean it should not be a goal.

I've given 2 examples on why you won't get your idealized version in practice. Both shows that if many users chooses to run only a single project, the "all project"-users will get more work in other projects instead.

There's also other reasons besides the Scheduling-server & Feeder that means you won't get your idealized version in practice. A few of these are:
1: Work-generation is so slow, that can't keep-up with higher demand.
2: Limited disk-storage, either wu-size and/or result-size is so large that exceeds disk-storage.
3: In case of short wu's, too many will give higher and higher database-load, and this will give performance-problems that project wants to minimize.

Both #1 and #2 can be project-limitations, and don't need to be anything WCG can do anything to solve.


BTW, without any numbers of how many users has really choosen a particular project or runs with "all projects" can only guess, but atleast my guess is the single-project-users will more or less cancel-out each others "extra" production, so there won't be a big advantage to any of them...

With the current production according to Sekerob's chart being 22.5% to 27.6% of daily production, there's atleast no indication 50% of users has choosen to run a single project...
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
[Jun 10, 2010 9:45:24 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 71   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread