Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 76
Posts: 76   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 129771 times and has 75 replies Next Thread
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Nice exposition, but... got 4 running now to see if your theories hold, plus 3 HFCC jobs in suspended LAIM animation and 2.56 GB being used per BOINC disk tab info and total Tasks being 9 on the machine. This is todays data. Strikes me that anyone going with moderate buffering is well able to run the whole of BOINC in that 3.5GB limit, on a quad. Overall, this is an "erörterung" to find if this works and the feasibility. Practice and hands on experimenting is so powerful to test theories.

Ingleside, feel free to continue your sleep
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Nov 3, 2010 9:11:25 PM]
[Nov 3, 2010 9:06:48 PM]   Link   Report threatening or abusive post: please login first  Go to top 
anhhai
Veteran Cruncher
Joined: Mar 22, 2005
Post Count: 839
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: RAMDisk

Are you volunteering to test the RAMdisk configuration? Yes, the tasks seem to run slightly faster too, in my setup.... 1-2 percent. Getting more credit with ZR jobs on the Linux platform is a rarity, but it's constant, so somethings must be working right.



Sekerob, I can test it out if you want. But I am not sure if 1 or 2% increase in performance is worth it.
----------------------------------------

[Nov 3, 2010 9:45:38 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Are you volunteering to test the RAMdisk configuration? Yes, the tasks seem to run slightly faster too, in my setup.... 1-2 percent. Getting more credit with ZR jobs on the Linux platform is a rarity, but it's constant, so somethings must be working right.



Sekerob, I can test it out if you want. But I am not sure if 1 or 2% increase in performance is worth it.

The drive behind this is CEP2 efficiency improvement,as outlined in the OP, particular when running multiple concurrent, the default setting to only assign 1 ''in progress'' to a host. The rest is a bonus in efficiency. Getting 11-12% more credit per task is secondary, but am happy to monopoly-bank that... and to underline, the Linux experience, where C4CW anyhow on the 64 bit platform runs 60% faster than on W7-64 bit.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Nov 3, 2010 9:53:20 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Hello anhhai,

The real interest in RamDisk is for the future. CEP2 for Windows has been delayed because Windows slows down when it has a lot of disk accesses. Linux is much more efficient, so there is little need for RamDisk on Linux.

Lawrence
[Nov 3, 2010 11:12:56 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: RAMDisk

Nice exposition, but... got 4 running now to see if your theories hold, plus 3 HFCC jobs in suspended LAIM animation and 2.56 GB being used per BOINC disk tab info and total Tasks being 9 on the machine. This is todays data. Strikes me that anyone going with moderate buffering is well able to run the whole of BOINC in that 3.5GB limit, on a quad. Overall, this is an "erörterung" to find if this works and the feasibility. Practice and hands on experimenting is so powerful to test theories.

Ingleside, feel free to continue your sleep

2.56 GB disk-usage is >> 1.5 GB.

Please remember, while you possibly can run 4 CEP2 at once in only 3.5 GB space, you'll need to have 2 GB free to download any CEP2-work.

Not remembering exactly how much HFCC uses, would still guess your 2 CEP2-jobs uses close to 2 GB, so a good starting-point would be 0.5 GB per task. On top of this you'll have atleast 0.2 GB for the CEP2-application-files. For 4 tasks this is 2.2 GB total.

Meaning, then you're running your 4 CEP2-tasks, you can't download any more work. Then one of the CEP2-tasks finish, disk-usage will drop down to 1.7 GB, but this is still too much to download more CEP2-work.

Only then computer is down to 2 running CEP2-tasks is there enough free space to download more CEP2-work.

If assumes every CEP2 on average takes 8 hours, this will mean 1 core will be idle on average for 2 hours before 2nd. task finish, and can download more work. So, if you wants "moderate" buffering, let's say 1 day, you'll sit idle for atleast 2 hours every time you re-fills with 1 days worth of work, this equals 96 hours cpu-time on a quad-core.

Meaning, atleast 2% of the time is idle-time, and therefore wasted time with a 1-day cache-setting.


Hmm, would using a larger cache-size work and therefore decrease the wasted time?
I doubt it, since AFAIK after the initial application-download and so on, a "normal" CEP2-download is "small". Meaning, after your 1st. scheduler-request, in the 11-second-delay before you've allowed to ask for more work, you can have finished downloading 1 more CEP2-task, and this has started to run, something that effectively kills work-request again due to not enough free space...

Even with a 1-day-cache, if you don't get a full days worth of work in a single scheduler-request, equaling 12 tasks, you'll likely to waste more than 2% of the time sitting idle...


Now, in my calculations I've assumed constant CEP2-disk-usage. I'm not sure if this is correct, or maybe the disk-usage is increasing during the run, or maybe even increasing for each sub-job for so decreasing again on start of next sub-job. If the average is 0.5 GB but disk-usage starts at 250 MB and increases to 750 MB or something, it's possible to become even worse off then it comes to wasted time...

And, if CEP2 routinely exceeds 750 MB, your quad-core is in danger of crashing into the 3.5 GB upper limit, something that you definitely don't want to do...
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
[Nov 3, 2010 11:27:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Plz call back when you have actual numbers instead of "assumed". The CEP2 slots peak on the Linux version at 608MB but most of the time sit at 562MB and since in real life they wont be writing their checkpoints simultaneous, do not expect for 4 concurrent needing more than 2.5GB if ran exclusively.

As for 2% idle, that's 1.92 hours on a quad in offset to a potential gain of 6+ hours lost to HD Disk IO and then loosing the disk IO improves the user experience even if all of it is lost in the server seeing there is insufficient space for all of that time... but then who says we're settling on the version from the OP link? We're seeking workable solutions and we love hands on experimenting and spending 9.99$ to get a bigger RAMDISK wont break the bank... not here at least.

Now it's shuteye time... we're optimistic that tomorrow morning my quad will still be crunching CEP2 jobs.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Nov 4, 2010 1:02:49 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Defeat... not quite ;O)

Thu 04 Nov 2010 07:31:58 AM CET World Community Grid Message from server: Beta - The Clean Energy Project - Phase 2 needs 521.97MB more disk space. You currently have 1526.03 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 07:31:58 AM CET World Community Grid Message from server: The Clean Energy Project - Phase 2 needs 521.97MB more disk space. You currently have 1526.03 MB available and it needs 2048.00 MB.

3 are running now, BOINC Disk says it's using 1.77GB of the explicitly limited 3.5GB. Says 1.73 is Free for BOINC to use in same Disk use tab view, yet insists to need 522MB more... The Disk use tab has an odd entry too that 613MB is used by "other"... what "other"?. And to "top" it, this says 2GB is used, so from that view, 1.5 free as reported during the work fetch is not "wrong".

When it reported a completed Clean Water task, the situation improved by 26MB. There's a dozen more in the cache to clear, but don't think they use space... just a task list entry.

Thu 04 Nov 2010 07:46:24 AM CET World Community Grid Message from server: No work is available for Discovering Dengue Drugs - Together - Phase 2 (Type A)
Thu 04 Nov 2010 07:46:24 AM CET World Community Grid Message from server: Beta - The Clean Energy Project - Phase 2 needs 496.17MB more disk space. You currently have 1551.83 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 07:46:24 AM CET World Community Grid Message from server: The Clean Energy Project - Phase 2 needs 496.17MB more disk space. You currently have 1551.83 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 07:46:24 AM CET World Community Grid Message from server: No work available for the applications you have selected. Please check your settings on the website.

Now why is 3.5 - 1.77 netting to 1,552 available?

For now, within the 3.5GB RAMDisk limit of the free version a mix profile of CEP2+C4CW will get me there too, albeit a bit slower but with less HD IO waste... but now I can spin down the Harddisk in the power settings without a potential slowdown or hiccup.

Will let the cache run empty to truly only have running CEP2's there and set buffer+connect to 0.00 and see if the finishing tasks will liberate enough space to fetch new CEP2 work and when.

We ''lost'' a bout and still win... knowledge :D

(Now, why is WCG speccing 2GB free on disk I cant remember... looking at the slot use peak for the Windows CEP2 Beta, of 697GB, 1GB Free min. would get all 4 cores to run them unhindered)/size]
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Nov 4, 2010 7:14:26 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: RAMDisk

Creepy crawly, let one of the suspended HFCC jobs finish :P

Thu 04 Nov 2010 08:13:20 AM CET World Community Grid Message from server: No work sent
Thu 04 Nov 2010 08:13:20 AM CET World Community Grid Message from server: No work is available for Discovering Dengue Drugs - Together - Phase 2 (Type A)
Thu 04 Nov 2010 08:13:20 AM CET World Community Grid Message from server: Beta - The Clean Energy Project - Phase 2 needs 412.59MB more disk space. You currently have 1635.41 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 08:13:20 AM CET World Community Grid Message from server: The Clean Energy Project - Phase 2 needs 412.59MB more disk space. You currently have 1635.41 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 08:13:20 AM CET World Community Grid Message from server: No work available for the applications you have selected. Please check your settings on the website.

1 more HFCC to go and then only buffered C4CW is left... and when I removed the old Beta apps from the WCG project folder:

Thu 04 Nov 2010 08:54:26 AM CET World Community Grid Message from server: No work sent
Thu 04 Nov 2010 08:54:26 AM CET World Community Grid Message from server: No work is available for Discovering Dengue Drugs - Together - Phase 2 (Type A)
Thu 04 Nov 2010 08:54:26 AM CET World Community Grid Message from server: Beta - The Clean Energy Project - Phase 2 needs 188.33MB more disk space. You currently have 1859.67 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 08:54:26 AM CET World Community Grid Message from server: The Clean Energy Project - Phase 2 needs 188.33MB more disk space. You currently have 1859.67 MB available and it needs 2048.00 MB.
Thu 04 Nov 2010 08:54:26 AM CET World Community Grid Message from server: No work available for the applications you have selected. Please check your settings on the website.

BOINC Disk tab says 2.05GB is free at disposal of BOINC.

... so think that if one starts out with a project reset to remove all crud and only set CEP2... stand by for an update muhahaha

The Madcap laughs
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Nov 4, 2010 7:56:56 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Jack007
Master Cruncher
CANADA
Joined: Feb 25, 2005
Post Count: 1604
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: RAMDisk

I'm IN.
I used to install programs in a RAMDISK many moons ago.
I had one little 247K game (with the updaged SVGA graphics)
that with max ships in my starfleet would take my 8086 laptop
30 seconds or more. I used that program to test computers
up to 386 I think, when there was no more discernable lag.
on a RAMDISK it was almost instant.
So for me to virtually eliminate Disk writes (so it might last a
lot longer hopefully) I'll pay up to $50 for the thrill of the ole
RAMDISK come home again. I have 12 GIGS available on this I7
(I'll be back after work to try and figure out what was said above,
I'm sure if I go step by step I could make sense out of it wink ).
----------------------------------------

[Nov 4, 2010 12:36:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
anhhai
Veteran Cruncher
Joined: Mar 22, 2005
Post Count: 839
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: RAMDisk

I just set up a ramdisk on my linux machine to see if it is doable. I gave it 1G in space (it can use more if needed since ram disk can grow by it self). However I am getting the following error when I ask for more work.

Message from server: The preference you set for maximum percent of disk space used by BOINC has been exceeded by
429.82 MB.

Not sure why this is happening since the directory is only about 400 MBs. My setting for this profile is 75% of disk space and leave .1 GB free.

I have told it to stop asking for more for now. I will see how it does in a few hrs. Side note, all 3 of my WUs that have completed since I started using the ramdisk went inconclusive, it may just be a coincidence.
----------------------------------------

[Nov 4, 2010 6:42:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 76   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread