Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 13
Posts: 13   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 19956 times and has 12 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Page faults

With CEP2 released for Windows, I have now been crunching a WU for 3:01 hours.

Can anyone point me to suggestions to reduce my Page faults. In 3 hours I am at 57.7M by wcgrid_cep2_qchem_6.35_windows_intelx86 with 3GB physical memory and 6.3GB virtual using Vista 32 business on a big HDD that is defragmented every month.
[Nov 8, 2010 10:24:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Page faults

Rather obscured where you seem to have given the big unreserved thumbs up on your 31 Beta tested CEP2 results, all valid. Think you should have spoken up and with such a number of tested results shared the profiling you of course did. Page faults itself are not important. It's the kernel time and the PF delta which are the lossy parts, but most importantly on this science the large amount of IO read/write and delta.

As for remedies, plz follow past 4 months of extensive discussions on all we tried to reduce that time. WCG certainly felt compelled to limit the default to 1 per device for a reason and create the override option. Other discussed solutions are Ramdisk, SSD (expensive), swap file action (e.g. reduce swappiness as can be done under Linux) and more. The conclusion is that one set of hardware does much better than others and there being a hard optimal. You have to try different settings and parms to find your set-ups best. Will test soon on W7-64 Q6600 to learn how that runs and wait on the Linux upgrade. At 90% throttle on the duo the kernel time sits at about 1.5 minutes per hour and gap time is rather limited and matching at said 1.5 minutes per hour (User time - Total time differential)

The few I had of 6.33 (3) and 6.35 (2) makes me think the 6.33 did better... half the gap time of the 6.35 on my duo, partly run concurrent, partly run sequential.

For those with pointed ears and running Linux: kernel 2.6.32.24 was good (OK a better term), 2.6.32.25 seemed to cause crunching problems for a number of members and last Friday upgraded to 2.6.32.26 and that made the USB stick configuration run considerably faster and gave massive download speed improvement over the Wifi connection. From the fix list there were a number of important backports for the 2.6.32 10.04.1 LTS release. It seems that the Lucid update to 10.04.2 is scheduled for Jan.19... sticking to Lucid Lynx LTS.

edit: added line to the W7 part.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 2 times, last edit by Sekerob at Nov 9, 2010 8:14:53 AM]
[Nov 9, 2010 8:04:27 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Page faults

Rather obscured where you seem to have given the big unreserved thumbs up on your 31 Beta tested CEP2 results, all valid. Think you should have spoken up and with such a number of tested results shared the profiling you of course did. Page faults itself are not important. It's the kernel time and the PF delta which are the lossy parts, but most importantly on this science the large amount of IO read/write and delta
Given that we have been through (4,5,6?) Win betas, I assumed that the current Beta was still just trying to get the WU to stop crashing. Sorry that I was not more on top of reporting the details that I saw.

Searched the forums for both PF delta and Page fault delta and couldn't find any relevant posts for Windows, only Linux. I'll bump up my swap file in size to see if that helps but I don't have the knowledge as to whatever other different settings and parms to try. I suppose a couple of 8GB sticks would solve the problem though. Thanks for responding.
[Nov 9, 2010 1:30:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
anhhai
Veteran Cruncher
Joined: Mar 22, 2005
Post Count: 839
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Page faults

astrolab. even if you have 1 TB of memory in the system, I don't think it is not going to help a lot. The biggest problem is that the WU wants to read/write to disk way too much, one solution that was propose was using ramdisk in another thread (link at the bottom). Personally I love the idea of CEP2, however, the WU seems to be a big pain. I am limiting my machines to run 1 or 2 of these WU at a time.

http://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,30244
----------------------------------------

[Nov 9, 2010 3:53:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
nanoprobe
Master Cruncher
Classified
Joined: Aug 29, 2008
Post Count: 2998
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Page faults

FWIW I think the speed of your HDD plays a big roll in this. I was having issues trying to run 8 WUs at a time with an i7 with HT enabled on a conventional 7200 RPM HDD. When I switched to an SSD everything ran smoothly crunching 8 at a time. Running quads on conventional HDDs did not have any issues.
----------------------------------------
In 1969 I took an oath to defend and protect the U S Constitution against all enemies, both foreign and Domestic. There was no expiration date.


[Nov 9, 2010 4:35:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Page faults

Does anything need to be done regarding the level of BOINC...? I am running with 6.10.56 on all of my machines.
[Nov 10, 2010 1:47:21 AM]   Link   Report threatening or abusive post: please login first  Go to top 
jay_Orlando
Senior Cruncher
USA
Joined: Jan 4, 2006
Post Count: 181
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Page faults

Hi,
Its been over two years since the last post.
Is there any info of why there are still a large number of page faults on the CEP2 project?

I did read the wiki at
http://wcg.wikia.com/wiki/Page_faults

I'm running two PCs with CEP2
1) an older quad-core AMD64 with Vista.
It runs 16,000 page faults a second running 1 CEP2 project.
I doesn't matter if I suspend all other BOINC tasks except for CEP2.
The windows task manager says I'm only using 37% of the 5886MB of memory.
So why the page faults.

2) A new octo-core AMD 64 with Debina Linux - Squeeze.
sar -B says I run between 4,000 and 19,000 faults/sec
Only 1 CEP2 project and it doesn't make any difference if I suspend all other
BOINC tasks.
With the one BOINC task running, the Linux system Monitor says it is using
747MiB of the 7.7GiB available.
So why the page faults?
On this system, the root '/' is on a solid State Drive.

Could you provide a sticky link to some of the discussions of Page faults and using a RAM disk - if this would help?
(Or is it just the nature of the CEP2 application?

I'm trying not to whine - but to get a better understanding of the task and
whether or not the page faults/sec could be detrimental to a SSD.

Thanks! (Really)
Jay E.
----------------------------------------

[Mar 3, 2013 5:09:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Page faults

Page faults look impressive, but the important numbers are the deltas. Long as the reported efficiency at the end is 97% or better I'd say you've got a good balanced system, noting that it's been long known that Windows [7/8 which I'm running], does better than most any Linux by several percentage points. My W7 laptop octo does 98.2% or better, the Linux box does 95-96% [if left alone]. The more concurrent you run of CEP2, the less efficient it becomes, due the huge disk I/O, where the VM is too.

For the rest, see the various discussions [in this forum, sticky link one inclusive]] on optimizing the system as there's nothing more to add, but to say that sometime this year there will be a new release with an updated Q-Chem engine. Till then, don't overstretch... start with 1 at the time, increase it slowly [upgrade to a 7.0.42 or higher client and use app_config.xml to control the concurrent number of CEP2] and stop where the efficiency drops below 95% (which is the limit for me).
[Mar 3, 2013 8:28:28 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Randzo
Senior Cruncher
Slovakia
Joined: Jan 10, 2008
Post Count: 339
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Page faults

Hello Jay,

as far as you have system with such high amount of RAM I would recommend you to disable swapping at all.
You will get performance improvement and it will offload I/O from your HDD/s a lot.

Just execute command swapoff -a as root.
I am using it on my Linux boxes and it helps a lot.

Regards,

Randzo
[Mar 4, 2013 6:04:48 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Page faults

Saved a comment for the TGIF day, not convinced of wisdom behind switching off swapping entirely when there's moments of complete memory cram. Visit previous posts discussing swappiness, which I routinely alter permanently from default 60 down to 10 **. Running this off my Corsair 64gb USB 3.0 drive and Sysmon says to have recorded zero hits on the swapfile. Not tried it, but saw a doc indicating how to move the swapfile off to a different drive [freeing up the memory at that, and still be save, not to wear out the memory stick. Of course if purely in RAM, there's no wear.]

** Jawdrop, even mentioned swappiness in this thread back in 2010. Search hits is half a dozen: http://www.worldcommunitygrid.org/forums/wcg/search?offset=0&key=swappiness

edit: To quote an explanation of a formula, with 3950 of usable RAM, "so what it mean is that when 10 %, 395 MB of ram left then it starts using swappiness"

http://askubuntu.com/questions/103915/how-do-i-configure-swappiness
----------------------------------------
[Edit 1 times, last edit by Former Member at Mar 8, 2013 2:19:58 PM]
[Mar 8, 2013 1:47:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 13   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread