World Community Grid - View Thread - Reducing the gap from elapsed time to CPU time

World Community Grid Forums

Category: Completed Research

Forum: The Clean Energy Project - Phase 2 Forum

Thread: Reducing the gap from elapsed time to CPU time

Quick Go »

No member browsing this thread

Thread Status: Active
Total posts in this thread: 37

[ ]

Author

This topic has been viewed 2554 times and has 36 replies

Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:

2 year badge for Human Proteome Folding - Phase 2

45 day badge for Help Fight Childhood Cancer

50 year badge for The Clean Energy Project - Phase 2

45 day badge for Computing for Clean Water

2 year badge for Drug Search for Leishmaniasis

2 year badge for GO Fight Against Malaria

100 year badge for Mapping Cancer Markers

100 year badge for Uncovering Genome Mysteries

100 year badge for Outsmart Ebola Together

50 year badge for FightAIDS@Home - Phase 2

10 year badge for Smash Childhood Cancer

50 year badge for Microbiome Immunity Project

20 year badge for Africa Rainfall Project

50 year badge for OpenPandemics - COVID-19


Re: Reducing the gap from elapsed time to CPU time

very demanding on RAM - you need 1 Giga per WU. So that is why your utilization is dropping significantly when running 8 concurrent tasks with 4 GB RAM only.

Cheers peace

That may be the case on Windows or MAC, I running on Ubuntu server and only using 8.5Gb of the available 48Gb - Now that's strange...

I'm currently running 64 wu's of CEP2 on 64 cores/one machine again

Just for reference:

HDD writes are currently peaking at around 40% but no constant writes
Not using the page/swap file which is showing as 0% used

----------------------------------------

Mamajuanauk is the Name! Crunching is the Game!

[Jun 17, 2014 6:53:16 PM]

branjo
Master Cruncher
Slovakia
Joined: Jun 29, 2012
Post Count: 1892
Status: Offline
Project Badges:

1 year badge for Human Proteome Folding - Phase 2

180 day badge for Help Fight Childhood Cancer

180 day badge for Help Cure Muscular Dystrophy - Phase 2

2 year badge for The Clean Energy Project - Phase 2

1 year badge for Computing for Clean Water

1 year badge for GO Fight Against Malaria

14 day badge for Computing for Sustainable Water

20 year badge for Mapping Cancer Markers

5 year badge for Uncovering Genome Mysteries

5 year badge for Outsmart Ebola Together

5 year badge for FightAIDS@Home - Phase 2

20 year badge for Smash Childhood Cancer

10 year badge for Microbiome Immunity Project

5 year badge for Africa Rainfall Project

5 year badge for OpenPandemics - COVID-19


Re: Reducing the gap from elapsed time to CPU time

Look at SysReq.

Yes, you are right, when I have run them on my Win rig they also have not taken whole 1GB of RAM per, but, IIRC, circa 0.5 - 0.75 in average. Since I have lot of RAM on my Mac, I have not checked its utilization for CEP2 at all.

Anyway, I had always dedicated required 1GB for each task.

Cheers good luck

----------------------------------------

Crunching@Home since January 13 2000. Shrubbing@Home since January 5 2006

----------------------------------------
[Edit 2 times, last edit by branjo at Jun 17, 2014 7:19:35 PM]

[Jun 17, 2014 7:14:48 PM]

Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:


Re: Reducing the gap from elapsed time to CPU time

Yep, aware of that thanks, however as most of my machines have 2:1 ratio on RAM:cores I don't tend to worry about it. It's mainly the 2 x 64 core machines that have less than that ratio, they both have 48:64 or .75

I have been looking at increaseing this, however, with CEP2 being one of the most demanding projects for RAM and it doesn't use much, I doubt I'll do it yet.

A little wastefull, but it saves having to monitor useeage of system resources...

----------------------------------------

Mamajuanauk is the Name! Crunching is the Game!

[Jun 17, 2014 8:06:59 PM]

Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1294
Status: Offline
Project Badges:

90 day badge for Discovering Dengue Drugs - Together

1 year badge for Nutritious Rice for the World

90 day badge for The Clean Energy Project

2 year badge for Help Fight Childhood Cancer

90 day badge for Influenza Antiviral Drug Search

2 year badge for Help Cure Muscular Dystrophy - Phase 2

2 year badge for Discovering Dengue Drugs - Together - Phase 2

2 year badge for Computing for Clean Water

2 year badge for Computing for Sustainable Water

2 year badge for Uncovering Genome Mysteries

20 year badge for Outsmart Ebola Together

20 year badge for FightAIDS@Home - Phase 2

5 year badge for Microbiome Immunity Project


Re: Reducing the gap from elapsed time to CPU time

Another possibility to increase the efficiency for CEP-tasks is the use of a RAM-disk.
Retried it again after years because of this thread and ran 4 CEP's on my 6 years old quad (AMD 9350e @2GHz, 8 GB RAM 800MHz), that I also use as my main PC.
I used a 3GB RAM-disk and lost a bit of efficiency while restarting the tasks in the beginning to get 4 tasks due to small disk.

Task                                                Elapsed / CPU-time  CPU efficiency   PC
E222920_336_K.22.C17FH9N2SSe.01189645.1.set1d06_0  05:03:49 / 04:46:16       94.2 %      RAM
E222918_363_K.21.C16FH9N2Se2.01115509.2.set1d06_0  11:01:32 / 10:34:14       95.9 %      RAM
E222919_258_K.21.C17FH11SSeSi.01286770.3.set1d06_0 11:45:34 / 11:18:31       96.2 %      RAM
E222920_180_K.22.C17FH9N2SSe.01268246.3.set1d06_0  11:50:30 / 11:25:45       96.5 %      RAM

The freed cores after a CEP-task finished were filled with a SIMAP-task to keep the 100% CPU-load.

----------------------------------------

----------------------------------------
[Edit 1 times, last edit by Crystal Pellet at Jun 17, 2014 9:36:40 PM]

[Jun 17, 2014 9:33:39 PM]

Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:


Re: Reducing the gap from elapsed time to CPU time

Thanks for the info CP, however, I'm not going to tune these machines as I'm moving the CEP2 crunching to Windows machines to take advantage of the better efficiency as mentioned above...

I did toy with RAM disks sometime ago, but I don't know enough about Ubuntu to make it work so left it for now... The expected increase was not enough to persevere...

I've moved a 24 core machine which has dual bood over to win7, and it's already returning work for CEP2. I also have a couple of other Win7 machines which I'll be moving over.

----------------------------------------

Mamajuanauk is the Name! Crunching is the Game!

[Jun 18, 2014 7:48:17 AM]

hnapel
Advanced Cruncher
Netherlands
Joined: Nov 17, 2004
Post Count: 82
Status: Offline
Project Badges:

90 day badge for Human Proteome Folding - Phase 2

5 year badge for The Clean Energy Project - Phase 2

90 day badge for Drug Search for Leishmaniasis

90 day badge for GO Fight Against Malaria

45 day badge for Computing for Sustainable Water

1 year badge for FightAIDS@Home - Phase 2

2 year badge for Microbiome Immunity Project

180 day badge for Africa Rainfall Project

20 year badge for OpenPandemics - COVID-19


Re: Reducing the gap from elapsed time to CPU time

It's waiting for anything else than the CPU I guess, since CEP2 is disk intensive look for disk issues, use the fastest disk available for BOINC projects or procure an SSD (I did).

[Jun 19, 2014 9:32:57 PM]

Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:


Re: Reducing the gap from elapsed time to CPU time

Thanks hnapel, however, please see post above...

The line

HDD writes are currently peaking at around 40% but no constant writes

is from that post...

It appears there is a lot of wasted time with the changing over of the various tasks within the WU's, something that has been an issue and looks like will be for some time to come...

----------------------------------------

Mamajuanauk is the Name! Crunching is the Game!

[Jun 19, 2014 9:43:15 PM]

[ ]