| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: CEP2 AMD 4P Optimization |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 58
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
well if i have 2.5 gigs of ram per core ... for example in a 48 core 128 Gig build ... how would a swap file help? ... yah sure i could put one on a SSD easy ... but not seeing the benefit ... what am i missing? I didn't experiment much with swap partition yet so I can't answer you question precisely, but on a laptop equipped with 4 cores-8 threads 4th generation Intel processor, 8 GB of RAM memory (2x4 GB, dual channel), 7 GB reserved for BOINC data folder, 500 GB HDD and Linux Mint OS I have increased swap partition from 1 GB to 2 GB succeding in increasing from 6 to 7 the number of concurrent CEP2 WUs crunched entirely without erros (except for their last job which my PC can't complete even with 1 WU at a time). Based on my experience so far I guess 1 GB of memory per WU is required and an additional 2 GB swap partion is fine. But I'm pretty sure soon I'll see this setting becoming insufficient and I'll have to increase swap partition more. No one has provided information about this procedure yet as far as I know so there's no sure answer to your question. I suggest you to make some attempts staring with a few cuncurrent WUs and a certain swap partition, increase the WUs number gradually untill you see one being terminated before the last job with the "no space left on device" error in result log, then increase the swap partition by 1 GB at least and see where you end up. |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Just to be sure, the 2.5GB free disk space, is a one time check per work fetch, whether that is getting one or 10 [it's not multiplied]. It's a safety that is fine/functional with running one at the time [BOINC is from the days of single core devices]. Nowadays on running tasks BOINC is supposed to keep track of total use, RAM/VM [which for me on Windows is set to auto-expand, with a minimum size, and in it's own partition, to prevent fragmentation]. If it runs out of memory, it is supposed to start parking tasks and giving a 'waiting for memory' message, but with LAIM on this turns Catch 23, paused tasks are held in memory
For Linux I never looked, it's 3GB on my 4 core machine [own partition as default on Linux], and hardly ever used, because Linux tries to keep everything in RAM that can be kept there... only when RAM is exhausted will it move pieces to VM [and there's a swappiness parm you can set how aggressive this has to happen, mine is set to 90, where default is like 60, the main reason why VM is hardly used]. The less VM use the faster the operations, particularly on CEP2. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
just waiting on UPS guy ... not much more i can do till then ... i did hear ... starting with one core ...doing a start up ... waiting a few minutes ... adding another core .... and repeating this process ... is the best way to get a staggerd start
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
As expected, in each of the last three runs of 8 cuncurrent CEP2 WUs up to two of them errored out (after the first job at least was done so I'll get credit anyway) lacking resources despite having a larger swap partition than before, 5+ GB vs 2 GB. It seems a bigger swap partition can help in reducing RAM usage only to a certain extent: the constraint in choosing how many WUs to run concurrently is RAM size * , then a bigger swap partition can slightly reduce the amount of memory needed but not so much to exclude the previous limit.
----------------------------------------* at least 1 GB needed per concurrent WU when mounting BOINC data folder on RAM and crunching only CEP2 [Edit 1 times, last edit by Former Member at Dec 15, 2015 8:44:00 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
ok ... thanks for all the good info ... it has become obvious to me that i currently dont have the hardware for an all CEP2 box ... but my boxes are an ongoing project that gets updated as funds become available ... so i have a build direction now ... currently my team is all about the Xmas race ... so im going to put off playing with RAM OS'es and SSD Swap files n sutch till after its over ... but keep putting up good info ... it helps
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
here is a NUMA-aware version of BOINC if anyone is interested
http://hardforum.com/showthread.php?t=1805578 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
ok i have put together 2x single processor systems ... "test dummies" for this project ... they have 32gigs of RAM in, 2 banks of quad channel, that is node, channel, and bank interleaved, a 16 core processor, (that should be fast enough to complete WU's in 18 hours), and a 240gig SATA III SSD, with 128gig swap file. and a Maxwell2 GPU. other systems specs should be irrelevant ... so moving forward from here ... i go to FatDog64 ... and i start reading, and reading, and more reading ... one thing i consider is ... is FD64 the best tool for the job? ... so i look up RAM resident Distros of linux ... and i find this ... https://en.wikipedia.org/wiki/List_of_Linux_distributions_that_run_from_RAM
----------------------------------------well. reading this is see "Additionally some distributions can be configured to run from RAM, such as Ubuntu using the toram option included in the Casper scripts.[2]" Really? there is "stuff" that can go from this install, like liber office, to make it "lighter" but why cant i just use Ubuntu, that i have installed and configured??? and how do i do this? or am i not understanding what i read?  [Edit 1 times, last edit by Former Member at Dec 28, 2015 10:20:57 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hay: ... got this link from Gilthanis @ [H] ... he is just a wealth of information ... thought i would share it as it seems to be the best answer i have come across and what i will be trying when i have the time
http://www.xtremesystems.org/forums/showthrea...mp;viewfull=1#post5223632 |
||
|
|
|