Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
![]() |
World Community Grid Forums
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 23
|
![]() |
Author |
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hi lavaflow,
I'm seeing some of this too, but my view is a little different from yours. I do see the System Idle Process come up the pile, but far too often for checkpoints to be the cause. Having kept an eye on things I think it's the result file which is continually being written. I suppose I would have expected the result data to have been buffered and written at the same frequency as the checkpoints, but that's clearly not the case. Also, I'm seeing different efficiencies for different batches which, I think, supports my conjecture. But I'm surprised that the OS isn't itself caching the result file as it's only a few MB on the ones I've looked at. Now it's a long time since I did any I/O programming, but maybe the code has specified a synchronous, forced write-to-disk? If so, I think that's a mistake! In the BOINC environment coding it to write at the same time as the checkpoint, or only making it synchronous at checkpoint time, would have been the correct thing to do, IMHO. |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Running with the noted 600 second write to disk setting, laim on. The efficiencies vary quite a bit, which may be related to the sizes of the compared sequences and the output file. They vary greatly.
Can't believe a wtd is not adhered to in line with the checkpoint interval at most setting. If there'd be one every 'internally unlogged checkpoint', i'd be fuming, but no, the 600 seconds has certainly raised the lowest efficiency well above 95 percent, in fact now running all cores ugm and the range is 96 to 98.6. The 3 hours average runtime, in my dreams. This not pitiful device gets 3:28 hours at best. The project average is just a fraction under 4 hours. Back on efficiencies, see the ugm effecting mcm. The more ugm run the lower too for mcm. With mcm only, 99+. Just wonder why all cores under ubuntu gives 99.8, one even hitting 99.97 percent efficiency, upon completion. Does size matter? It seems. |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I'm obtaining 99% efficiency with 8 UGM threads on a fully-loaded i7-4770K under Win 8.1, checkpoints every 120 sec. The primary HDD is an ancient Maxtor DiamondMax 10 SATA2 250GB but with a Crucial C300 64GB SSD as cache (Intel SRT, simultaneous-write mode). The beta units were similar.
|
||
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7655 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I am getting 100 % efficiency on Windows Vista on a stock q6600. On a Linux machine I was getting 99%+ efficiency on a core 2 Duo at 2.67ghz. I see efficiency as good or better than the beta runs.
----------------------------------------(Got to get the bronze and then back to MCM1) ![]() Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I have three Haswell machines, two with six cores shared between UGM and MCM1, and the other one with four cores (all Win7 64-bit). Each machine also has two cores devoted to GPU support. In general, I have been getting 99+% for both UGM and MCM1.
But on my third machine with only four cores running (two others supporting GPUs and two cores free), I just took a tumble down to below 50% on both a UGM and an MCM1 for no apparent reason (all machines have fast SSDs by the way). I have never seen this before. At any rate, a simple reboot did not fix it. But I needed to change the drivers for my Nvidia cards (GTX 750 Tis) anyway, so I went back from 344.11 to 340.52, and the CPUs returned to their normal 99+%. This may be just a coincidence and due to other things, but if you have an Nvidia card installed you might want to check it. |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Did some tweaks, to include excluding the data-dir from av scanning, and switching os priority from background to foreground processes and getting 99 percent or better on ugm, but it keeps degrading the more are run, and keeps impacting mcm too. Coded up an app_config to set for 2 / 6 ugm / mcm run ratio, which are the allowed sciences at this time for the node's associated profile. Slightly risky, as the ratio of ugm/mcm is getting 1 in 9, so it could happen a ugm core goes idling. Surely the technicians will change this once they've figured out the storage needs with a few weeks live and no assimilated results, at lowest feeder preference.
<app_config> <app> <name>cep2</name> <max_concurrent>1</max_concurrent> </app> <app> <name>mcm1</name> <max_concurrent>6</max_concurrent> </app> <app> <name>fahv</name> <max_concurrent>7</max_concurrent> </app> <app> <name>faah</name> <max_concurrent>1</max_concurrent> </app> <app> <name>ugm1</name> <max_concurrent>2</max_concurrent> </app> </app_config> To do still is create a swap file specific partition, stopping fragmentation, per http://www.online-tech-tips.com/computer-tips...figuring-the-paging-file/ . Even a few tenth of a percent improvement counts up to lots of hours extra computing in a year. |
||
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Slightly risky, as the ratio of ugm/mcm is getting 1 in 9, so it could happen a ugm core goes idling. Surely the technicians will change this once they've figured out the storage needs with a few weeks live and no assimilated results, at lowest feeder preference. I was wondering why I wasn't getting more UGM, though the ratio seems to be about 1 UGM to 2 MCM now. The run times are about the same at the moment, being around 2 1/2 hours. [Edit 1 times, last edit by Jim1348 at Oct 22, 2014 5:32:54 PM] |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The ratio seemed to have bettered a bit. Now about 18 hours further, got 5 ugm and 30 mcm in ready to start for a 0.5 day buffer, making it about a 1:6 for this project combination. The ready to start dropped as low as 3, but so far the re-activated simap -a zero weight project- to fill any idle cores, has not had to be called on.
Efficiency continues at just 99 percent with a 2-6 computing share. Wish there was a way to log the file sizes coming down. This way maybe could correlate performance to the size of the txt files. The latest were 6.1mb for 2 tasks out of batch 574. Looking in the project folder, noticed the time stamps of the slideshow image files are being updated to that of the last task downloaded or updated. For instance the ugm slides have a creation date of 10.14 but a modification date/time of just few minutes ago on oct.23. This is with all project slide files, faah/fahv/cep2/mcm/ugm. Date modified being constantly updated may be something else impacting performance. Need to find why and if this can be switched off. Yuk, just happened again, timestamps moved 11 minutes forward on all the slide files, some 40, and one mcm txt file, which is the one just started at that moment. Is that with all the projects the case? Testing, suspended a running ugm, the app_config forcing another ugm to start and, and timestamp modding. Forced another ugm to start and it happens, next mcm starts, all timestamps renewed again. @ technicians: Why are the slide file time/dates modified when a wcg task launches, all slides for all projects, this on windows. Hmmm, the search indexer would not like that and get hyperactive. Need to exclude this boinc datadir structure from index inclusion, if it can be, and it can be http://www.sevenforums.com/general-discussion...lude-folder-indexing.html and http://www.sevenforums.com/tutorials/36098-search-options-change-restore.html and done. |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Think there's a little hosanna in place as efficiencies have climbed uniformly by several tenths of percentage points since excluding the boinc datadir from windows search & indexing selections. There could be more to come over the full runtime of a task.
The slide date modified keeps altering. How to not set it for folders i've found. There's comment also on thumbnails, but setting to medium icon, there's no thumbnail images, opposed to the .png files. This http://superuser.com/questions/808831/how-can...folder-last-modified-date implies a connection. |
||
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Think there's a little hosanna in place as efficiencies have climbed uniformly by several tenths of percentage points since excluding the boinc datadir from windows search & indexing selections. There could be more to come over the full runtime of a task. Interesting point; I always disable the Windows Search service when installing Win7 to eliminate unnecessary disk accesses, since it isn't really needed with a fast SSD anyway. |
||
|
|
![]() |