Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 23
Posts: 23   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3886 times and has 22 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Hi lavaflow,

I'm seeing some of this too, but my view is a little different from yours.

I do see the System Idle Process come up the pile, but far too often for checkpoints to be the cause. Having kept an eye on things I think it's the result file which is continually being written. I suppose I would have expected the result data to have been buffered and written at the same frequency as the checkpoints, but that's clearly not the case.

Also, I'm seeing different efficiencies for different batches which, I think, supports my conjecture.

But I'm surprised that the OS isn't itself caching the result file as it's only a few MB on the ones I've looked at. Now it's a long time since I did any I/O programming, but maybe the code has specified a synchronous, forced write-to-disk? If so, I think that's a mistake! In the BOINC environment coding it to write at the same time as the checkpoint, or only making it synchronous at checkpoint time, would have been the correct thing to do, IMHO.
[Oct 16, 2014 12:53:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Running with the noted 600 second write to disk setting, laim on. The efficiencies vary quite a bit, which may be related to the sizes of the compared sequences and the output file. They vary greatly.

Can't believe a wtd is not adhered to in line with the checkpoint interval at most setting. If there'd be one every 'internally unlogged checkpoint', i'd be fuming, but no, the 600 seconds has certainly raised the lowest efficiency well above 95 percent, in fact now running all cores ugm and the range is 96 to 98.6.

The 3 hours average runtime, in my dreams. This not pitiful device gets 3:28 hours at best. The project average is just a fraction under 4 hours.

Back on efficiencies, see the ugm effecting mcm. The more ugm run the lower too for mcm. With mcm only, 99+. Just wonder why all cores under ubuntu gives 99.8, one even hitting 99.97 percent efficiency, upon completion. Does size matter? It seems.
[Oct 17, 2014 8:20:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

I'm obtaining 99% efficiency with 8 UGM threads on a fully-loaded i7-4770K under Win 8.1, checkpoints every 120 sec. The primary HDD is an ancient Maxtor DiamondMax 10 SATA2 250GB but with a Crucial C300 64GB SSD as cache (Intel SRT, simultaneous-write mode). The beta units were similar.
[Oct 17, 2014 9:04:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7655
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

I am getting 100 % efficiency on Windows Vista on a stock q6600. On a Linux machine I was getting 99%+ efficiency on a core 2 Duo at 2.67ghz. I see efficiency as good or better than the beta runs.
(Got to get the bronze and then back to MCM1) tongue
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Oct 19, 2014 3:01:13 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

I have three Haswell machines, two with six cores shared between UGM and MCM1, and the other one with four cores (all Win7 64-bit). Each machine also has two cores devoted to GPU support. In general, I have been getting 99+% for both UGM and MCM1.

But on my third machine with only four cores running (two others supporting GPUs and two cores free), I just took a tumble down to below 50% on both a UGM and an MCM1 for no apparent reason (all machines have fast SSDs by the way). I have never seen this before. At any rate, a simple reboot did not fix it. But I needed to change the drivers for my Nvidia cards (GTX 750 Tis) anyway, so I went back from 344.11 to 340.52, and the CPUs returned to their normal 99+%. This may be just a coincidence and due to other things, but if you have an Nvidia card installed you might want to check it.
[Oct 21, 2014 4:56:59 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Did some tweaks, to include excluding the data-dir from av scanning, and switching os priority from background to foreground processes and getting 99 percent or better on ugm, but it keeps degrading the more are run, and keeps impacting mcm too. Coded up an app_config to set for 2 / 6 ugm / mcm run ratio, which are the allowed sciences at this time for the node's associated profile. Slightly risky, as the ratio of ugm/mcm is getting 1 in 9, so it could happen a ugm core goes idling. Surely the technicians will change this once they've figured out the storage needs with a few weeks live and no assimilated results, at lowest feeder preference.

<app_config>
<app>
<name>cep2</name>
<max_concurrent>1</max_concurrent>
</app>
<app>
<name>mcm1</name>
<max_concurrent>6</max_concurrent>
</app>
<app>
<name>fahv</name>
<max_concurrent>7</max_concurrent>
</app>
<app>
<name>faah</name>
<max_concurrent>1</max_concurrent>
</app>
<app>
<name>ugm1</name>
<max_concurrent>2</max_concurrent>
</app>
</app_config>

To do still is create a swap file specific partition, stopping fragmentation, per http://www.online-tech-tips.com/computer-tips...figuring-the-paging-file/ . Even a few tenth of a percent improvement counts up to lots of hours extra computing in a year.
[Oct 22, 2014 3:41:28 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Slightly risky, as the ratio of ugm/mcm is getting 1 in 9, so it could happen a ugm core goes idling. Surely the technicians will change this once they've figured out the storage needs with a few weeks live and no assimilated results, at lowest feeder preference.

I was wondering why I wasn't getting more UGM, though the ratio seems to be about 1 UGM to 2 MCM now. The run times are about the same at the moment, being around 2 1/2 hours.
----------------------------------------
[Edit 1 times, last edit by Jim1348 at Oct 22, 2014 5:32:54 PM]
[Oct 22, 2014 5:24:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

The ratio seemed to have bettered a bit. Now about 18 hours further, got 5 ugm and 30 mcm in ready to start for a 0.5 day buffer, making it about a 1:6 for this project combination. The ready to start dropped as low as 3, but so far the re-activated simap -a zero weight project- to fill any idle cores, has not had to be called on.

Efficiency continues at just 99 percent with a 2-6 computing share. Wish there was a way to log the file sizes coming down. This way maybe could correlate performance to the size of the txt files. The latest were 6.1mb for 2 tasks out of batch 574.

Looking in the project folder, noticed the time stamps of the slideshow image files are being updated to that of the last task downloaded or updated. For instance the ugm slides have a creation date of 10.14 but a modification date/time of just few minutes ago on oct.23. This is with all project slide files, faah/fahv/cep2/mcm/ugm. Date modified being constantly updated may be something else impacting performance. Need to find why and if this can be switched off.

Yuk, just happened again, timestamps moved 11 minutes forward on all the slide files, some 40, and one mcm txt file, which is the one just started at that moment. Is that with all the projects the case? Testing, suspended a running ugm, the app_config forcing another ugm to start and, and timestamp modding. Forced another ugm to start and it happens, next mcm starts, all timestamps renewed again.

@ technicians: Why are the slide file time/dates modified when a wcg task launches, all slides for all projects, this on windows.

Hmmm, the search indexer would not like that and get hyperactive. Need to exclude this boinc datadir structure from index inclusion, if it can be, and it can be http://www.sevenforums.com/general-discussion...lude-folder-indexing.html and http://www.sevenforums.com/tutorials/36098-search-options-change-restore.html and done.
[Oct 23, 2014 8:24:46 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Think there's a little hosanna in place as efficiencies have climbed uniformly by several tenths of percentage points since excluding the boinc datadir from windows search & indexing selections. There could be more to come over the full runtime of a task.

The slide date modified keeps altering. How to not set it for folders i've found. There's comment also on thumbnails, but setting to medium icon, there's no thumbnail images, opposed to the .png files. This http://superuser.com/questions/808831/how-can...folder-last-modified-date implies a connection.
[Oct 23, 2014 10:48:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Seeing lower than beta19 efficiencies

Think there's a little hosanna in place as efficiencies have climbed uniformly by several tenths of percentage points since excluding the boinc datadir from windows search & indexing selections. There could be more to come over the full runtime of a task.

Interesting point; I always disable the Windows Search service when installing Win7 to eliminate unnecessary disk accesses, since it isn't really needed with a fast SSD anyway.
[Oct 23, 2014 11:50:56 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 23   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread