Just that the amount and frequency of communication between boinc.exe and the applications would not cause that much system calls. It is somewhere inside the application.

I'd agree if its not actually going to disk it'll be *relatively* less bad, but the sheer quantity of them is having an effect. I looked at the app at one point and it was reporting something like 8 minutes in kernel out of 70 minutes, total, that's an awful lot of wasted time in the kernel. It's not that big a memory hog either. I just can't see how any 'normal' memory usage and code would do anything like this when there is so much system memory free. I've got apps here using more memory than HCC and they don't do it. And other crunching tasks don't do it. The only other time I see that much kernel usage is normally real disk activity, though I've noticed 3d games doing it which I put down to the weird 3D accelerator/bus activity etc. Obviously HCC won't be doing that!

If you aren't explicitly calling Open/CreateFileMapping do you have a dll there with #pragma data_seg on some commonly used area or something? or any other 'wierd' attributes to the data? I would avoid that as I *believe* it creates an implicit shared memory area. If you aren't looking at boinc shared memory often I'm thinking it might be something like that. If there IS a shared memory thing somewhere, it might explain: the paging, the kernel time, and also as Windows will surely have to maintain the shared bit across cores might explain poor performance on multicore CPUs. So it might explain everything, which is why I'm flogging this particular horse :)

I'm wondering ...

I was just messing with an app I have source for that page faults a bit and noticed as I stepped through has little page fault blips when I allocate a not that big chunk of memory on the heap.

I did a test of a little stupid loop (VC++), in a RELEASE version of a program a-la:
for( int n = 0; n < 1000000; n++)
{
int * p = new int[rand() + 1]; // Page fault city
//int * p = new int[100000]; // Faults if larger, this value ok.
//int * p = new int[RAND_MAX]; // pretty much no faults.
delete [] p;
}
of course this is a stupid loop, and of course the CPU maxes out, but the page faults can go absolutely loopy, 80% kernel usage. This code, while stupid, isnt' technically 'wrong' in that it doesn't leak. The program sits on a constant memory usage. I'd have thought it might realloc from the freed heap space, but it's only does this at lower constant values, which allocs and frees with little effort, at higher or random values, it's hammering the kernel. I'm guessing the random size means there is a heap fragmentation thing or something, even though it could fit in the same size as the static version. If anything else is touching the heap I wouldn't rely on a 'realloc'
anyway. I'm wondering how a multicore system handles memory allocs as it will end up in the main memory sometime so it must be synchronised?

Perhaps it's something like this. Perhaps you have somewhere in the algortithm an on paper 'correct' non leaky allocation and dellocation that could be moved outside some loop, made some suitable size and reused. It's an optimisation you might make anyway, but the above may be something to have a good look for!

Let me know if any of this solves things. I've got me teeth into this now :)
PS RAND_MAX will be 32767, not THAT much memory these days(x4 of course for the byte pedantic!).

ADDENDUM
I tried using old C straight malloc in a loop to see what happened, results were:
// Loads CPU but no page faults
void * p = malloc( RAND_MAX);
void * p2 = malloc( RAND_MAX);

Loads CPU but no page faults
void * p = malloc( rand());

// Following pair 350,000 PF Delta, 75-80% kernel
void * p = malloc( rand());
void * p2 = malloc( RAND_MAX);
// (Changing p2 to rand() makes 300k PF, ~70% less kernel)

free( p); // Each test had relelvant free()s
free( p2);

Hi steveleg, HCC has been the highest priority since November. It's just that this particular problem is only one of several. Right now we are getting HCC up on Linux, which will give us a platform to compare with Windows.

I like Highwire's approach because he is looking at possibilities that can be cured. My own thinking tends toward difficulties that are not so amenable, but I want to stay alert to more hopeful possibilities.

It isn't necessary to try to alert the techs to this. They have been looking at it for weeks.

Lawrence

We're with you Lawrence.
That tends to get lost at times but we're all after the same thing.
Our best to the techs and they will get to this when they can.
Reading some of the above and only comprehending some of it I wish I'd gotten more interested in programming than the hardware end..