I have noticed something similar with one of my Linux systems. I have come to suspect the system itself encounters some condition - unknown at the moment- and creates some kind of internal blockage. If I initiate a manual update it takes right off and gets more workunits. As far as I can determine it happens randomly. It is a dedicated system which only runs WCG 24/7.

Cheers

Unixchick,

Nice summary of the client behaviour!

It's worth noting that when the client reports completed work it is not guaranteed that it will ask for new work -- that can be quite annoying if a user with lots of threads is depending on reports to also be requests. In my view, there is still room for improvement in client behaviour regarding requests :-)

Also (regarding "luck of the draw"), I suspect that if lots of simultaneous requests for work go in, smaller requests are more likely to be satisfied (less iterations in the work-issuing loop!)

So I suspect that folks who set up for small caches and fast turnaround may be less likely to run out of work, especially if they prod the client every 15, 20 or 30 minutes to wake up the request mechanism. However, large caches and prodding the client every 5 (or less) minutes may be counter-productive!

Cheers - Al.

P.S. I'd rather return work promptly and run the risk of running out than maintain the sort of cache size that some users seem to do (accounting for the 10 to 15% of my MCM1 tasks that see at least one wingman missing deadline on a typical day...) -- big caches for huge machines, yes (though trying to fetch 1000 tasks at once may be doomed to failure); 5+ day caches for smaller machines, no :-)

Only when the tasks are put into high priority mode which only happens when they are seen to be in danger of missing their deadline.

First of all, the hardware is different,

the good one has 2 Xeon 2696 V2 cpu's (48 threads, 4023 Whetstone, 88393 Dhrystone), 384 Gb memory
the bad one has 2 Xeon 2696 V4 cpu's (88 threads, 4923 Whetstone, 78313 Dhrystone), 128 Gb memory
Both are available for 100% for WCG.

The good one consistently runs 48 WU's and has a buffer containint 950-1000 wu's.

It runs 350-400 WU's per day:

2023-12-27 0:047:22:19:00 203531 385
2023-12-26 0:044:03:29:52 188538 358
2023-12-25 0:048:10:42:13 202057 390
2023-12-24 0:048:03:43:20 210460 400
2023-12-23 0:045:11:33:05 196557 378
2023-12-22 0:045:11:53:00 193925 366
2023-12-21 0:047:21:24:56 203064 388

The bad one never has more than 200 in the buffer, often only 30-50 and runs out of work frequently. Yesterday for example was a bad day:

2023-12-27 0:040:12:26:34 203124 387
2023-12-26 0:081:17:47:44 358064 711
2023-12-25 0:061:17:06:47 270355 512
2023-12-24 0:079:07:05:42 346041 649
2023-12-23 0:064:18:53:11 283429 545
2023-12-22 0:046:18:23:06 218234 429
2023-12-21 0:074:22:09:40 319908 607

There is no difference in global_prefs.xml or global_prefs_override.xml except for the max_cpus value, of course. Both have the same profile in the device manager (default, runs for maximum output). The only thing I can find is that the good one was, according to the device manager, installed in November 2022, and the bad one was installed in November 2023 (well, it got a new motherboard then).

All in all, I don't see a cause for the significant difference here.

Thunder 7,

In the profile you are using, have you set the project limits to "unlimited" ? I have a 32 thread machine with a 2 day cache set that way. It works better than the setting of 64 and does not overload the machine.

Cheers