Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 28
Posts: 28   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1773 times and has 27 replies Next Thread
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2167
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Other than looking at the properties while the task is running, is there an api which would give the executable of the completed work unit ? I know it shows up in the results log for each unit, but I don't know how to have a quick and easy way to list this information for each of the completed units.
Thanks

Cheers

If you want to list the information for each of the completed tasks belonging to a certain workunit and you know the workunit ID, then here's an answer.

First of all, if you don't know the workunit ID and you don't know the name of the workunit yet, you could use the Linux program wcgresults to give you workunit results. I'm using it - at least - daily (or even hourly) to show me the latest results.

That's not entirely true: while wcgresults can show you the latest results, it can also record the results into a logfile (even automatically, using your crontab, in a more permanent way) and then you can use another program to show the latest results, for which I'm using the Linux program wcglog. This program can show your latest results - while wcgresults is recording them hourly - by constantly following the output of wcgresults, let's say some sort of 'tail -f', if you wish by using ctrl-Z (to background) and 'fg' to foreground the command again.

The important part is that wcglog can show workunit IDs.

Here is a short sample of a session:
$ wcglog -f -W@l1
App    CpuTime Elapsed Claimed Granted ModTime    Exit Outc SentTime            ReceivedTime        WorkunitId Name
mcm1 0.96 0.96 75.8 76.9 1729962475 0 1 2024-10-23T04:17:47 2024-10-25T08:44:28 619472043 MCM1_0226857_6610_0
^Z

One hour later:
$ fg
mcm1 1.35 1.36 79.1 80.1 1729962967 0 1 2024-10-23T11:10:07 2024-10-25T10:55:58 619679444 MCM1_0226862_8495_0
^Z

In this sample above you'll notice the ID of the workunit that is the outlier, using up 1.35 hours of CpuTime instead of the projected hour (0.96) of processing time. So the workunit ID of the outlier is 619679444.

Now, the quick and easy way is to use the Linux program wcgstats. It can guide you to the workunit if you know (part of) its name; or, if you know its (workunit) ID, it will be even quicker to show the error log for both tasks - and in each error log you will find the name of the executable for the task.

Sample of the Linux session:
$ wcgstats -w -errr= 619679444
workunit 619679444
MCM1_0226862_8495_0  Fedora Linux  Valid  2024-10-23T11:10:07  2024-10-25T10:55:58    1.35/1.36      79.1/80.1
MCM1_0226862_8495_1 Linux Valid 2024-10-23T11:09:43 2024-10-26T17:16:01 3.09/3.11 81.2/80.1
Details: ---------------------------------------------------------------------------------------------------------------------------------------
MCM1_0226862_8495_0  Fedora Linux  Valid  2024-10-23T11:10:07  2024-10-25T10:55:58    1.35/1.36      79.1/80.1
Logfile:
<core_client_version>7.20.2</core_client_version>
<stderr_txt>
Commandline = ../../projects/www.worldcommunitygrid.org/wcgrid_mcm1_map_7.61_i686-pc-linux-gnu -SettingsFile MCM1_0226862_8495.txt -DatabaseFile dataset-curatedOvarian_EarlyLate_v1.0.txt
[snip---8<---snip---8<]
</stderr_txt>
MCM1_0226862_8495_1 Linux Valid 2024-10-23T11:09:43 2024-10-26T17:16:01 3.09/3.11 81.2/80.1
Logfile:
<core_client_version>7.6.31</core_client_version>
<stderr_txt>
Commandline = ../../projects/www.worldcommunitygrid.org/wcgrid_mcm1_map_7.61_i686-pc-linux-gnu -SettingsFile MCM1_0226862_8495.txt -DatabaseFile dataset-curatedOvarian_EarlyLate_v1.0.txt
Settings File
DateOfDesign = 08/05/2014
Designer = Krembil-cubes-2023-09-22
WorkOrderID = 0226862_8495
[snip---8<---snip---8<]
[08:59:21] Initializing
[08:59:26] Running
[08:59:26] EvaluateFitnessOfStartingGeneSignatures 2555
[12:15:53] Writing final output
[12:15:53] Closing Output Stream
[12:15:53] Cleaning up
Result.out = 1666.000000
Run complete, CPU time: 11133.991612
12:15:53 (8387): called boinc_finish(0)

</stderr_txt>

I klnow it's a long message, so after reading all this, you might be interested in the location of wcgresults, wcglog and wcgstats.
The home of WCGtools: https://sourceforge.net/projects/wcgtools/
Documentation: https://sourceforge.net/p/wcgtools/wiki/Home/ - it is important that you read this! Especially 'Configuration'.
Software - wcgresults, wcglog and wcgstats: https://sourceforge.net/projects/wcgtools/files/

Adri
PS I have described only one particular way to use the programs.
NB The ways in which to use the programs are countless, so there's some learning curve.
----------------------------------------
[Edit 1 times, last edit by adriverhoef at Oct 28, 2024 9:23:04 AM]
[Oct 27, 2024 12:35:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TonyEllis
Senior Cruncher
Australia
Joined: Jul 9, 2008
Post Count: 261
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

I have some recent statistics
Thanks Adri. Both of my i686 WUs verified. I note they were a bit quicker than one of the results you provided. Not too shabby for an 18 year old laptop pulled from the trash together with its power supply. It is in perfect condition - not a mark or scratch. Only surprise is the keyboard keys marked in both US English and Arabic. Maxed out the memory to 4 GB and installed a PATA SSD from the spares box to improve performance.

geophi - I use the data obtained from Adri's "wcgresults" as the basis for some of the WCG statistics on my website. It's a very useful program (Thanks Adri). Well worth investigating.

Edit typo
----------------------------------------
----------------------------------------
[Edit 1 times, last edit by TonyEllis at Oct 27, 2024 4:11:30 PM]
[Oct 27, 2024 4:10:23 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7670
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Thanks Adri.
I can get the work unit ID from the daily download I keep of each days valid results in a spreadsheet file. Ideally I would batch each day's results and then I could keep some statistics on the various flavors of the executables.
This will be something I will study when I get some time in the future.

Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Oct 27, 2024 10:40:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
AgrFan
Senior Cruncher
USA
Joined: Apr 17, 2008
Post Count: 376
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

https://www.worldcommunitygrid.org/contribution/workunit/655544899

Both units ran on Windows 10 64-bit systems.
- Enterprise x64 (wingman)
- Core x64 (mine)

Both units used the 32-bit executable --> wcgrid_mcm1_map_7.61_windows_intelx86.

I would have expected the 64-bit executable to be used --> wcgrid_mcm1_map_7.61_windows_x86_64.

MCM1_0230461_5467_0 Microsoft Windows 10 Enterprise x64 Edition, (10.00.19045.00) Valid 2025-01-28 02:52:52 UTC 2025-02-02 09:11:53 UTC 1.67 / 1.67 77.1 / 78.9
MCM1_0230461_5467_1 Microsoft Windows 10 Core x64 Edition, (10.00.19045.00) Valid 2025-01-28 02:53:25 UTC 2025-01-28 13:16:02 UTC 2.3 / 2.3 80.7 / 78.9

Both units ran under BOINC 8.0.2.

confused
----------------------------------------
[Edit 12 times, last edit by AgrFan at Feb 4, 2025 4:47:21 AM]
[Feb 4, 2025 4:28:50 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Advanced Cruncher
Joined: Feb 19, 2021
Post Count: 129
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Use this as cc_config.xml:

<cc_config>
<options>
<no_alt_platform>1</no_alt_platform>
</options>
</cc_config>

Or add the line in the middle if you already have a cc_config.xml file. Restart the client after the change and the issue should be gone. Remember that it's for all projects, so projects, which have only 32-bit applications won't send you any work (for example Einstein's FGRP5 on Windows).
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by Link64 at Feb 4, 2025 11:01:29 AM]
[Feb 4, 2025 10:57:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
AgrFan
Senior Cruncher
USA
Joined: Apr 17, 2008
Post Count: 376
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Use this as cc_config.xml:

<cc_config>
<options>
<no_alt_platform>1</no_alt_platform>
</options>
</cc_config>

Or add the line in the middle if you already have a cc_config.xml file. Restart the client after the change and the issue should be gone. Remember that it's for all projects, so projects, which have only 32-bit applications won't send you any work (for example Einstein's FGRP5 on Windows).

My machines run 32-bit and 64-bit work fine. I wouldn't want to limit one or the other.

My question was why the 32-bit executable was used when both units ran on 64-bit Windows systems.

Maybe this unit needed to run on 32-bit and the executable is chosen by a setting specific to the unit?
----------------------------------------
[Edit 4 times, last edit by AgrFan at Feb 5, 2025 1:54:04 PM]
[Feb 5, 2025 1:50:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Advanced Cruncher
Joined: Feb 19, 2021
Post Count: 129
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

My machines run 32-bit and 64-bit work fine. I wouldn't want to limit one or the other.
Well, they should run without errors of course, but it's highly unlikely that both are same fast and there's no reason not to exclude the slower one unless you also run projects, which only have 32-bit applications. My Ryzen 5700G completes one MCM WU with the 64-bit application in about 1h20m, with the 32-bit application it needs IIRC about 30 minutes more. So why would I want to allow the 32-bit application and get less work done for the project?

My question was why the 32-bit executable was used when both units ran on 64-bit Windows systems.
Because without <no_alt_platform> in cc_config.xml your BOINC client reports your system as both to the server: 32-bit and 64-bit. And than the server will send you both and try to determine which is faster.
Once you have run few WUs, it will assign the most WUs to the faster application, but every now and than it will assign one WU to the slower application to verify, that it is still slower.
That way, in case something changes like different workload which completes faster on the other application for whatever reason or if the initial performance estimation was done on some weird WUs (or even on different Hardware in case the User transfers the data folder to new computer), the server can always adapt and send WUs to this application, which is fastest now and not the one which was fastest in the past or the one, which is fastest in theory.
----------------------------------------

[Feb 5, 2025 4:35:13 PM]   Link   Report threatening or abusive post: please login first  Go to top 
AgrFan
Senior Cruncher
USA
Joined: Apr 17, 2008
Post Count: 376
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Because without <no_alt_platform> in cc_config.xml your BOINC client reports your system as both to the server: 32-bit and 64-bit.

This makes perfect sense. I was thinking the BOINC client reports to the server based on operating system only.

I added <no_alt_platform> to cc_config.xml on the machines that have been getting 32-bit units. No issues getting work after restarting the clients.

Thanks for the detailed explanation!
----------------------------------------
[Edit 4 times, last edit by AgrFan at Feb 5, 2025 5:12:09 PM]
[Feb 5, 2025 5:04:50 PM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 971
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

I wonder whether WCG sets the scheduler's prefer_primary_platform flag? If that is set, it shouldn't do that "which is faster" dance [unless there are multiple applications for the primary platform!] :-) -- the rarity with which I see 32-bit tasks suggests that the flag is set!

That would then leave the question as to why the first request to pick up a task for a new work-unit got a 32-bit task, and whether the scheduler just hands off the wingman duties to the first requester it sees that can cope (rather than waiting for another 32-bit-preferred client to ask!) -- the possible reasons for an initial 32-bit send can include someone running a 32-bit client (old build or old hardware?)...

Cheers - Al.

P.S. I have seen the "what app is best" dance on some other platforms that have multiple applications for one platform, but those seemed mostly to be either which floating-point instructions to use (SSE or not?) or choices of GPU application...
----------------------------------------
[Edit 1 times, last edit by alanb1951 at Feb 5, 2025 5:52:38 PM]
[Feb 5, 2025 5:49:56 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Advanced Cruncher
Joined: Feb 19, 2021
Post Count: 129
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: x86 task

Because without <no_alt_platform> in cc_config.xml your BOINC client reports your system as both to the server: 32-bit and 64-bit.

This makes perfect sense. I was thinking the BOINC client reports to the server based on operating system only.
Well, it does, but unlike for example Itanium based systems, x86-64 is both, the old 32-bit and the new 64-bit (as long as the OS supports both of course, this might change).

I wonder whether WCG sets the scheduler's prefer_primary_platform flag? If that is set, it shouldn't do that "which is faster" dance [unless there are multiple applications for the primary platform!] :-) -- the rarity with which I see 32-bit tasks suggests that the flag is set!
Shouldn't in this case 64-bit hosts get only 64-bit tasks without any 32-bit experiments by the server?
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by Link64 at Feb 5, 2025 8:10:36 PM]
[Feb 5, 2025 7:50:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 28   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread