Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 116
Posts: 116   Pages: 12   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 16769 times and has 115 replies Next Thread
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0

Still toying with the tool, it's grown out of proportion to be honest, but it's fun to find the limits, and move them. Latest addition, an error analysis [which in my next dream will hotlink to the actual result status record for detail... click and hypervisor over]. Todays rough out...



The timestamp is WCG's converted to local time, so it may correlate to something daft being done at the time of the task's demise. Host names hidden ;>)

For me, just mosey over to the Error Report tab, but put your coffee down first and keep the prosac ready [pre-emptive strikers take one first (someone said crunching was fun tongue.]

(And Marshmallow installed on the tablet last Friday and since, zero, not, null, nada errors, even while crunching on battery while gaming.)
[Nov 10, 2015 7:59:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

LoL de LoL, on "[which in my next dream will hotlink to the actual result status record for detail... click and hypervisor over]",
... it proved absolutely trivial... hover and see, click and go.



And with a little tweak the url became

#ResultXml!B6343:AA6343

To select the whole record from the cell where the result name is, which showed it bombed after 0.0001 CPUTime ;O

Next step, open the detail record in a easily viewable form

Hasta la bami goreng, picante drooling
[Nov 11, 2015 1:01:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

One better, show result name and on hover see the record, click and jump to the full detail... muahahaha
[Nov 11, 2015 1:40:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

Today was button generation day [with artful descriptors], from navigation central jump straight to the various reports of interest [all auto-generated, ready to peruse or get exited over].

Also, just implemented code to max out the cores use during the key processes [Archiving and Result fetching can be run in up to 1024 threads in Excel, to get knreed thoroughly wound up... after all he was the one who limited a max fetch of 250 at a time ;]. Start at 00:06:11 and finish at 00:06:13. Of course that's not going to happen as the CPU on which this is running 'only' has 8 cores and still at a leisurely mid-summer speed of 1.9Ghz.

(And while typing this, everything finished in no time)

Hasta la Fuyong Hai
[Nov 12, 2015 4:55:32 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

Of course, when maxing out doing my own stuff, BOINC gets to play second fiddle....

3016 11/12/2015 5:55:08 PM Suspending computation - CPU is busy
3017 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting E234668_908_S.216.C27H31N1S1.BPMDLFDZFGSEEZ-UHFFFAOYSA-N.18_s1_14_0 (left in memory)
3018 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_0397_0 (left in memory)
3019 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_0970_1 (left in memory)
3020 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_1959_1 (left in memory)
3021 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_2136_1 (left in memory)
3022 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_2105_0 (left in memory)
3023 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19804_0641_0 (left in memory)
3024 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19808_1947_1 (left in memory)
3025 11/12/2015 5:55:18 PM Resuming computation


At least, the tasks don't get unloaded, nor crash due time-outs, which CEP2 is rather sensitive to.
[Nov 12, 2015 5:12:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7697
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

I have been following this and am impressed with how far you have taken this with all the enhancements. I use a simpler setup but not anywhere near as elegant as you have. I had a person write a little program to fetch the results and pop them into a text file which increments each time. I run this twice a day and just pop the results into a spreadsheet about once a week where I get rid of the duplicates. I am just dealing with the valid entries so I keep it simple. The download of the results just takes a few seconds. My hat is off to you for all the work you have put into your tool.

Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Nov 12, 2015 10:50:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0l

'Over the top' it's gone for sure, but still fun... if there's a blockage on progress, I will find a solution. Today "On Error Resume Next". Sometimes a scraped page is not reachable, so the script hangs [yes we still have stuff needing scraping the old previous century way]. This one just skips the first offending [non-critical] line, continuing on... stats ran very long last night :(.

Most irritating was that in Excel the result archiving fetches a filtered record set [and we're limited by WCG to 250], but if none of the records meet the conditions, Excel copies the whole filtered out set instead of copying nothing [skip on or present an error code to handle]. An additional [initial ugly code] test determines if filtered map results in zero, then continues with the next page [max 250 again], in the loop.

Old days '>>', append additional input to existing text file, circa 1990 it must have been I last used that function, but nothing wrong with it :O)
[Nov 13, 2015 10:47:14 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0m

Today continuing on from adding hunting tool related standard office buttons that show as extra on the Quick Access Toolbar [QAT]. Created several 'custom' buttons to that bar [only showing when the tool is loaded]. One is a 'Home' button with the oh-so familiar browser icon, and assigned a shortcut key to this as well. Hit Ctrl+m and you get transported back to control central, setting the selector right below the buttons. [If you use the default assignment of Office to Ctrl+m, none far as I can find, you're SOOL].

... and a very very nasty bug erased 100+ and 99- gives a differential of 1, but really there's a 199 problem, which was the type of issue. And something got put in last night that got a this morning's 13000 record fetch down to under 2 minutes and recompute [Mr. Data could not keep up with this ;P]. Rather surprising considering we're here still on copper wired internet [and sat, because cable does not exist this side of the Alps.

Think we're going to have a Hasta la Kebab meal (with spiced pila rice).
[Nov 14, 2015 3:27:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0o

Crisis what crisis? The testing data set expanded from < 600 results to 9000 overnight and Office did not like that [the way WCG offers a max of 250 per Results request]. Fortunately I knew where to look and hit Google 100% score on the top item, 5 minutes understanding, 30 minutes preparation of an input map into a for-next, discovering how many records to come [the API tells you on the first record received], and off to listobject sizing, this time the testing was only 5 minutes of the ~60 total time spent.

Only now noticed MSDN-Dev Center is using GitHub for development and documenting, same as Berkeley BOINC is kept on]. The development speedup actually came mostly from a code I wrote yesterday, automating the column sizing for 34 fields, which runs in less than a blink of an eye [A consistent presentation of the data view is always pleasing ;P]. Re-usable code is great... another module added to the Hunting too, v2.0o

Happy birthday WCG, back to a run in 2 minutes, over copper telephone wire. smile
[Nov 16, 2015 10:13:12 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 2.0o

Btw, moved development down to Office 2010. Upward compatibility is guaranteed, the reverse from 2013/2016 to 2010 was not, no matter how much effort [backward compatibility testing time].

And thanks goes out to thé member making the 'big' data available.


[Nov 16, 2015 10:29:32 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 116   Pages: 12   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 | Next Page ]
[ Jump to Last Post ]
Post new Thread