Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
![]() |
World Community Grid Forums
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 116
|
![]() |
Author |
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Still toying with the tool, it's grown out of proportion to be honest, but it's fun to find the limits, and move them. Latest addition, an error analysis [which in my next dream will hotlink to the actual result status record for detail... click and hypervisor over]. Todays rough out...
![]() The timestamp is WCG's converted to local time, so it may correlate to something daft being done at the time of the task's demise. Host names hidden ;>) For me, just mosey over to the Error Report tab, but put your coffee down first and keep the prosac ready [pre-emptive strikers take one first (someone said crunching was fun ![]() (And Marshmallow installed on the tablet last Friday and since, zero, not, null, nada errors, even while crunching on battery while gaming.) |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
LoL de LoL, on "[which in my next dream will hotlink to the actual result status record for detail... click and hypervisor over]",
... it proved absolutely trivial... hover and see, click and go. ![]() And with a little tweak the url became #ResultXml!B6343:AA6343 To select the whole record from the cell where the result name is, which showed it bombed after 0.0001 CPUTime ;O Next step, open the detail record in a easily viewable form Hasta la bami goreng, picante ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
One better, show result name and on hover see the record, click and jump to the full detail... muahahaha
|
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Today was button generation day [with artful descriptors], from navigation central jump straight to the various reports of interest [all auto-generated, ready to peruse or get exited over].
Also, just implemented code to max out the cores use during the key processes [Archiving and Result fetching can be run in up to 1024 threads in Excel, to get knreed thoroughly wound up... after all he was the one who limited a max fetch of 250 at a time ;]. Start at 00:06:11 and finish at 00:06:13. Of course that's not going to happen as the CPU on which this is running 'only' has 8 cores and still at a leisurely mid-summer speed of 1.9Ghz. (And while typing this, everything finished in no time) Hasta la Fuyong Hai |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Of course, when maxing out doing my own stuff, BOINC gets to play second fiddle....
3016 11/12/2015 5:55:08 PM Suspending computation - CPU is busy 3017 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting E234668_908_S.216.C27H31N1S1.BPMDLFDZFGSEEZ-UHFFFAOYSA-N.18_s1_14_0 (left in memory) 3018 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_0397_0 (left in memory) 3019 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_0970_1 (left in memory) 3020 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_1959_1 (left in memory) 3021 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_2136_1 (left in memory) 3022 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19800_2105_0 (left in memory) 3023 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19804_0641_0 (left in memory) 3024 World Community Grid 11/12/2015 5:55:08 PM [cpu_sched] Preempting ugm1_ugm1_19808_1947_1 (left in memory) 3025 11/12/2015 5:55:18 PM Resuming computation At least, the tasks don't get unloaded, nor crash due time-outs, which CEP2 is rather sensitive to. |
||
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7697 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I have been following this and am impressed with how far you have taken this with all the enhancements. I use a simpler setup but not anywhere near as elegant as you have. I had a person write a little program to fetch the results and pop them into a text file which increments each time. I run this twice a day and just pop the results into a spreadsheet about once a week where I get rid of the duplicates. I am just dealing with the valid entries so I keep it simple. The download of the results just takes a few seconds. My hat is off to you for all the work you have put into your tool.
----------------------------------------Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
'Over the top' it's gone for sure, but still fun... if there's a blockage on progress, I will find a solution. Today "On Error Resume Next". Sometimes a scraped page is not reachable, so the script hangs [yes we still have stuff needing scraping the old previous century way]. This one just skips the first offending [non-critical] line, continuing on... stats ran very long last night :(.
Most irritating was that in Excel the result archiving fetches a filtered record set [and we're limited by WCG to 250], but if none of the records meet the conditions, Excel copies the whole filtered out set instead of copying nothing [skip on or present an error code to handle]. An additional [initial ugly code] test determines if filtered map results in zero, then continues with the next page [max 250 again], in the loop. Old days '>>', append additional input to existing text file, circa 1990 it must have been I last used that function, but nothing wrong with it :O) |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Today continuing on from adding hunting tool related standard office buttons that show as extra on the Quick Access Toolbar [QAT]. Created several 'custom' buttons to that bar [only showing when the tool is loaded]. One is a 'Home' button with the oh-so familiar browser icon, and assigned a shortcut key to this as well. Hit Ctrl+m and you get transported back to control central, setting the selector right below the buttons. [If you use the default assignment of Office to Ctrl+m, none far as I can find, you're SOOL].
... and a very very nasty bug erased 100+ and 99- gives a differential of 1, but really there's a 199 problem, which was the type of issue. And something got put in last night that got a this morning's 13000 record fetch down to under 2 minutes and recompute [Mr. Data could not keep up with this ;P]. Rather surprising considering we're here still on copper wired internet [and sat, because cable does not exist this side of the Alps. Think we're going to have a Hasta la Kebab meal (with spiced pila rice). |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Crisis what crisis? The testing data set expanded from < 600 results to 9000 overnight and Office did not like that [the way WCG offers a max of 250 per Results request]. Fortunately I knew where to look and hit Google 100% score on the top item, 5 minutes understanding, 30 minutes preparation of an input map into a for-next, discovering how many records to come [the API tells you on the first record received], and off to listobject sizing, this time the testing was only 5 minutes of the ~60 total time spent.
Only now noticed MSDN-Dev Center is using GitHub for development and documenting, same as Berkeley BOINC is kept on]. The development speedup actually came mostly from a code I wrote yesterday, automating the column sizing for 34 fields, which runs in less than a blink of an eye [A consistent presentation of the data view is always pleasing ;P]. Re-usable code is great... another module added to the Hunting too, v2.0o Happy birthday WCG, back to a run in 2 minutes, over copper telephone wire. ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Btw, moved development down to Office 2010. Upward compatibility is guaranteed, the reverse from 2013/2016 to 2010 was not, no matter how much effort [backward compatibility testing time].
And thanks goes out to thé member making the 'big' data available. |
||
|
|
![]() |