Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 116
Posts: 116   Pages: 12   [ Previous Page | 3 4 5 6 7 8 9 10 11 12 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 15783 times and has 115 replies Next Thread
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

Building on the previous implementation of 'ludicrous speed', had a serious Oink moment just before reaching home, as the predicted rain started to come down... the objective IS to fetch the raw data ASAP after period end at 12:06:07 and 00:06:07, so fetching 250 records, then populating the map formula [250x12], then the next 250, populating the formula and so on and up to 540 times in the loop, was not a good idea, so moved the formula populating to AFTER the result fetches complete, and we have 'ludicrous speed ^ 2'

devilish
[Apr 6, 2017 4:48:06 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

"To Case or not to Case" was the follow on question and after sorting/tallying millions of results, it came out that the order in which to test worked out to be [for the particular mix of results]
190     Select Case valst 'Case may be unsorted, numerical and string!
Case 1: Statuses = "Valid"
Case 0: Statuses = "Pending Validation"
Case 4: Statuses = "Pending Verification"
Case 2: Statuses = "Invalid"
Case 3: Statuses = "Too Many Errors on Wingmen"
Case 5: Statuses = "Too Late"
195 End Select
Hardcoded now, as why look something up that has not changed in the past 15 years, or however long BOINC has been with the crunching world (bar that Case 4, Pending Verification, originally was 'Inconclusive'). At any rate, the number of Valid (1) on the result status pages far outstrip PV (0+4), making this the top item to show in the Result 'Status' field, sort of a present ratio of 94835 : 3635 ATM, or IOW, test the case and most of the times the prog can stop looking at Case 1 and not waste nanoseconds on first passing by Case 0. If that is repeated many many, many many times, the nanoseconds become a serious peak-shaving event... when you're waiting for it :D

Have a nice day

(The technique is only build in the 'biggest' version of the hunting tool... the small and medium account size versions just do not benefit in any form or fashion noticeable... no need to spend time on porting back ;)
[Apr 9, 2017 4:09:39 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

Another of the highly coincidental stumbles... buttons and shapes can be linked to computed fields, but Excel being infamous of NOT WYSIWYG, would at random times not show on shapes, buttons, graphs etc what's really momentarily there... what they would call 'artifacting'... what was there but no longer is laughing . The stumble today was injecting the computed result as text, so instead of the simple

=ButtonTxt0

coupled to the shape in the worksheet, you get

785 sALL.Shapes("Button3").TextFrame.Characters.Text = [ButtonTxt0].Value2

in VBA and low and behold, suddenly the info button shows the latest, always, I seem to observe, every time the script has run updating the Result Status information pull, to show now

Results Status
84 832 Returned, 345 pgs
4 161 Cached, 16 pgs

Yes, internationalized, or francophoned if you will, the thousand separator is a space... so that reads eightyfourthousandeighthundredandthirtytwo smug

Have a nice day.

(now another fix is open... 4 161 divided by 250 is 17 (roundup)... the do-while cycles until the remaining fetch is less than 250, the max the techs allow to retrieve in a single call, the counter needs a late +1 it appears, cased of course)
----------------------------------------
[Edit 1 times, last edit by SekeRob* at Apr 16, 2017 9:14:30 AM]
[Apr 16, 2017 9:12:09 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

This morning was merge day, fruit way up in the tree, but did it pay off to make the climb... Various complex helper field formulas previously converted to UDFs, integrated, courtesy of VBA, into a single UDF that produces the result mask, the 6 bit string such as 111111, which means no bad exit code, outcome ok, credit granted, validation state great, modtime in present stats period and not yet on archive, as when it is, the newly fetched result record can be ignored, which is a mask of 111100... easy peesy. Sum of the 3 UDFs into a single is 25% less code, 4 less If/Case tests, 4 less fields to fetch and up to 2x134,000 life results to replicate the UDFs to, and compute on days the FAHV shorts are flowing heavy. All in a matter of hours pensing on the how to (climb the tree safely) and 45 minutes coding.

Made my day.
Have a good one kiss
[Apr 20, 2017 9:26:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

Ouch, ouch, ouch (shame on me). All the Case-ing conversions and here I forget a test in one UDF function, if the Status array is already loaded, to test a different array dependency presence, since I hardcoded that particular one, and 4 tables being reloaded again and again and again, for up to that known max number of 134,000 times. That really hurt, both on recalculation and on workbook saving time (when the whole assembly gets fullcalcrebuild prior to). The StopWatch procedure build into many routines, to know how long steps take, gave instant feedback

CalcFull start 21-4-2017 12:05 end 21-4-2017 12:19 0:13:57 previous 0:17:19 Calc Rebuild

This is 4 minutes 38 seconds off, or 26.7% less... And this is with only 17K results live, and a just yesterday shrunk archive from 942K to 153K.

Made my day (though it's pouring outside opposed to Weatherunderground prediction sad )
[Apr 21, 2017 10:34:26 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

While cycling, and let WCG be WCG until the last mile, which encompasses a half mile stretch at 18% (18 meters up per 100 meters, to get that pulse racing) the final switch was decided on... no more formula storing, just the output and we have ludicrous speed power 2 with turbo... the yellow with previous runtime on right and new runtime at left, and this is with 143*250 current live results and 177K prior period results on active archive.



Most shocking was the save file time got slashed by a factor 10, the resultonly pull and calc full halved again.

Surprisingly the actual saved file size did not shrink... seems Excel is pretty good in compressing series of identical [complex] formula by keeping one and then tokenize, which is the piece that makes saving and reopening slow.

Time for some vino rosso, doc.

saluti tutti

btw, this is the final. done, dusted, 3.0K build 33-7 cowboy
[Apr 22, 2017 7:22:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
widdershins
Veteran Cruncher
Scotland
Joined: Apr 30, 2007
Post Count: 674
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

Hi Sek,

If you're finished with the tweaks (will you ever be truly finished??) could you send me details of how to obtain it please?

Thanks

Widdershins
----------------------------------------
[Edit 1 times, last edit by widdershins at Apr 25, 2017 7:17:08 PM]
[Apr 25, 2017 7:16:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

For laughs, sometime on entering your account there was Log In and when leaving Sign Out, and was testing on first scrape for Sign Out left top, yes we still prior century scrape for lack of xml stats pages, but Bill visited and changed one of the 2 states, so now it is Log In and Log Out. If the test was <> Sign Out, the Log In procedure was run and the page was scraped a second time... so that started failing "As it so happened", and round and round and round it went endlessly scraping. sad

Another discovery, Avast is rejecting security of the WCG site, because...



Under Firefox, that's not a problem, there's just the yellow padlock notice, but under IE it is, and MS requires Office to use IE. Think I asked about secure linking of signature picks and what not, as they can harbor iffyness, but think the question was not understood. sad
[Apr 26, 2017 2:21:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

In anticipation, in view of the transfer to WCG's home cloud on May 15, subject to change of course, this will need hunting tool users to hit F5 (Goto) and enter ActDayUnix, to then modify the statistics beginning date/time to e.g.

17-5-2017 00:00:00 which will then run stats through 12:00:00

Should the grid come back sooner, then 16-5-2017 12:00:00 needs entering in subject field, so all is computed through 17-5-2017 00:00:00

yes dd-mm-yyyy is the format, the year 4 digits.

As far as the tool is concerned all results returned/validated will then be slotted in a single 12 hour period... not at all aware if the first post cut-over stats will be soon after or much belated i.e. first let the reporting tsunami subside before commencement.

Of course WCG runs by the Meet and Exceed customer expectations principle, so it is sooner, rather than later anticipated whistling

happy landing.
----------------------------------------
[Edit 1 times, last edit by SekeRob* at Apr 28, 2017 1:51:25 PM]
[Apr 28, 2017 1:47:42 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K

The bike's front suspension gave up, a Suntur they frown upon at the bike garage, so flipped the dial from open to lock, and there I had not only a hard tail, but a hard top as well :O)))

While rattling back the last third, some gray cells woke up and produced some synaptic action of the oh-yes output type on late validations, our good techs deem to still needing to go into the last stats period, like 15 results this time, and never seen 'that' late...

Last ModTime
17-04-29 12:06:13
17-04-29 12:06:14
17-04-29 12:06:14
17-04-29 12:06:13
17-04-29 12:06:13
17-04-29 12:06:14
17-04-29 12:06:14
17-04-29 12:06:15
17-04-29 12:06:13
17-04-29 12:06:15
17-04-29 12:06:15
17-04-29 12:06:13
17-04-29 12:06:15
17-04-29 12:06:15

worth -15 0:02:51:30 WCG ran late

The cut-off officially was at 4/29/17 12:06:03 (UTC) [2 hour(s) ago, and sure enough, just needed to extend the chronological modtime selection with these and bingo. Now, when these are post-period identified, usually fully automatic on post-archive run, they are also moved to archive and would have disappeared in the pile, currently some 200,000 results, that 'oh yes, why not?' event came up with an additional code line to have these late validations suffixed in the Archive Notes column with "Missed", and sure enough, from Total Recall to Total Recon, and back... simple db filter for last period with a 'missed' note, and we have the 15 broken back for reminiscence. dancing

Whilst on a short roll, churned out a half dozen more pop-up comments for the users who wonder 'what's that?', along the line of 'It's this, but DON'T do that then and then!' and away we are.

Time for late late pranzo. chicken
----------------------------------------
[Edit 1 times, last edit by SekeRob* at Apr 29, 2017 2:32:11 PM]
[Apr 29, 2017 2:30:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 116   Pages: 12   [ Previous Page | 3 4 5 6 7 8 9 10 11 12 | Next Page ]
[ Jump to Last Post ]
Post new Thread