Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 116
Posts: 116   Pages: 12   [ Previous Page | 3 4 5 6 7 8 9 10 11 12 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 15661 times and has 115 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.11

Ohlahlah, had to emergency implement some code, JIC, as it does happen that users do not have any completed result validating in a 12 hour stats period, at which point Step II and III of period end would not executed... why would they if there's no results to count sad

And since it's a topic of discussion, the umpteened time, why WCG can't set up merging the same way ONLY OF THE STATS like other BOINC projects [with the standard checks for clearly same device with install over install], continues to be a riddle (Techs, time to dump that UD era shell!), am designing, in me head, how to roll up the stats of devices with same name, user definable, meaning users tag all 'same devices' with the same alias, with some buts as Androids get this truncated name on the device stats history to all be called Android (a long standing puzle why that's not being fixed to show the same reference as is used on the Result Status pages). I'll find a bandaid for that idiosyncrasy too... I will, because I can biggrin

TTYL
----------------------------------------
[Edit 2 times, last edit by Former Member at Jun 1, 2017 12:14:10 PM]
[Jun 1, 2017 12:11:37 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.11

Fatto: Borrowed a bit of code here, borrowed a bit of code there (inside the hunting tool code library) and we have an automated table of 'unique' device names merging the whole host's history of contributed time, results and points. There's though a but... the device history proves, again, to not add up. Not sure how far the problem goes back, but suspect it's going back to the UD days.

Anyway, if a user manages to yet again create a new device install, the DevMerge tab will not hesitate to roll up all previous device contributions by the same host name and plant the total under the new device registration ID. By and large, maintenance free, lest a user goes wild and add many... then at this time, the user needs to extend the table to make it fit. (Yes Hands On wink )
[Jun 1, 2017 9:28:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.11

This last, hot hour, succumbed to stepping out of another international agreement, and injected into the web-scrape routine near start that ',' is decimal separator and '.' is thousands separator, to then on seconds thoughts afterwards quickly reverse that cranial shortout, and put

.DecimalSeparator = .International(xlDecimalSeparator) 'add 04.06.2017
.ThousandsSeparator = .International(xlThousandsSeparator) 'add 04.06.2017

back, so everyone can look at the results in their own homey number format style.

Yesterday added a few new custom functions

=SecondsTime(cellref)
=DigitalTime(cellref)
=GetDevId(cellref)

Extremely simple to get times converted from 1:23:45:56 notation into seconds and seconds back into the more human comprehendable 1:23:45:56 and to extract the numeric and unique Device ID out of the URLs on the various, in html, scraped pages. No matter how often the same device name is used, the device id will, without fail, connect the latest stats and roll them up for all with the same device name... got mydesktoppowerhouse1 10 times installed... no problem, not to speak of the multiple 'Android' featuring on the device statistics history whistling
[Jun 4, 2017 1:11:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

It is "Make Europe Greater Then The First", when I'm done, and I am on https://www.theguardian.com/news/datablog/201...-the-month-before-the-day, the scraped Statistics History page date column in the tool looks like below, where tool version actually tells you what the date format is in, the original isn't cool

Statistics Date
(dd/mm/yyyy)
06/06/2017
05/06/2017
04/06/2017
03/06/2017
02/06/2017
01/06/2017
31/05/2017
30/05/2017
29/05/2017
28/05/2017
27/05/2017
26/05/2017
25/05/2017
24/05/2017
23/05/2017
22/05/2017
21/05/2017
20/05/2017
19/05/2017
18/05/2017
17/05/2017
16/05/2017
15/05/2017
14/05/2017
13/05/2017
12/05/2017
11/05/2017
10/05/2017
09/05/2017
08/05/2017

Get used to it tongue
[Jun 6, 2017 11:35:12 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

One of the few active testers reported on a bug in Step II, the archiving of the last period results, then trying to run Step III, Update Histories (of the project contributions), which tests if step II ran and if not do it on the go [Yes/No?] msgbox, but was not so sure it did. To fill that 'where am I?' feeling, the tool's center main screen will show a new Icon, an orange tick mark with a II, indicating that archiving did complete. smile



There's no Step III completed icon, yet, as this step opens up the new period when done i.e. if the blue IP icon shows up, the project level histories were updated successfully (something WCG is not offering ;P). A little overkill, as when history was written correctly, the accompanying new period record opens with 0:00:00:00 runtime and the comment 'Prd in Progress'. biggrin

No, the tool is still not done, the export of the project histories is still on the most wanted list... just in case the tool crashes and a restore is needed to a previous copy. blushing
[Jun 8, 2017 2:40:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

There's no Step III completed icon, yet

There is, there is, see above post with updated pic... converted yet again a complex nested formula into a UDF which made the IF layers so much easier to analyze and organize, and voila, another ElseIf condition test permitted to determine the post step 3 state and provide positive feedback the period end process has completed. Just run a Result Status pages API pull and away u are into the next In Progress cycle.

HtH, while pausing on the way to Altes Blockhaus ๐Ÿ™‹๐Ÿ
[Jun 10, 2017 10:55:43 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

The close shave, Mach 5 blade, vibrating type, batteries included, determining how much space is needed after the first fetch pass (ResultsAvailable field), then size the output table in one go to that number instead of sizing up by 250 records max per fetch call. 55 seconds instead of 66. Courtesy of the techs who've as yet not resumed deleting old validated records, my account grew to 5500 results, a new personal record biggrin , 23 pages, so can test a full pass reasonably quickly. Network suspended on the client, so we have a good static set to evenly compare from run to run (with a stable ISP at this time :0).

dancing
[Jun 29, 2017 12:13:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

You may have noticed that ResultsAvailable might change during a fetch run. As an example, today I have seen an increment of 25 results during a fetch run (with my own script 'wcgresults'). Therefore it's possible that you will 'miss' a fetch if a boundary (of size 250) is exceeded.
Nevertheless it's not that big a deal โ€” my script 'wcgresults' will fetch any forgotten results in the next run (not after waiting too long sleep, of course).
[Jun 29, 2017 1:15:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

Please, Pleased Me, the last control run, full complement at the bottom

Per 62500 results, went from 74 minutes at start to 14:15 with the last re-ordering tweaks.

Boils down to 1 octo and 1800 active results, serverstate 5, done in under half a minute, computed at all, while crunching full out.

More dancing

whistling
[Jul 2, 2017 8:56:24 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Hunting Tool 3.0K build 33.14

We can all agree that 62500 is 250 squared, that's 250 fetches in a fetchrun. If a single fetch (plus check if not present and store) would last 2.5 seconds, a fetchrun of 250 (fetches and stores) would take 250x2.5 = 25x25 = 625 seconds (ยฑ10,5 minutes).

62500 results ... with only one octo and tasks that last a mere 2 minutes it would take the octo 62500/4 = 15625 minutes = 10 days, 20 hours and 25 minutes to crunch them all.
[Jul 2, 2017 12:15:46 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 116   Pages: 12   [ Previous Page | 3 4 5 6 7 8 9 10 11 12 | Next Page ]
[ Jump to Last Post ]
Post new Thread