Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
![]() |
World Community Grid Forums
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 116
|
![]() |
Author |
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Ohlahlah, had to emergency implement some code, JIC, as it does happen that users do not have any completed result validating in a 12 hour stats period, at which point Step II and III of period end would not executed... why would they if there's no results to count
----------------------------------------![]() And since it's a topic of discussion, the umpteened time, why WCG can't set up merging the same way ONLY OF THE STATS like other BOINC projects [with the standard checks for clearly same device with install over install], continues to be a riddle (Techs, time to dump that UD era shell!), am designing, in me head, how to roll up the stats of devices with same name, user definable, meaning users tag all 'same devices' with the same alias, with some buts as Androids get this truncated name on the device stats history to all be called Android (a long standing puzle why that's not being fixed to show the same reference as is used on the Result Status pages). I'll find a bandaid for that idiosyncrasy too... I will, because I can ![]() TTYL [Edit 2 times, last edit by Former Member at Jun 1, 2017 12:14:10 PM] |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Fatto: Borrowed a bit of code here, borrowed a bit of code there (inside the hunting tool code library) and we have an automated table of 'unique' device names merging the whole host's history of contributed time, results and points. There's though a but... the device history proves, again, to not add up. Not sure how far the problem goes back, but suspect it's going back to the UD days.
Anyway, if a user manages to yet again create a new device install, the DevMerge tab will not hesitate to roll up all previous device contributions by the same host name and plant the total under the new device registration ID. By and large, maintenance free, lest a user goes wild and add many... then at this time, the user needs to extend the table to make it fit. (Yes Hands On ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
This last, hot hour, succumbed to stepping out of another international agreement, and injected into the web-scrape routine near start that ',' is decimal separator and '.' is thousands separator, to then on seconds thoughts afterwards quickly reverse that cranial shortout, and put
.DecimalSeparator = .International(xlDecimalSeparator) 'add 04.06.2017 .ThousandsSeparator = .International(xlThousandsSeparator) 'add 04.06.2017 back, so everyone can look at the results in their own homey number format style. Yesterday added a few new custom functions =SecondsTime(cellref) =DigitalTime(cellref) =GetDevId(cellref) Extremely simple to get times converted from 1:23:45:56 notation into seconds and seconds back into the more human comprehendable 1:23:45:56 and to extract the numeric and unique Device ID out of the URLs on the various, in html, scraped pages. No matter how often the same device name is used, the device id will, without fail, connect the latest stats and roll them up for all with the same device name... got mydesktoppowerhouse1 10 times installed... no problem, not to speak of the multiple 'Android' featuring on the device statistics history ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
It is "Make Europe Greater Then The First", when I'm done, and I am on https://www.theguardian.com/news/datablog/201...-the-month-before-the-day, the scraped Statistics History page date column in the tool looks like below, where tool version actually tells you what the date format is in, the original isn't
![]() Statistics Date (dd/mm/yyyy) 06/06/2017 05/06/2017 04/06/2017 03/06/2017 02/06/2017 01/06/2017 31/05/2017 30/05/2017 29/05/2017 28/05/2017 27/05/2017 26/05/2017 25/05/2017 24/05/2017 23/05/2017 22/05/2017 21/05/2017 20/05/2017 19/05/2017 18/05/2017 17/05/2017 16/05/2017 15/05/2017 14/05/2017 13/05/2017 12/05/2017 11/05/2017 10/05/2017 09/05/2017 08/05/2017 Get used to it ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
One of the few active testers reported on a bug in Step II, the archiving of the last period results, then trying to run Step III, Update Histories (of the project contributions), which tests if step II ran and if not do it on the go [Yes/No?] msgbox, but was not so sure it did. To fill that 'where am I?' feeling, the tool's center main screen will show a new Icon, an orange tick mark with a II, indicating that archiving did complete.
![]() ![]() There's no Step III completed icon, yet, as this step opens up the new period when done i.e. if the blue IP icon shows up, the project level histories were updated successfully (something WCG is not offering ;P). A little overkill, as when history was written correctly, the accompanying new period record opens with 0:00:00:00 runtime and the comment 'Prd in Progress'. ![]() No, the tool is still not done, the export of the project histories is still on the most wanted list... just in case the tool crashes and a restore is needed to a previous copy. ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
There's no Step III completed icon, yet There is, there is, see above post with updated pic... converted yet again a complex nested formula into a UDF which made the IF layers so much easier to analyze and organize, and voila, another ElseIf condition test permitted to determine the post step 3 state and provide positive feedback the period end process has completed. Just run a Result Status pages API pull and away u are into the next In Progress cycle. HtH, while pausing on the way to Altes Blockhaus ๐๐ |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
The close shave, Mach 5 blade, vibrating type, batteries included, determining how much space is needed after the first fetch pass (ResultsAvailable field), then size the output table in one go to that number instead of sizing up by 250 records max per fetch call. 55 seconds instead of 66. Courtesy of the techs who've as yet not resumed deleting old validated records, my account grew to 5500 results, a new personal record
![]() ![]() |
||
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2153 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
You may have noticed that ResultsAvailable might change during a fetch run. As an example, today I have seen an increment of 25 results during a fetch run (with my own script 'wcgresults'). Therefore it's possible that you will 'miss' a fetch if a boundary (of size 250) is exceeded.
Nevertheless it's not that big a deal โ my script 'wcgresults' will fetch any forgotten results in the next run (not after waiting too long ![]() |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Please, Pleased Me, the last control run, full complement at the bottom
![]() Per 62500 results, went from 74 minutes at start to 14:15 with the last re-ordering tweaks. Boils down to 1 octo and 1800 active results, serverstate 5, done in under half a minute, computed at all, while crunching full out. More ![]() ![]() |
||
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2153 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
We can all agree that 62500 is 250 squared, that's 250 fetches in a fetchrun. If a single fetch (plus check if not present and store) would last 2.5 seconds, a fetchrun of 250 (fetches and stores) would take 250x2.5 = 25x25 = 625 seconds (ยฑ10,5 minutes).
62500 results ... with only one octo and tasks that last a mere 2 minutes it would take the octo 62500/4 = 15625 minutes = 10 days, 20 hours and 25 minutes to crunch them all. |
||
|
|
![]() |