| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: Unplanned website outage 2017-06-25. |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 265
|
|
| Author |
|
|
Zanth
Advanced Cruncher USA Joined: Aug 18, 2008 Post Count: 88 Status: Offline Project Badges: 
|
Yuck, no fun. Thanks for the updates and best of luck getting it all going again!
---------------------------------------- |
||
|
|
DL120
Cruncher Joined: Apr 21, 2008 Post Count: 3 Status: Offline Project Badges: 
|
still cannot upload data........
---------------------------------------- |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I would imagine that once the servers in the cluster are activated, there will be a lot of filesystem checking and resyncing going on prior to mounting. Depending on the size of the filesystems, could take hours. Hopefully, it won't require a restore....
|
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
still cannot upload data........ when he rushed to the 'virtual' office to fix the issue, he forgot the main item in his toolkit... the 25" power-wrench.  (got nearly 400 results to upload and report) |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1957 Status: Offline Project Badges: 
|
I would imagine that once the servers in the cluster are activated, there will be a lot of filesystem checking and resyncing going on prior to mounting. Depending on the size of the filesystems, could take hours. Actually, this should be happening "on the fly", no need to wait for a "checking and resync", that's the whole idea of having a cloud based storage cluster, running a distributed file system like ZFS.To have a whole cluster failing, that is far from common, be it on-premise or "in the cloud". There must be a bigger underlying issue at hand here, and that is most likely the reason why things are taking that long. Ralf  |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12439 Status: Offline Project Badges:
|
Bear in mind that once everything is working properly, they will have a backlog of, say, 24 hours of uploads, reporting and downloads, so don't expect the catch-up to be immediate.
Unless they normally have a lot of excess capacity the catch-up could take as long as the outage. Mike |
||
|
|
KLiK
Master Cruncher Croatia Joined: Nov 13, 2006 Post Count: 3108 Status: Offline Project Badges:
|
32.546.247 beers later...yes, it's still out!
----------------------------------------Let's open another one... ;) |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges:
|
To have a whole cluster failing, that is far from common, be it on-premise or "in the cloud". There must be a bigger underlying issue at hand here, and that is most likely the reason why things are taking that long. It must have been a cluster bomb. |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1957 Status: Offline Project Badges:
|
To have a whole cluster failing, that is far from common, be it on-premise or "in the cloud". There must be a bigger underlying issue at hand here, and that is most likely the reason why things are taking that long. It must have been a cluster bomb. Though not that funny once you had to deal with the effects/remnants of a real cluster bomb... (Been there, removed them, Cambodia 1992).  Ralf |
||
|
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 2987 Status: Offline Project Badges: 
|
Boy, this is certainly taking some sorting out - just goes to show, that not all clouds are "fluffy white, nimbus" affairs - but, on the odd occasion, "dark, menacing thunder clouds with the odd occasional flash of lightening and hailstones".
----------------------------------------I just hope that WCG are throwing all they possibly can to get this resolved - and it's not just Keith who's having to 'unpick the mess'. What's certain, is that this demonstrates that it's not "all rosy in the cloud" and that bad things can (and, unfortunately, do) happen there also.  |
||
|
|
|