Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 
World Community Grid Forums
  Category: Support
  Forum: BOINC Agent Support
  Thread: Scheduler request failed: HTTP internal server error - [solved]

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 36
Posts: 36   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 4412 times and has 35 replies Next Thread
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi everybody,
I've updated the ca-bundle.crt file following Ageless's recommendation.
I rebooted the machine.
No change.
Everything seems to run OK excepted the reporting step and consequently the inability to fetch new work.
Now the machine runs dry.
It is a pity since the machine (8 threads) runs 24/7 and was considered until now as reliable.
Yves
----------------------------------------

Décrypthon team progress - KerSamson's contribution

----------------------------------------
[Edit 1 times, last edit by KerSamson at Jun 21, 2020 9:15:21 PM]
[Jun 21, 2020 9:06:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2172
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
This is what I found so far:
Lifewire says it is most definitely a server-side error.
Mundayweb states that these problems usually fix themselves in the end.
One post at CERN's LHC forum suggests to add some debug flags (Event Log Diagnostic Flags from the Advanced menu or by pressing CTRL+Shift+F) to try to find out what's going on.

The strange thing is that if only one user (in a wide audience) reports the problem, one might think that the problem must lie on the 'client' side, not on the server's side. However, now that there are two persons reporting the case, something might be going on …
----------------------------------------
[Edit 1 times, last edit by adriverhoef at Jun 22, 2020 12:12:44 AM]
[Jun 21, 2020 9:54:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi Adri,
I thank you for your input.
Investigating on my side, I come also to the conclusion that this is a "server's side error" what fits with the Error Code 500.
The strange thing is that if only one user (in a wide audience) reports the problem, one might think that the problem must lie on the 'client' side, not on the server's side.

At the same location, I have 6 machines - 2 recently installed RPi, 3 Linux, 1 Windows 7 - that all work very well. Suddenly, one machine - Linux Mint 18 - experiences reporting problem (upload still working) even the last update is already about 2 days old and no change has been applied on the machine).
There is no reasonable explanation!
If nothing happens during the next week, as soon as I will have a couple of free hours, I will probably consider to re-install the machine (I was actually waiting for the final release of Linux Mint 20).
I am very interested for some Tech Team's feedback (host name: enezsun; ID: 3876637).
Cheers,
Yves
---
PS: I am close to think that there is some database corruption at WCG's side.
----------------------------------------

Décrypthon team progress - KerSamson's contribution
[Jun 22, 2020 7:01:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Last chance, switch off the affected device, switch off all your network equipment between that device and the internet. Before, clear out DNS caches. Add WCG to the Hosts file with the real IP, so it will not go to their cloud, be it the device itself or your DNS server if you run one on your network. Who knows

Seem to recollect knreed left instructions where in the BOINC root or project data directory to change some www line too for someone to try a download and that seemed to work as well to overcome some science app element download.
[Jun 22, 2020 7:36:25 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi Lavaflow,
thank you for your advice. However the machine succeeds to contact WCG's servers.
2020-06-22 11:05:19	[http] [ID#0] Sent header to server: 	
2020-06-22 11:05:19	[http] [ID#1] Info:  TLSv1.2 (IN), TLS change cipher, Client hello (1):	
2020-06-22 11:05:19	[http] [ID#1] Info:  TLSv1.2 (IN), TLS handshake, Finished (20):	
2020-06-22 11:05:19	[http] [ID#1] Info:  SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384	
2020-06-22 11:05:19	[http] [ID#1] Info:  ALPN, server did not agree to a protocol	
2020-06-22 11:05:19	[http] [ID#1] Info:  Server certificate:	
2020-06-22 11:05:19	[http] [ID#1] Info:  	 subject: C=US; ST=New York; L=Armonk; O=International Business Machines Corporation; CN=*.worldcommunitygrid.org
2020-06-22 11:05:19	[http] [ID#1] Info:  	 start date: Apr 10 00:00:00 2018 GMT
2020-06-22 11:05:19	[http] [ID#1] Info:  	 expire date: Jul 13 00:00:00 2020 GMT
2020-06-22 11:05:19	[http] [ID#1] Info:  	 subjectAltName: scheduler.worldcommunitygrid.org matched
2020-06-22 11:05:19	[http] [ID#1] Info:  	 issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=Thawte TLS RSA CA G1
2020-06-22 11:05:19	[http] [ID#1] Info:  	 SSL certificate verify ok.
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: POST /boinc/wcg_cgi/fcgi HTTP/1.1	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Host: scheduler.worldcommunitygrid.org	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: User-Agent: BOINC client (x86_64-pc-linux-gnu 7.6.31)	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Accept: */*	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Accept-Language: en_US	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Content-Length: 374569	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: Expect: 100-continue	
2020-06-22 11:05:19	[http] [ID#1] Sent header to server: 	
2020-06-22 11:05:19	[http] [ID#1] Received header from server: HTTP/1.1 100 Continue	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: HTTP/1.1 200 OK	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Date: Mon, 22 Jun 2020 09:05:19 GMT	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Server: Apache	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: X-Powered-By: Servlet/3.1	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: ETag: 288203349	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: X-Content-Type-Options: nosniff	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: X-XSS-Protection: 1; mode=block	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Cache-Control: no-cache, no-store, max-age=0, must-revalidate	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Pragma: no-cache	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Expires: 0	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Strict-Transport-Security: max-age=31536000; includeSubDomains	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: X-Frame-Options: SAMEORIGIN	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Vary: Accept-Encoding	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Content-Encoding: gzip	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Referrer-Policy: no-referrer-when-downgrade	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.googletagmanager.com https://tagmanager.google.com https://d2bnxibecyz4h5.cloudfront.net https://www.google	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Content-Type: text/xml; charset=UTF-8	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Content-Language: en-US	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Set-Cookie: WCG-SESSION=00003DsPA0d7f9xD6K7xFtezLRj:-1; Expires=Mon, 22-Jun-20 09:35:18 GMT; Path=/; Domain=.worldcommunitygrid.org; Secure; HttpOnly	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: Transfer-Encoding: chunked	
2020-06-22 11:05:19	[http] [ID#0] Received header from server: 	
2020-06-22 11:05:19	[http] [ID#0] Info:  Connection #23 to host www.worldcommunitygrid.org left intact	
2020-06-22 11:05:19	[http] [ID#1] Info:  We are completely uploaded and fine	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: HTTP/1.1 500 Internal Server Error	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: Date: Mon, 22 Jun 2020 09:05:19 GMT	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: Server: Apache	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: Content-Length: 527	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: Content-Type: text/html; charset=iso-8859-1	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: 	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <html><head>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <title>500 Internal Server Error</title>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: </head><body>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <h1>Internal Server Error</h1>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <p>The server encountered an internal error or	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: misconfiguration and was unable to complete	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: your request.</p>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <p>Please contact the server administrator at 	
2020-06-22 11:05:21	[http] [ID#1] Received header from server:  root@localhost to inform them of the time this error occurred,	
2020-06-22 11:05:21	[http] [ID#1] Received header from server:  and the actions you performed just before this error.</p>	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: <p>More information about this error may be available	
2020-06-22 11:05:21	[http] [ID#1] Received header from server: in the server error log.</p>	
2020-06-22 11:05:21	[http] [ID#1] Info:  Connection #24 to host scheduler.worldcommunitygrid.org left intact	
2020-06-22 11:05:21	Scheduler request failed: HTTP internal server error

The certificate is accepted.
The communication between machine and server is operational.
The only problem I can identify could be located here.
2020-06-22 11:05:19	[http] [ID#1] Info:  TLSv1.2 (IN), TLS change cipher, Client hello (1):	
2020-06-22 11:05:19	[http] [ID#1] Info:  TLSv1.2 (IN), TLS handshake, Finished (20):	
2020-06-22 11:05:19	[http] [ID#1] Info:  SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384	
2020-06-22 11:05:19	[http] [ID#1] Info:  ALPN, server did not agree to a protocol	 
2020-06-22 11:05:19	[http] [ID#1] Info:  Server certificate:	
2020-06-22 11:05:19	[http] [ID#1] Info:  	 subject: C=US; ST=New York; L=Armonk; O=International Business Machines Corporation; CN=*.worldcommunitygrid.org
2020-06-22 11:05:19	[http] [ID#1] Info:  	 start date: Apr 10 00:00:00 2018 GMT
2020-06-22 11:05:19	[http] [ID#1] Info:  	 expire date: Jul 13 00:00:00 2020 GMT
2020-06-22 11:05:19	[http] [ID#1] Info:  	 subjectAltName: scheduler.worldcommunitygrid.org matched
2020-06-22 11:05:19	[http] [ID#1] Info:  	 issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=Thawte TLS RSA CA G1
2020-06-22 11:05:19	[http] [ID#1] Info:  	 SSL certificate verify ok.

ping scheduler.worldcommunitygrid.org provides the correct IP address, i.e. 169.47.63.74
Frankly I don't have any idea about what is incorrect on machine's side. It is very probably a problem at server's side only.
Cheers,
Yves
----------------------------------------

Décrypthon team progress - KerSamson's contribution
[Jun 22, 2020 9:37:01 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
@Tech Team: you can try to contact me directly by e-mail.
Yves
----------------------------------------

Décrypthon team progress - KerSamson's contribution
[Jun 22, 2020 9:40:55 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
My point is not the connecting but where it connects at WCG.

Found the post with all the cloud IP addresses from 2014, so no sure how current or stale that list is

https://www.worldcommunitygrid.org/forums/wcg/viewpostinthread?post=446102
[Jun 22, 2020 10:19:01 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2172
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi Yves,
You found this line:
ALPN, server did not agree to a protocol

ALPN explained:
What Is ALPN?

ALPN, or Application-Layer Protocol Negotiation, is a TLS extension that includes the protocol negotiation within the exchange of hello messages. ALPN is able to negotiate which protocol should be handled over a secure connection in a way that is more efficient and avoids additional round trips. The ever-growing in popularity HTTP/2 protocol, makes use of ALPN to further decrease website load times and encrypt connections faster.

I'm seeing that ALPN line here, too, after enabling the http_debug flag; boinc-client-7.16.6-3.fc31.x86_64 and no problems here.

Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: Trying 169.47.63.74:443...
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TCP_NODELAY set
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: Connected to www.worldcommunitygrid.org (169.47.63.74) port 443 (#16036)
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: ALPN, offering h2
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: ALPN, offering http/1.1
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: successfully set certificate verify locations:
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: CAfile: /etc/pki/tls/certs/ca-bundle.crt
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: CApath: none
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: TLSv1.2 (IN), TLS handshake, Finished (20):
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: ALPN, server did not agree to a protocol
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: Server certificate:
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: subject: C=US; ST=New York; L=Armonk; O=International Business Machines Corporation; CN=*.worldcommunitygrid.org
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: start date: Apr 10 00:00:00 2018 GMT
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: expire date: Jul 13 00:00:00 2020 GMT
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: subjectAltName: host "www.worldcommunitygrid.org" matched cert's "*.worldcommunitygrid.org"
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=Thawte TLS RSA CA G1
Mon 22 Jun 2020 13:13:27 CEST | | [http] [ID#0] Info: SSL certificate verify ok.
[Jun 22, 2020 11:28:54 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi Adri,
thank you very much for the feedback smile
Consequently the log entry is not the failure cause.
Cheers,
Yves
----------------------------------------

Décrypthon team progress - KerSamson's contribution
[Jun 22, 2020 11:34:27 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1679
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Scheduler request failed: HTTP internal server error
Hi Lavaflow,
I really appreciate your inputs and I was not aware of resp. I forgot Knreed's message. Even if I remember, this time when WCG started to use CDN.
However the problem is located in the communication with the scheduler (reporting/fetching).
Manual tests show that the machine can reach the scheduler. Only the dialogue between server and host is not running well ! ... thinking
Cheers,
Yves
----------------------------------------

Décrypthon team progress - KerSamson's contribution
[Jun 22, 2020 11:40:50 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 36   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread