Skip to content
Snippets Groups Projects
Commit 8faa1196 authored by Marco Malavolti's avatar Marco Malavolti
Browse files

Fixed bugs

parent a6650dd9
Branches
Tags
No related merge requests found
Hide whitespace changes
Inline Side-by-side
README.md
+ 10
− 4
View file @ 8faa1196
...@@ -43,7 +43,7 @@ The check executed by the service follows these steps: ...@@ -43,7 +43,7 @@ The check executed by the service follows these steps:
1. It retrieves the eduGAIN IdPs from eduGAIN Operator Team database via a JSON interface 1. It retrieves the eduGAIN IdPs from eduGAIN Operator Team database via a JSON interface
2. For each IdP that is was not manually disabled by the eduGAIN Operations Team, the check creates a Wayfless URL for each SP involved and retrieves the IdP login page. It expects to find the HTML form with a username and password field. Therefore, no complete login will happen at the Identity Provider because the check stops at the login page. 2. For each IdP that is was not manually disabled by the eduGAIN Operations Team, the check creates a Wayfless URL for each SP involved and retrieves the IdP login page. It expects to find the HTML form with a username and password field. Therefore, no complete login will happen at the Identity Provider because the check stops at the login page.
The SPs used for the check are "Test SP shib 2.4" (https://sp-demo.idem.garr.it/shibboleth) from IDEM GARR AAI and the "AAI Viewer Interfederation Test" (https://attribute-viewer.aai.switch.ch/interfederation-test/shibboleth) from SWITCHaai. These SPs might change in the future if needed. The SPs used for the check are "SP Demo provided by GARR" (https://sp-demo.idem.garr.it/shibboleth) from IDEM GARR AAI and the "AAI Viewer Interfederation Test" (https://attribute-viewer.aai.switch.ch/interfederation-test/shibboleth) from SWITCHaai. These SPs might change in the future if needed.
The SAML authenticatin request is not signed. Therefore, authentication request for any eduGAIN SP could be created because the SP's private key is not needed. The SAML authenticatin request is not signed. Therefore, authentication request for any eduGAIN SP could be created because the SP's private key is not needed.
# Limitations # Limitations
...@@ -70,9 +70,15 @@ The test eduGAIN Connectivity Check web pages is available at: https://technical ...@@ -70,9 +70,15 @@ The test eduGAIN Connectivity Check web pages is available at: https://technical
The tool uses following status for IdPs: The tool uses following status for IdPs:
* ERROR (red): * ERROR (red):
* The IdP's response contains an HTTP Error or the web page returned does not look like a login page. The most probable causes for this error are HTTP errors (e.g.: 404 error) * The IdP's response contains an HTTP Error or the web page returned does not look like a login page.
* The IdP most likely does not consume the eduGAIN metadata correctly or it hasn't does not return a web page that looks like a login form. A typical case that falls into this category is when an IdP returns a message "No return endpoint available for relying party" or "No metadata found for relying party". * **Invalid-Form**: considers those IdPs that do not load a standard username/password login page and do not return messages like "No return endpoint available for relying party" or "No metadata found for relying party".
* The IdP has a problem with its SSL certificate. * **Timeout**: considers those IdPs that do not load a standard username/password login page within 60 seconds.
* **Connection-Error**: considers those IdPs that are not reachable due to a connection problem. View the "Page Source" value to discover which problem the IdP has.
* The IdP most likely does not consume the eduGAIN metadata correctly.
A typical case that falls into this category is when an IdP returns a message "No return endpoint available for relying party" or "No metadata found for relying party":
* **No-eduGAIN-Metadata**
* The IdP has a problem with its SSL certificate:
* **SSL-Error**
* OK (green): * OK (green):
* The IdP most likely correctly consumes eduGAIN metadata and returns a valid login page. This is no guarantee that login on this IdP works for all eduGAIN services but if the check is passed for an IdP, this is probable. * The IdP most likely correctly consumes eduGAIN metadata and returns a valid login page. This is no guarantee that login on this IdP works for all eduGAIN services but if the check is passed for an IdP, this is probable.
* DISABLED (white) * DISABLED (white)
......
eccs2properties.py
+ 1
− 1
View file @ 8faa1196
...@@ -48,7 +48,7 @@ ECCS2SPS = [ ...@@ -48,7 +48,7 @@ ECCS2SPS = [
ROBOTS_USER_AGENT = "ECCS/2.0 (+https://technical-test.edugain.org/eccs2)" ROBOTS_USER_AGENT = "ECCS/2.0 (+https://technical-test.edugain.org/eccs2)"
# PATTERNS # PATTERNS
METADATAPATTERN = "Unable.to.locate(\sissuer.in|).metadata(\sfor|)|no.metadata.found|profile.is.not.configured.for.relying.party|Cannot.locate.entity|fail.to.load.unknown.provider|does.not.recognise.the.service|unable.to.load.provider|Nous.n'avons.pas.pu.(charg|charger).le.fournisseur.de service|Metadata.not.found|application.you.have.accessed.is.not.registered.for.use.with.this.service|Message.did.not.meet.security.requirements|Unsupported.Request|Not.Authorized" METADATAPATTERN = "Unable.to.locate(\sissuer.in|).metadata(\sfor|)|no.metadata.found|profile.is.not.configured.for.relying.party|Cannot.locate.entity|fail.to.load.unknown.provider|does.not.recognise.the.service|unable.to.load.provider|Nous.n'avons.pas.pu.(charg|charger).le.fournisseur.de service|Metadata.not.found|application.you.have.accessed.is.not.registered.for.use.with.this.service|Message.did.not.meet.security.requirements|Unsupported.Request|Not.Authorized|METADATANOTFOUND|Unknown.login.requester|is.unspecified.or.unsupported|Unknown.service.provider|Richiesta.non.supportata|Metadati.non.trovati|untrusted.provider|Unregistered.Service"
USERNAMEPATTERN = '<input[\s]+[^>]*((type=\s*[\'"](text|email)[\'"]|user)|(name=\s*[\'"](name)[\'"]))[^>]*>' USERNAMEPATTERN = '<input[\s]+[^>]*((type=\s*[\'"](text|email)[\'"]|user)|(name=\s*[\'"](name)[\'"]))[^>]*>'
PASSWORDPATTERN = '<input[\s]+[^>]*(type=\s*[\'"]password[\'"]|password)[^>]*>' PASSWORDPATTERN = '<input[\s]+[^>]*(type=\s*[\'"]password[\'"]|password)[^>]*>'
REFUSEDPATTERN = '(^http)(.*\.png$)|(.*\.css$)|(.*\.js$)|(.*\.gif$)|(.*\.svg$)|(.*\.jpg$)' REFUSEDPATTERN = '(^http)(.*\.png$)|(.*\.css$)|(.*\.js$)|(.*\.gif$)|(.*\.svg$)|(.*\.jpg$)'
......
runEccs2.py
+ 1
− 5
View file @ 8faa1196
...@@ -11,9 +11,6 @@ import utils ...@@ -11,9 +11,6 @@ import utils
from subprocess import PIPE from subprocess import PIPE
#from utils import get_list_feds, get_list_eccs_idps, get_reg_auth_dict, get_idp_list, gen_output
#from eccs2properties import ECCS2FAILEDCMD, ECCS2FAILEDCMDIDP, ECCS2STDOUT, ECCS2STDERR, ECCS2STDOUTIDP, ECCS2STDERRIDP, ECCS2DIR, ECCS2NUMPROCESSES, ECCS2LISTIDPSURL, ECCS2LISTIDPSFILE, ECCS2LISTFEDSURL, ECCS2LISTFEDSFILE, ECCS2OUTPUTDIR, ECCS2RESULTSLOG, ECCS2AUXDIR
# Run Command # Run Command
# https://docs.python.org/3/library/asyncio-queue.html#examples # https://docs.python.org/3/library/asyncio-queue.html#examples
# https://docs.python.org/3/library/asyncio-subprocess.html # https://docs.python.org/3/library/asyncio-subprocess.html
...@@ -31,7 +28,7 @@ async def run(name,queue,stdout_file,stderr_file,cmd_file): ...@@ -31,7 +28,7 @@ async def run(name,queue,stdout_file,stderr_file,cmd_file):
stdout, stderr = await proc.communicate() stdout, stderr = await proc.communicate()
print(f'[{name} exited with {proc.returncode}] - {cmd!r}') #print(f'[{name} exited with {proc.returncode}] - {cmd!r}')
if stdout: if stdout:
stdout_file.write(f'-----\n[cmd]\n{cmd}\n\n[stdout]\n{stdout.decode()}') stdout_file.write(f'-----\n[cmd]\n{cmd}\n\n[stdout]\n{stdout.decode()}')
...@@ -151,5 +148,4 @@ if __name__=="__main__": ...@@ -151,5 +148,4 @@ if __name__=="__main__":
cmd_file.close() cmd_file.close()
end = time.time() end = time.time()
#utils.gen_output(e2p.ECCS2AUXDIR,f"{e2p.ECCS2OUTPUTDIR}/{e2p.ECCS2RESULTSLOG}")
print("Time taken in hh:mm:ss - ", str(datetime.timedelta(seconds=end - start))) print("Time taken in hh:mm:ss - ", str(datetime.timedelta(seconds=end - start)))
utils.py
+ 18
− 9
View file @ 8faa1196
...@@ -138,6 +138,7 @@ def get_idp_contacts(idp,contactType): ...@@ -138,6 +138,7 @@ def get_idp_contacts(idp,contactType):
ctcList.append('missing email') ctcList.append('missing email')
return ctcList return ctcList
# Write the login page source code into its file # Write the login page source code into its file
def store_page_source(page_source,idp,sp,test): def store_page_source(page_source,idp,sp,test):
if (test): if (test):
...@@ -145,13 +146,14 @@ def store_page_source(page_source,idp,sp,test): ...@@ -145,13 +146,14 @@ def store_page_source(page_source,idp,sp,test):
return True return True
else: else:
# Put the page_source into an appropriate HTML file # Put the page_source into an appropriate HTML file
with open(f"{e2p.ECCS2HTMLDIR}/{e2p.DAY}/{sha1(['entityID'])}---{get_label(sp)}.html","w") as html: with open(f"{e2p.ECCS2HTMLDIR}/{e2p.DAY}/{sha1(idp['entityID'])}---{get_label(sp)}.html","w") as html:
try: try:
html.write(page_source) html.write(page_source)
return True return True
except IOError: except IOError:
return False return False
# Get the Google Chrom Selenium Driver # Get the Google Chrom Selenium Driver
def get_driver_selenium(idp=None,sp=None,debugSelenium=False): def get_driver_selenium(idp=None,sp=None,debugSelenium=False):
...@@ -188,6 +190,7 @@ def get_driver_selenium(idp=None,sp=None,debugSelenium=False): ...@@ -188,6 +190,7 @@ def get_driver_selenium(idp=None,sp=None,debugSelenium=False):
driver = webdriver.Chrome(e2p.PATHCHROMEDRIVER, options=chrome_options) driver = webdriver.Chrome(e2p.PATHCHROMEDRIVER, options=chrome_options)
return driver return driver
# ECCS2 Check made by Selenium
def check_idp_response_selenium(sp,idp,test): def check_idp_response_selenium(sp,idp,test):
# Disable SSL requests warning messages # Disable SSL requests warning messages
...@@ -230,13 +233,13 @@ def check_idp_response_selenium(sp,idp,test): ...@@ -230,13 +233,13 @@ def check_idp_response_selenium(sp,idp,test):
# Catch SSL Exceptions and block the ECCS check # Catch SSL Exceptions and block the ECCS check
except requests.exceptions.SSLError as e: except requests.exceptions.SSLError as e:
if (test): page_source = f"\nAn SSL Error occurred while opening https://{fqdn_idp}/robots.txt.\n\n{e}\n\nCheck it on SSL Labs: https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}" if (test): page_source = f"\nAn SSL Error occurred while opening https://{fqdn_idp}/robots.txt:\n\n{e}\n\nCheck it on SSL Labs: https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}"
else: page_source = f"<h1>SSL ERROR</h1><h2>An SSL error occurred for the server {fqdn_idp}:</h2><p>{e}</p><p>Check it on SSL Labs: <a href='https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}'>Click Here</a></p>" else: page_source = f"<h1>SSL ERROR</h1><h2>An SSL error occurred for the server {fqdn_idp}:</h2><p>{e}</p><p>Check it on SSL Labs: <a href='https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}'>Click Here</a></p>"
store_page_source(page_source,idp,sp,test) store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","SSL-Error",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","SSL-Error",webdriver_error)
except requests.exceptions.ConnectionError as e: except requests.exceptions.ConnectionError as e:
if (test): page_source = f"\nA Connection error occurred while opening https://{fqdn_idp}/robots.txt.\n\n{e}" if (test): page_source = f"\nA Connection error occurred while opening https://{fqdn_idp}/robots.txt:\n\n{e}"
else: page_source = f"<h1>CONNECTION ERROR:</h1><h2>A Connection error occurred while opening <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>" else: page_source = f"<h1>CONNECTION ERROR:</h1><h2>A Connection error occurred while opening <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>"
store_page_source(page_source,idp,sp,test) store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
...@@ -248,7 +251,7 @@ def check_idp_response_selenium(sp,idp,test): ...@@ -248,7 +251,7 @@ def check_idp_response_selenium(sp,idp,test):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
except requests.exceptions.TooManyRedirects as e: except requests.exceptions.TooManyRedirects as e:
if (test): page_source = f"\nToo many redirects occurred while opening: https://{fqdn_idp}/robots.txt.\n\n{e}" if (test): page_source = f"\nToo many redirects occurred while opening: https://{fqdn_idp}/robots.txt:\n\n{e}"
else: page_source = f"<h1>TOO MANY REDIRECTS</h1><h2>Too many redirects occurred while opening: <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>" else: page_source = f"<h1>TOO MANY REDIRECTS</h1><h2>Too many redirects occurred while opening: <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>"
store_page_source(page_source,idp,sp,test) store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
...@@ -299,21 +302,27 @@ def check_idp_response_selenium(sp,idp,test): ...@@ -299,21 +302,27 @@ def check_idp_response_selenium(sp,idp,test):
stored = store_page_source(pgsrc,idp,sp,test) stored = store_page_source(pgsrc,idp,sp,test)
if (stored): if (stored):
return (idp['entityID'],wayfless_url,check_time,http_code,"No-eduGAIN-Metadata",webdriver_error) return (idp['entityID'],wayfless_url,check_time,http_code,"No-eduGAIN-Metadata",webdriver_error)
else: elif(page_source == "<html><head></head><body></body></html>"):
if (test): pgsrc = f"Timeout: No access form found in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds" if (test): pgsrc = f"Timeout: No valid login form loaded in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds"
else: pgsrc = page_source else: pgsrc = page_source
stored = store_page_source(pgsrc,idp,sp,test) stored = store_page_source(pgsrc,idp,sp,test)
if (stored): if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Timeout",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","Timeout",webdriver_error)
else:
if (test): pgsrc = f"Invalid-Form: No valid login form found in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds"
else: pgsrc = page_source
stored = store_page_source(pgsrc,idp,sp,test)
if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Invalid-Form",webdriver_error)
except WebDriverException as e: except WebDriverException as e:
error = e.__dict__['msg'].split('(')[0].rstrip() error = e.__dict__['msg'].split('(')[0].rstrip()
if (test): pgsrc = f"\nERROR: {error}" if (test): pgsrc = f"\nA Connection error occurred while opening {wayfless_url}:\n\n{error}"
else: pgsrc = f"<h1>ECCS2 CHECK FAILED</h1><h2>The IdP Login failed the check due the following error:</h2><p>{error}</p>" else: pgsrc = f"<h1>CONNECTION ERROR</h1><h2>A Connection error occurred while opening <a href='{wayfless_url}'>{wayfless_url}</a>:</h2><p>{error}</p>"
webdriver_error = 1 webdriver_error = 1
stored = store_page_source(pgsrc,idp,sp,test) stored = store_page_source(pgsrc,idp,sp,test)
if (stored): if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","ERROR",webdriver_error) return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
finally: finally:
driver.quit() driver.quit()
......
web/eccs2.js
+ 6
− 6
View file @ 8faa1196
...@@ -193,10 +193,10 @@ function getCheckResult(checkResult){ ...@@ -193,10 +193,10 @@ function getCheckResult(checkResult){
return '<div class="tooltip">OK <span class="tooltiptext tooltip-top tooltip-ok">The IdP is consuming correctly the eduGAIN metadata and return a valid login page</span></div> '+infoCircle; return '<div class="tooltip">OK <span class="tooltiptext tooltip-top tooltip-ok">The IdP is consuming correctly the eduGAIN metadata and return a valid login page</span></div> '+infoCircle;
} }
else if (checkResult == "Timeout"){ else if (checkResult == "Timeout"){
return '<div class="tooltip">Timeout <span class="tooltiptext tooltip-top tooltip-timeout">The IdP does not load a valid login page within 60 seconds</span></div> '+infoCircle; return '<div class="tooltip">Timeout <span class="tooltiptext tooltip-top tooltip-timeout">The IdP does not load a valid login page within 30 seconds</span></div> '+infoCircle;
} }
else if (checkResult == "Invalid-Form"){ else if (checkResult == "Connection-Error"){
return '<div class="tooltip">Invalid-Form <span class="tooltiptext tooltip-top tooltip-invalid-form">The IdP does not load a valid login page</span></div> '+infoCircle; return '<div class="tooltip">Connection Error <span class="tooltiptext tooltip-top tooltip-invalid-form">Check failed due a connection error</span></div> '+infoCircle;
} }
else if (checkResult == "No-eduGAIN-Metadata"){ else if (checkResult == "No-eduGAIN-Metadata"){
return '<div class="tooltip">No-eduGAIN-Metadata <span class="tooltiptext tooltip-top tooltip-no-edugain-metadata">The IdP is not consuming correctly edugGAIN metadata stream</span></div> '+infoCircle return '<div class="tooltip">No-eduGAIN-Metadata <span class="tooltiptext tooltip-top tooltip-no-edugain-metadata">The IdP is not consuming correctly edugGAIN metadata stream</span></div> '+infoCircle
...@@ -229,7 +229,7 @@ function format ( d ) { ...@@ -229,7 +229,7 @@ function format ( d ) {
'<td>'+d.contacts.support+'</td>'+ '<td>'+d.contacts.support+'</td>'+
'<td class="strong">Check Time</td>'+ '<td class="strong">Check Time</td>'+
'<td class="strong">Check Result</td>'+ '<td class="strong">Check Result</td>'+
'<td class="strong">HTTP Code</td>'+ //'<td class="strong">HTTP Code</td>'+
'<td class="strong">Page Source</td>'+ '<td class="strong">Page Source</td>'+
'<td class="strong">Retry Check</td>'+ '<td class="strong">Retry Check</td>'+
'</tr>'+ '</tr>'+
...@@ -238,7 +238,7 @@ function format ( d ) { ...@@ -238,7 +238,7 @@ function format ( d ) {
'<td>https://'+getHostname(d.sp1.wayflessUrl)+'</td>'+ '<td>https://'+getHostname(d.sp1.wayflessUrl)+'</td>'+
'<td>'+d.sp1.checkTime+'</td>'+ '<td>'+d.sp1.checkTime+'</td>'+
'<td>'+getCheckResult(d.sp1.checkResult)+'</td>'+ '<td>'+getCheckResult(d.sp1.checkResult)+'</td>'+
'<td>'+d.sp1.httpCode+'</td>'+ //'<td>'+d.sp1.httpCode+'</td>'+
'<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp1.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+ '<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp1.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+
'<td><a href="'+d.sp1.wayflessUrl+'" target="_blank">Click to retry</a></td>'+ '<td><a href="'+d.sp1.wayflessUrl+'" target="_blank">Click to retry</a></td>'+
'</tr>'+ '</tr>'+
...@@ -247,7 +247,7 @@ function format ( d ) { ...@@ -247,7 +247,7 @@ function format ( d ) {
'<td>https://'+getHostname(d.sp2.wayflessUrl)+'</td>'+ '<td>https://'+getHostname(d.sp2.wayflessUrl)+'</td>'+
'<td>'+d.sp2.checkTime+'</td>'+ '<td>'+d.sp2.checkTime+'</td>'+
'<td>'+getCheckResult(d.sp2.checkResult)+'</td>'+ '<td>'+getCheckResult(d.sp2.checkResult)+'</td>'+
'<td>'+d.sp2.httpCode+'</td>'+ //'<td>'+d.sp2.httpCode+'</td>'+
'<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp2.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+ '<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp2.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+
'<td><a href="'+d.sp2.wayflessUrl+'" target="_blank">Click to retry</a></td>'+ '<td><a href="'+d.sp2.wayflessUrl+'" target="_blank">Click to retry</a></td>'+
'</tr>'+ '</tr>'+
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment