The purpose of the eduGAIN Connectivity Check is to identify eduGAIN Identity Providers (IdP) that are not properly configured. In particular it checks if an IdP properly loads and consumes SAML2 metadata which contains the eduGAIN Service Providers (SP). The check results are published on the public eduGAIN Connectivity Check web page ([https://technical.edugain.org/eccs](https://technical.edugain.org/eccs)). The main purpose is to increase the service overall quality and user experience of the eduGAIN interfederation service by making Federation and Identity Provider operators aware of configuration problems.
The check is performed by sending a SAML authentication request to each eduGAIN IdP and then follow the various HTTP redirects. The expected result is a login form that allows users to authenticate (typically with username/password) or an error message of some form. For those Identity Providers that output an error message, it can be assumed that they don't consume eduGAIN metadata properly or that they suffer from another configuration problem. There are some cases where the check will generate false positives, therefore IdPs can be excluded from checks as is described below.
The check is performed by sending a SAML authentication request to each eduGAIN IdP and then follow the various HTTP redirects until the user login form. The expected result is a login form that allows users to authenticate themselves (typically with username/password) or an error message of some form. For those Identity Providers that return an error message, it can be assumed that they don't consume eduGAIN metadata properly or that they suffer from another configuration problem. There are some cases where the check will generate false positives, therefore IdPs can be excluded from checks as is described below.
The Identity Providers are checked once per day. Therefore, the login requests should not have any significant effect on the log entries/statistics of an Identity Provider. Also, no actual login is performed because the check cannot authenticate users due to missing username and password for the IdPs. Only Identity Providers are checked but not the Service Providers.
...
...
@@ -44,19 +44,19 @@ The check follows the steps:
1. It retrieves the eduGAIN IdPs from eduGAIN Operator Team database via a JSON interface
2. For each IdP, that hasn't been disabled manually by the eduGAIN Operations Team or dynamically by `robots.txt` (explained below) and that has a valid SSL certificate on its HTTP-Redirect Location, it performs an IdP-initiated SSO with SAML Authentication Request for two SP belonging two different NREN, members of eduGAIN interfederation, and for one fake SP. It expects to find the HTML form with username and password fields on the "good" SPs and an error or other on the "bad" SP. If an IdP uses frames on its Login page, the check follows only the first one on each redirected pages. If an IdP uses HTTP Basic Authentication, the check searches '401 Unauthorized' string into the web page content presented and establish the correct behaviour of the IdP. Therefore, no complete login will happen at the Identity Provider because the check stops at the login page or at SSL Certificate validation.
The SAML authentication request is not signed. Therefore, an authentication request for any eduGAIN SP could be created because the SP's private key is not needed.
The SPs HTTP-Post Assertion Consumer Service URLs used by the check are retrieved by `sps-metadata.xml`into the "input" directory. The 'validation' method used to validate the "sps-metadata.xml" is a deployer decision, but a solution is provided on the `README-SPS-METADATA.md` file.
2. For each IdP, that hasn't been disabled manually by the eduGAIN Operations Team or dynamically by `robots.txt` (explained below) and that has a valid SSL certificate on its HTTP-Redirect Location, it performs an IdP-initiated SSO with SAML Authentication Request for two SP belonging two different NREN, members of eduGAIN interfederation, and for another randomnly generated fake SP. It expects to find the HTML form with username and password fields for the NREN SPs and an error or other result for the fake one. If an IdP uses frames on the Login page, the check follows only the first one on each nested pages. If an IdP uses HTTP Basic Authentication, the check searches '401 Unauthorized' string into the web page content returned or 401 HTTP Status Code from the request. Therefore, no complete login will happen at the Identity Provider because the check stops at the login page.
The SAML Authentication Request is not signed. Therefore, an authentication request for any eduGAIN SP could be created because the SP's private key is not needed.
The SPs HTTP-Post Assertion Consumer Service URLs used by the check are retrieved by `sps-metadata.xml`frile from the "input" directory. The 'validation' method used to validate the "sps-metadata.xml" is a deployer decision, but a solution is provided on the `README-SPS-METADATA.md` file.
3. If the check fails for an IdP the first time, it will be checked again at the end of the execution for a second time before exit.
3. If the check fails for an IdP the first time, a second attempt will be done at the end of all other checked IdP, before exit.
4.It keeps the results of the last 7 days, but it can be increased by the deployers.
4.The results are kept for the last 7 days, but the deployers can increase it as they wish.
## Limitations
There are some situations where the check cannot work reliably. In those cases it is possible to disable the check for a particular IdP. The so far known cases where the check might generate a false negative are:
* IdP does not support HTTP or HTTPS with at least SSLv3 or TLS1 or newer (these IdPs are insecure anyway)
* IdP does not support HTTPS with at least SSLv3 or TLS1 or newer (these IdPs are insecure anyway)
* IdP is part of a Hub & Spoke federation (some of them manually have to first approve eduGAIN SPs)
* IdP does not use web-based login form (e.g. Account Chooser Authentication or X.509 login)
...
...
@@ -69,7 +69,7 @@ User-agent: ECCS
Disallow: /
```
If an IdP is not able to create its own `robots.txt`, it can be disabled by an eduGAIN Operation Team member by setting the dictionary `IDPS_DISABLED_DICT` into `eccs_properties.py` with a line in the form:
If an IdP is not able to create its own `robots.txt`, it can be disabled by an eduGAIN Operation Team member by setting the dictionary `IDPS_DISABLED_DICT` into `eccs_properties.py` with a line like:
`<idp-entity-id>':'<eccs-check-disabling-reason>`
...
...
@@ -79,24 +79,26 @@ If an IdP is not able to create its own `robots.txt`, it can be disabled by an e
The tool uses following status for IdPs:
* ERROR (red):
* The IdP's response contains an HTTP Error or the web page returned does not look like a login page.
***Unable-To-Check**: considers those IdPs that do not load a standard username/password login page and do not return messages like "*No return endpoint available for relying party*" or "*No metadata found for relying party"*.
* The IdP's response contains an error or the web page is not returned due a Timeout, Connection or IdP Generic error.
***Timeout**: considers those IdPs that do not load a standard username/password login page within 60 seconds.
***Connection-Error**: considers those IdPs that are not reachable due to a connection problem. View the "Page Source" value to discover which problem the IdP has.
***IdP-Error**: considers those IdPs that the web page returned does not contain a Login Form and reports an unspecified error such as "*An error occured*". This has been seen on Micrsoft ADFS based IdPs
***Connection-Error**: considers those IdPs that are not reachable due to a connection problem. View the "Page Source" content to discover which problem has the IdP.
***IdP-Generic-Error**: considers those IdPs that the returned web page does not contain a Login Form, but an unspecified error such as "*An error occured*". This kind of error has been seen on Micrsoft ADFS based IdPs.
* The IdP most likely does not consume the eduGAIN metadata correctly.
A typical case that falls into this category is when an IdP returns a message "*No return endpoint available for relying party*" or "*No metadata found for relying party*":
***No-eduGAIN-Metadata**
* The IdP has a problem with its SSL certificate:
***No-SP-Metadata-Error**: considers those IdPs that returns a message like "*No return endpoint available for relying party*" or "*No metadata found for relying party*" instead of the Login Page.
* The IdP has an SSL problem on the HTTP-Redirect Location used by the check:
***SSL-Error**
* OK (green):
* The IdP most likely correctly consumes eduGAIN metadata and returns a valid login page. This is no guarantee that login on this IdP works for all eduGAIN services but if the check is passed for an IdP, this is probable.
* The IdP most likely correctly consumes eduGAIN metadata and returns a valid username/password login page. This is no guarantee that login on this IdP works for all eduGAIN services but if the check is passed for an IdP, this is probable.
* UNKNOWN (grey):
* The IdP can't be checked because the returned Login Page content is not recognized or the Login Page is always returned, also for the fake SP.
***Unable-To-Check**: considers those IdPs that do not load a standard username/password login page and do not return messages like "*No return endpoint available for relying party*" or "*No metadata found for relying party"*.
* DISABLED (white)
* The IdP is excluded because it cannot be checked reliably. The "*Page Source*" column, when an entity is disabled, shows the reason of the disabling.
* The IdP is excluded because it cannot be checked reliably. The "*Page Source*" column content, when an entity is disabled, shows the reason of the disabling.
***Disabled**: considers those IdPs that are disabled from the check by an eduGAIN Operator or "robots.txt" file.
## Requirements Hardware
* OS: Debian 9, CentOS 7.8 (tested)
* OS: Debian 11, CentOS 7.8 (tested)
* HDD: 10 GB
* RAM: 4 GB
* CPU: >= 2 vCPU (suggested)
...
...
@@ -106,7 +108,8 @@ The tool uses following status for IdPs:
* Apache Server + WSGI
* Python 3 (tested with v3.9.1, v3.10.4)
* Selenium + Google Chrome Web Brower (tested with v91.0.4472.164, v100.0.4896.127)
* Selenium (tested with v4.1.3)
* Google Chrome Web Brower (tested with v91.0.4472.164, v100.0.4896.127, v101.0.4951.64)
* Chromedriver (tested with v91.0.4472.101, v100.0.4896.60)
* Git
* PHP
...
...
@@ -272,8 +275,8 @@ After the initial download, it is recommended that you occasionally go through t
The script takes about 2 hours to check 4666 IDPs, so its execution is suggested in the early morning,before the users start using the tool.
The `eccs-cron.log` file will contains:
* The execution time of the entire ECCS script
* Each failed IdP checked again and its result
* The result of the entire ECCS script executed
* Each failed IdP checked again and their results
* The result of the entire ECCS script
### Execute
...
...
@@ -358,23 +361,24 @@ To perform a restart after an API change use the following command:
* 'OK'
* 'ERROR'
* 'DISABLED'
* 'UNKNOWN'
*`check_result=`
*`OK`
*`Timeout`
*`Connection-Error`
*`IdP-Error`
*`No-eduGAIN-Metadata`
*`IdP-Generic-Error`
*`No-SP-Metadata-Error`
*`SSL-Error`
*`Unable-To-Check`
*`DISABLED`
*`Disabled`
*`reg_auth=https://reg.auth.example.org` (select a specific Registration Authority)
*`format=simple` (retrieve results in a simple format)
