Submission of Certification Results for OpenID Connect RPs

This page describes how to run conformance tests and gather testing results for Financial-grade API (FAPI) Client Initiated Backchannel Authentication (CIBA) OpenID Providers (OPs).

The parts of FAPI-CIBA that are not currently tested are described on our wiki: https://gitlab.com/openid/conformance-suite/wikis/OP-FAPI-CIBA-Test-Status

If your server complies with the UK OpenBanking specification, you should run only the ‘FAPI-RW-ID2: Authorization server test’ and in the ‘variant’ menu select the appropriate ‘openbankinguk-‘ one(s) for the client authentication types supported. These allow you to demonstrate compliance with both FAPI-RW and the OpenBanking UK Profile of FAPI-RW. The ‘openbankinguk’ variants are a superset of the FAPI-RW-ID2 tests, with the major difference being that OB compliant authorization servers requiring pre-registration of intent using an OB specific API for an authorization to succeed.

Setup Steps

Register two clients in your authorisation server and the necessary keys/certificates. Two clients are necessary as the suite tests various mixup attacks, for example, using a valid access token from one client with the TLS client certificate from the other client.

For ‘ping’ mode tests, the notification URL for the clients must be set to:

  • https://www.certification.openid.net/test/a/ALIAS/ciba-notification-endpoint

ALIAS must be replaced with a string that is unique to you/your organisation, to avoid other conformance suite users affecting your testing.

As per the FAPI-CIBA specification, all certification requests must include results for poll mode. Ping mode is optional to support.

The IP address the conformance suite will make outgoing calls from is available within the suite on the home page after login in. (The IP address may change from time to time as the conformance suite runs on a Kubernetes cluster.)

Instructions for Running Tests

  1. Open https://www.certification.openid.net/.
  2. Login using a Google or GitLab account, or any Op that supports WebFinger.
  3. Ensure “Multi test plan” is selected. (If “Single test” is shown, toggle it to “Multi test plan”).
  4. Select the relevant test plan from the “Select a Test Plan” dropdown menu. The tests you should run depends on your client authentication type(s). For example, on a FAPI-CIBA compliant server that supports oauth-mtls client authentication and the ‘poll’ mode you should run FAPI-CIBA: test plan and select variant poll-mtls.
  5. DO NOT pick a test plan that has ‘client’ in the name. These are not applicable when testing an OpenID Provider.
  6. Fill in the configuration form (as per the guidance given in the form fields when empty). You can switch to the “JSON” tab to view/edit the configuration in the underlying JSON format. Changes to the form are automatically reflected in the JSON and vice versa. The JSON can be copied and saved locally to be pasted back in later. The server will automatically remember the most recent JSON you successfully created a test with.
  7. Note that both client JWKS sets must contain an ‘alg’ field. For FAPI-CIBA this is most usually ‘PS256’, but ‘ES256’ is also permitted.
  8. A resource server URL must be provided. This must be a simple GET endpoint that returns JSON and is used to test certificate binding of access tokens etc. (Different in OpenBanking UK tests; see below.)
  9. institution_id: any value is okay and is sent in a x-fapi-financial-id header. (This is a hangover from FAPI-R-ID1 and is only retained due to an error in an OpenBanking specific specification.)
  10. Press “Launch Test Plan”. You will be taken to a list of all the test modules in the plan.
  11. Press “Run New Test” on the first test module.
  12. Please read the description of the test in the light blue box near the top of the log page – this may contain specific instructions, for example one test requires that the user rejects the authentication process.
  13. If applicable, the test will ask you to approve (or deny or ignore) the request on the relevant authentication device.
  14. When the test has completed, press “Continue Plan” to start the next test, or “Return to Plan” to view your progress.

If you require support, please email certification@oidf.org. If it relates to a test failure, please include a link to the relevant log-detail.html, or if using a local install the downloaded log file.

Once you have successfully completed testing, please follow the submission instructions to complete the certification process.

Dynamic client registration

The tests support dynamic client registration, but in the case of MTLS client authentication they only support the token_endpoint_auth_method value self_signed_tls_client_auth and create a self-signed TLS certificate on the fly. The dynamic client registration was added to support requirements of OpenID Foundation’s internal CI system, and is not within the scope of the certification program (tests run using dynamic client registration are accepted for certification purposes, but the certification is the same as that given for systems that use static client registration).

Automated testing

As there is no web based Authorization Endpoint, testing cannot be automated using the suite’s built in selenium browser automation. To allow the FAPI-CIBA tests to be run in a CI (continuous integration) system, a “automated approval url” can be configured in the JSON configuration, for example:

"automated_ciba_approval_url": "https://cibasim.authlete.com/api/authenticate/actionize?workspace=authlete/fapidev&action={action}&token={auth_req_id}"

(There’s no configuration form box for this, you do it in the JSON tab.)

{auth_req_id} will be automatically substituted for the current auth_req_id by the conformance suite, {action} will be allow or deny depending on the test.

This url will be called once the suite wants the authentication to complete, so if you can implement a suitable endpoint you don’t need any UI at all for the authentication device, this is how we run these tests in our own CI.

This may also be used when creating a certification submission, however if your server accepts the binding message in the “request-with-potentially-bad-binding-message” test you are required to provide a screenshot of the correct display of that message, which cannot be automated. This test must be run manually instead; to do so this segment must be added to the JSON config to remove the automated approval url for that test, for example:

    "override": {
        "fapi-ciba-ensure-authorization-request-with-potentially-bad-binding-message": {
            "automated_ciba_approval_url": ""
        }
    }

About the Conformance Suite

For more information, please see about the conformance suite.

OpenBanking UK Specific Notes

The conformance suite can also run tests against implementations and live deployments compliant to the FAPI-aligned OpenBanking UK security standard. These tests are a superset of the FAPI-CIBA tests, and compliance with the FAPI-OB test profile also demonstrates compliance to the FAPI-RW profile. Note that currently these tests are not ready to run as OpenBanking has not yet finalised the mechanism for passing the Intent ID in a FAPI-CIBA compliant manner.

The main differences are:

  1. OpenBanking UK authorization servers require an Intent ID to be created prior to authorisation using an access token obtained with the client credentials grant, which is then passed to the back channel authorization endpoint
  2. OpenBanking UK defines ecosystem-specific ACR values.

The process of creating clients for the Conformance Suite is akin to on-boarding a Third Party, except that the suite and the credentials issued remain in the control of the ASPSP. The client credentials should be revoked after testing. The suite only requires AISP permission, as only this protocol is accessed during the flow, and only the ReadAccountsBasic permission is used. The suite does not require any significant customer data to be available; a dummy / reference account can be used.

When configuring the suite, the resource server url should be set to the URL for the AISP endpoints, for example:
https://rs.example.com/open-banking/v3.1/aisp/.