This page describes how to run conformance tests and gather testing results for Financial-grade API (FAPI) Client Initiated Backchannel Authentication (CIBA) OpenID Providers (OPs).
The parts of FAPI-CIBA that are not currently tested are described on our wiki: https://gitlab.com/openid/conformance-suite/wikis/OP-FAPI-CIBA-Test-Status
If your server complies with the UK OpenBanking specification, you should run only the ‘FAPI-RW-ID2: Authorization server test’ and in the ‘variant’ menu select the appropriate ‘openbankinguk-‘ one(s) for the client authentication types supported. These allow you to demonstrate compliance with both FAPI-RW and the OpenBanking UK Profile of FAPI-RW. The ‘openbankinguk’ variants are a superset of the FAPI-RW-ID2 tests, with the major difference being that OB compliant authorization servers requiring pre-registration of intent using an OB specific API for an authorization to succeed.
Register two clients in your authorisation server and the necessary keys/certificates. Two clients are necessary as the suite tests various mixup attacks, for example, using a valid access token from one client with the TLS client certificate from the other client.
For ‘ping’ mode tests, the notification URL for the clients must be set to:
ALIAS must be replaced with a string that is unique to you/your organisation, to avoid other conformance suite users affecting your testing.
As per the FAPI-CIBA specification, all certification requests must include results for poll mode. Ping mode is optional to support.
The IP address the conformance suite will make outgoing calls from is available within the suite on the home page after login in. (The IP address may change from time to time as the conformance suite runs on a Kubernetes cluster.)
Instructions for Running Tests
- Open https://www.certification.openid.net/.
- Login using a Google or GitLab account, or any Op that supports WebFinger.
- Ensure “Multi test plan” is selected. (If “Single test” is shown, toggle it to “Multi test plan”).
- Select the relevant test plan from the “Select a Test Plan” dropdown menu. The tests you should run depends on your client authentication type(s). For example, on a FAPI-CIBA compliant server that supports oauth-mtls client authentication and the ‘poll’ mode you should run
FAPI-CIBA: test planand select variant
- DO NOT pick a test plan that has ‘client’ in the name. These are not applicable when testing an OpenID Provider.
- Fill in the configuration form (as per the guidance given in the form fields when empty). You can switch to the “JSON” tab to view/edit the configuration in the underlying JSON format. Changes to the form are automatically reflected in the JSON and vice versa. The JSON can be copied and saved locally to be pasted back in later. The server will automatically remember the most recent JSON you successfully created a test with.
- Note that both client JWKS sets must contain an ‘alg’ field. For FAPI-CIBA this is most usually ‘PS256’, but ‘ES256’ is also permitted.
- A resource server URL must be provided. This must be a simple GET endpoint that returns JSON and is used to test certificate binding of access tokens etc. (Different in OpenBanking UK tests; see below.)
- institution_id: any value is okay and is sent in a x-fapi-financial-id header. (This is a hangover from FAPI-R-ID1 and is only retained due to an error in an OpenBanking specific specification.)
- Press “Launch Test Plan”. You will be taken to a list of all the test modules in the plan.
- Press “Run New Test” on the first test module.
- Please read the description of the test in the light blue box near the top of the log page – this may contain specific instructions, for example one test requires that the user rejects the authentication process.
- If applicable, the test will ask you to approve (or deny or ignore) the request on the relevant authentication device.
- When the test has completed, press “Continue Plan” to start the next test, or “Return to Plan” to view your progress.
If you require support, please email firstname.lastname@example.org. If it relates to a test failure, please include a link to the relevant log-detail.html, or if using a local install the downloaded log file.
About the Conformance Suite
For more information, please see about the conformance suite.
OpenBanking UK Specific Notes
The conformance suite can also run tests against implementations and live deployments compliant to the FAPI-aligned OpenBanking UK security standard. These tests are a superset of the FAPI-CIBA tests, and compliance with the FAPI-OB test profile also demonstrates compliance to the FAPI-RW profile. Note that currently these tests are not ready to run as OpenBanking has not yet finalised the mechanism for passing the Intent ID in a FAPI-CIBA compliant manner.
The main differences are:
- OpenBanking UK authorization servers require an Intent ID to be created prior to authorisation using an access token obtained with the client credentials grant, which is then passed to the back channel authorization endpoint
- OpenBanking UK defines ecosystem-specific ACR values.
The process of creating clients for the Conformance Suite is akin to on-boarding a Third Party, except that the suite and the credentials issued remain in the control of the ASPSP. The client credentials should be revoked after testing. The suite only requires AISP permission, as only this protocol is accessed during the flow, and only the ReadAccountsBasic permission is used. The suite does not require any significant customer data to be available; a dummy / reference account can be used.
When configuring the suite, the resource server url should be set to the URL for the AISP endpoints, for example: