This page describes how to run OpenID Connect conformance tests and gather testing results for Relying Parties (RPs). These instructions are for the new Java-based system (the previous python system used for OpenID Connect certifications was retired in August 2020, as described at Migration of OpenID Connect Certification Software.).

Overview

When you are testing a relying party, you configure it to use the conformance suite as the identity provider (so for example, if your relying party usually sends users to Google to login, you would instead send the user to the conformance suite). The conformance suite essentially provides a “fake” OpenID Provider / OAuth2 Authorization Server.

When you start a test, you will see the ‘issuer’ and ‘discoveryurl’ in the ‘exported values’ section (or if you select not to use OpenID Discovery you would need to manually configure the authorization endpoint etc) – you should use one of these values in the configuration of your relying party where the identity provider to be used is configured. The test will show as ‘WAITING’ status and in most cases the next step is to start the login flow in your relying party.

Available Certification Profiles

The first step in using the Relying Party tests is to decide which conformance profiles you plan to test.
Conformance profile plan names are self explanatory and contain the protocol, profile and target audience name, for example “OpenID Connect Core: Basic Certification Profile Relying Party Tests” or “OpenID Connect Core: Hybrid Certification Profile Relying Party Tests”.

Please note that plans marked as “(not currently part of certification program)” cannot be used for certification, but they can be used for testing use cases that are not required for certification. These plans provide more configuration options than certification profiles and allow users to test more scenarios.

For example, you can test your implementation using tls_client_auth client authentication type which is not in scope for certification by selecting the plan named “OpenID Connect Core: Relying party tests (not currently part of certification program)” from the “Test Plan” dropdown.

If you have problems starting testing, you can send questions to certification@oidf.org.

Client Profiles

You can run the conformance test plans for Basic, Implicit and Hybrid profiles, depending on which response_type values your implementation supports.

It is also possible to test the same profiles using form_post response mode by selecting “Form Post” versions of the profiles.

Logout Profiles

Session management and logout testing profiles are also available for RPs. You can run them by selecting a plan under “Test a Relying Party / OAuth2 Client Logout Support” group in the “Test Plan” dropdown.

Logout profiles require users to test using all supported response types; therefore Basic, Hybrid and Implicit plans are provided for logout profiles as well.

Configuring Tests

Open https://www.certification.openid.net/. Login using a Google or GitLab account, or any OpenID Connect OP that supports WebFinger.

First press “Create a new test plan”, then select the test plan for the profile you want to test. Make sure to select a test plan in one of “Test a Relying Party / OAuth2 Client” or “Test a Relying Party / OAuth2 Client Logout Support” sections of the menu.

Request Type Selection:

You need to select “Request Type” which can be one of “plain_http_request”, “request_object” or “request_uri”. All tests will be run using the selected request type (except tests which always use a specific request type regardless of selected request type, such as tests for testing request_uri support). You can certify using any request type.

Client Registration Type:

You can use dynamic client registration or a configure static client.

Dynamic Clients

When using dynamic client registration, registered clients will not be stored permanently and they will be available only for the duration of the test; they will become unusable as soon as the test ends. You don’t need to do anything to unregister clients.

You can register clients with any supported algorithm to test using specific signing and encryption algorithms, e.g to use PS256 as the id_token_signed_response_alg value. Please note that certification profiles contain tests that always use RS256, regardless of the algorithms requested during client registration.

Static Clients

When using static clients, additional form fields will be displayed automatically. But a form field will not be available for every possible client metadata. You can use the “JSON” tab to override client metadata fields for which a form field is not available.

For example you can use the “JSON” configuration tab to use specific signing and encryption algorithms.

Test Plan Configuration Fields

Additional configuration fields may be shown/hidden based on selected configuration options.

By default, “alias”, “description” and “publish” fields are displayed for all test plans.

You must select an “ALIAS” to use. This will form part of the redirect uri and should be unique to yourself, for example your company name. (If you use the same alias as another user, your tests will interfere with each other.)

As described above in “Static clients” section, you can use the JSON configuration tab for advanced configuration.

Running Tests

Once you have created your test plan, you should run each test in the test plan. Please be sure to pay attention to the instructions in the blue box at the top of each test.

Certification of a profile requires that you have ‘PASSED’, ‘REVIEW’, ‘WARNING’ or ‘SKIPPED’ results for all tests in the profile. You cannot certify with any FAILED or INTERRUPTED results for the profile. In some cases, a screen capture is uploaded as evidence of passing the test.

Tests typically expect a certain sequence of requests (e.g discovery, authentication, token and userinfo requests) and are marked as finished when the expected sequence of requests are received.

Some tests, tests intended to test a negative case expected to result in an error on the client side, wait for a few seconds before they transition into FINISHED state. This timeout is 5 seconds by default and can be modified using the “waitTimeoutSeconds” advanced configuration option. Please wait for the test to transition into FINISHED state before proceeding with the next test.

Once you have finished testing, proceed to Submission of Results for RPs. Any questions can be sent to certification@oidf.org.