This page describes how to run conformance tests and gather testing results for OpenID Providers (OPs).
Establishing Your Testing Configuration
The first step in using the OpenID Provider tests is to decide which conformance profiles you plan to test. Then you establish a testing configuration on the host op.certification.openid.net. You can do this in a self-service manner at https://op.certification.openid.net:60000/. If you have problems establishing your configuration, you can send questions to firstname.lastname@example.org.
Most OPs will test the Basic OP profile and many will also test the Implicit OP and Hybrid OP profiles. You can switch between the response_type values that these profiles use dynamically while testing.
Most OPs will also test the Config OP profile, which tests your discovery information published at your .well-known/openid-configuration endpoint. You can’t switch whether to test this while testing, so you need to decide whether to do this up front.
Some OPs will also test the Dynamic OP profile, which tests your support for Dynamic Client Registration and related features. You can’t switch whether to test this while testing, so you need to decide whether to do this up front.
Most OPs will support signing, but not unsigned ID Tokens, or encryption. OPs supporting only response_type=code (the Basic OP profile) can choose not to support signing and only “sign” ID Tokens with “none”. Crypto decisions can be changed dynamically while testing.
NEW! Some OPs will choose to test the Form Post Response Mode with the Form Post OP profile. These tests can be added by checking a box at configuration time or by dynamically changing your configuration later.
You need to provide these test configuration parameters as well: login_hint – the username you’ll use to log into your OP, ui_locales – suggested value “fr en”, claims_locales – suggested value “fr en”, acr_values – suggested value “2 1”, WebFinger URL – the path for an ID on your OP – if applicable, WebFinger email – suggested value same as login_hint.
Completing your configuration will give you a testing site using a port allocated for your use on op.certification.openid.net. Your testing site will be at a URL like https://op.certification.openid.net:61234/, but with a different port number. Finally, before testing, you will have had to add the redirect_uri value https://op.certification.openid.net:61234/authz_cb to your application, and in addition, if you are running the Form Post OP tests, you will have also had to add the redirect_uri value https://op.certification.openid.net:61234/authz_post to your application (again, with the actual port number allocated for the configuration).
Running Tests and Viewing Results
Go to your testing site, which will be at a URL like https://op.certification.openid.net:61234/, but with a different port number.
Run tests by clicking on the black circle next to the test name. Successful tests will turn the circle green. Warnings will turn it yellow and failures will turn it red. You can view the log for a test that has been run by clicking on the (i) button next to a completed test.
You can view logs from all the tests you have run by adding the path /log to your testing site URL. This path would be https://op.certification.openid.net:61234/log, but with a different port.
Some profiles require you to run tests with multiple response_type values. The Implicit OP profile requires you to test both
id_token+token. The Hybrid OP profile requires you to test three combinations. (Basic OP only requires you to test
response_type=code.) The Form Post OP profile requires you to run tests for all the response_type values supported by your implementation.
You can change which response_type you are testing by clicking the link to change it near the beginning of your testing page.
Certification of a profile requires that you have Success or Warning results for all tests in the profile. You cannot certify with any Failed results for the profile. Note that some tests where errors are supposed to occur will not create a log file showing a complete result – such as the OP-redirect_uri-NotReg (Sent redirect_uri does not match a registered redirect_uri) test, in which success consists of an error being shown in the browser. In these cases, a screen capture of the error is used as evidence of passing the test.
Third Party-Initiated Login OP Profile
Third Party-Iniated Login OP tests are currently available in the pilot phase. The two Third Party-Iniated Login OP tests are enabled by selecting support for “extra” tests and “dynamic client registration” in the OP test instance setup. They are:
OP-3rd_party-init-login: The test suite dynamically registers a client with the “initiate_login_uri” claim set in the registration request and verifies that this claim is echoed back in the dynamic client registration response generated by the OP.
OP-3rd_party-init-login-nohttps: The test suite dynamically register a client with the “initiate_login_uri” claim set in the registration request to a plain http (i.e. non-https) URL value and verifies that the OP returns a Client Registration Error Response as defined in https://openid.net/specs/openid-connect-registration-1_0.html#RegistrationError.