Conformance Testing for OpenID AuthZEN Authorization API 1.0
This page describes how to run the conformance tests for OpenID AuthZEN PDP. At the time of writing, the set of tests currently available in production are in an early stage. Do not hesitate to register issues in the conformance suite project (https://gitlab.com/openid/conformance-suite) if you're observing unexpected or incorrect behavior, or if you have questions regarding the tests.
The OpenID Foundation conformance suite tests a Policy Decision Point (PDP) Server that implements the AuthZEN Authorization API 1.0. The suite acts as the Policy Enforcement Point (PEP): it calls your PDP’s API endpoints and checks the responses against the specification and the AuthZEN certification profile.
References
- AuthZEN Authorization API 1.0
- AuthZEN Authorization API 1.0 Certification Scenario (the certification profile)
- Conformance suite
The AuthZEN tests are at an early stage. Please report anything unexpected (see Getting help). The six plans under Available test plans are the certification plans.
Getting started
- Go to https://www.certification.openid.net and log in with your Google or GitLab account.
- Click Create a new test plan.
- Choose one of the AuthZEN test plans listed under Test an AuthZEN PDP server (see Available test plans).
- Select the variants that describe your PDP (see Test plan variants).
- Fill in the configuration (see Configuring the test).
- Click Create Test Plan and continue to Running the tests.
Prepare your PDP: the certification fixture
The certification profile (section 4) defines a small fixture of subjects, resources and actions that your PDP must load before testing. The suite only validates decision values for the requests the fixture defines; every other request is checked for structure and status codes only.
| Entity | Type | Identifiers and properties |
|---|---|---|
| Subjects | user | alice; bob (property role = "admin" when sent as a property) |
| Resources | record | record-1 (property status = "active"); record-2 (property status = "archived") |
| Actions | — | read; write; delete (property soft = true or false) |
Decisions the suite validates (certification profile section 4.4)
| # | Subject | Action | Resource | Decision |
|---|---|---|---|---|
| Core (identifiers only) | ||||
| 1 | alice | read | record-1 | true |
| 2 | alice | write | record-1 | true |
| 3 | bob | read | record-1 | true |
| 4 | bob | write | record-1 | false |
| Properties | ||||
| 5 | alice | write | record-2 (status=archived) | false |
| 6 | bob (role=admin) | write | record-2 (status=archived) | true |
| 7 | alice | delete (soft=true) | record-1 | true |
| 8 | alice | delete (soft=false) | record-1 | false |
Search results the suite expects (certification profile section 4.5) include, for example: users who can read record-1 include alice and bob; records alice can read include record-1; actions alice can perform on record-1 include read and write. The Properties search scenarios add the admin and archived-record cases. Your PDP may return additional entities beyond the fixture.
Available test plans
Plans 1 to 5 each test one AuthZEN API endpoint, and plan 6 tests the PDP metadata. Run every plan that matches what your PDP implements.
1. AuthZEN 1.0: PDP server test for evaluation
- Test plan name
authzen-pdp-evaluation-test-plan- Endpoint
- Access Evaluation API (spec section 6)
- Profile level
- Basic (certification profile section 5)
Covers single decisions for the fixture rules, request validation (missing or malformed subject, action and resource), unknown fields being ignored, transport behaviour (HTTPS POST, JSON content type, malformed JSON, empty body), X-Request-ID echo (on error responses too), idempotency, and, when the PDP authenticates callers, 401 responses for missing or invalid credentials.
2. AuthZEN 1.0: PDP server test for batch evaluations
- Test plan name
authzen-pdp-evaluations-test-plan- Endpoint
- Access Evaluations API (spec section 7)
- Profile level
- Batch (certification profile section 6); the profile requires the Basic level as a prerequisite
Covers batch requests with an evaluations array, inheritance of top-level subject, action, resource and context defaults, per-evaluation overrides, the execute_all semantic, decision ordering, the backwards-compatible single-evaluation forms, transport behaviour, X-Request-ID, idempotency, and, when the PDP authenticates callers, 401 responses for missing or invalid credentials.
3–5. AuthZEN 1.0: PDP server tests for Search
| Plan | Test plan name | Endpoint |
|---|---|---|
| Subject Search | authzen-pdp-subject-search-test-plan | Subject Search API (spec section 8.4) |
| Resource Search | authzen-pdp-resource-search-test-plan | Resource Search API (spec section 8.5) |
| Action Search | authzen-pdp-action-search-test-plan | Action Search API (spec section 8.6) |
Together these make up the Search level (certification profile section 7). They cover the expected search results, request validation, ids that must be ignored in a search request, unknown fields, empty results for unknown entities, response and page object structure, pagination (the subject search plan follows page.next_token across pages with page.limit set), transport behaviour, X-Request-ID, idempotency, and, when the PDP authenticates callers, 401 responses for missing or invalid credentials.
6. AuthZEN 1.0: PDP server test for Discovery
- Test plan name
authzen-pdp-discovery-test-plan- Endpoint
- PDP metadata at
/.well-known/authzen-configuration(spec section 9) - Profile level
- Discovery (certification profile section 9). Discovery is an independent level: it does not require, and is not required by, any other level.
Has a single test, which fetches your PDP’s metadata and checks it:
- HTTP 200 with
Content-Type: application/jsonand a JSON object body; policy_decision_pointequal to the PDP Identifier;access_evaluation_endpointpresent, and every endpoint an HTTPS URL;capabilities, if present, an array of strings;signed_metadata, if present, a JWT with anissclaim whose signature verifies against the PDP JWK Set.
It calls no API endpoint. The metadata endpoint must be available: an HTTP 404 fails the test. The plan has no variants, and the suite sends no credentials when it fetches the metadata, as with other OpenID discovery documents: the metadata endpoint must be reachable without authentication.
The discovery variant on plans 1 to 5 runs the same metadata checks before it takes the endpoints from the metadata, but those plans certify their API level, not Discovery. To certify Discovery, run plan 6.
Not part of the certification program
AuthZEN 1.0: PDP server test for batch evaluations, including optional evaluation semantics
- Test plan name
authzen-pdp-evaluations-comprehensive-test-plan- Endpoint
- Access Evaluations API (spec section 7)
Runs everything in plan 2, plus the optional short-circuiting evaluations_semantic values (deny_on_first_deny, permit_on_first_permit) and the rejection of an unknown evaluations_semantic value. A PDP does not have to support these (spec section 7.1.2.1), so the plan reports no certification profile. Use it to check a PDP that claims these semantics; use plan 2 to certify.
Certification profile names
When you publish results for certification, each plan reports the profile it certifies. For plans 1 to 5 it depends on the AuthZEN Specification Support variant:
| Plan | authzen_support=core | authzen_support=properties |
|---|---|---|
| evaluation | AuthZEN Evaluation | AuthZEN Evaluation w/properties |
| batch evaluations | AuthZEN Evaluations | AuthZEN Evaluations w/properties |
| Subject Search | AuthZEN Subject Search | AuthZEN Subject Search w/properties |
| Resource Search | AuthZEN Resource Search | AuthZEN Resource Search w/properties |
| Action Search | AuthZEN Action Search | AuthZEN Action Search w/properties |
The Discovery plan reports AuthZEN Discovery.
Test plan variants
Plans 1 to 5 each have three dropdowns. Create a separate plan for each combination your PDP supports. The Discovery plan has none.
PDP Authentication Type (pdp_auth_type)
How the suite authenticates to your PDP. The AuthZEN specification leaves PEP authentication out of scope (section 11.2) and the certification profile does not require it, so choose what your deployment uses. If your PDP supports several methods, test each one in its own plan.
| Value | Meaning |
|---|---|
none | The PDP does not authenticate callers. |
client_secret_basic | HTTP Basic authentication with a client id and secret. |
api_key | An API key, sent as a Bearer token in the Authorization header. |
mtls | Mutual TLS: the suite presents a client certificate in the TLS handshake. |
With client_secret_basic and api_key, each plan for an API endpoint (plans 1 to 5) also checks that the PDP returns HTTP 401 for a request with missing credentials and for one with invalid credentials (spec section 11.3).
PDP server metadata location (pdp_server_metadata)
| Value | Meaning |
|---|---|
discovery | The suite fetches your PDP’s metadata from /.well-known/authzen-configuration and takes the endpoints from it (spec section 9). Recommended if your PDP supports it, as it needs less configuration and also tests the metadata. To certify the metadata itself, also run the Discovery plan. If the metadata does not contain the endpoint a plan tests, the PDP does not support that API (spec section 9.1.1) and the plan’s tests stop; add the endpoint to the metadata, or use static. The suite fetches the metadata without credentials, as with other OpenID discovery documents, so the metadata endpoint must be reachable without authentication even when your PDP authenticates API calls. |
static | You enter the endpoints in the test configuration and the suite never calls the metadata endpoint. |
AuthZEN Specification Support (authzen_support)
| Value | Meaning |
|---|---|
core | Your PDP makes decisions from entity identifiers (type, id, name) only. Requests may still carry properties, but they must not change the decision. |
properties | Your PDP also bases decisions on the properties of the subject, resource and action (spec sections 5.1.1, 5.2.1, 5.3.1). This adds the Properties fixture rules 5 to 8 and the property-based search scenarios to each plan, and certifies the “w/properties” profile. Choose it if your PDP supports the Properties sub-level of the certification profile. In the profile, Batch Properties requires Basic Properties, and Search Properties requires Search Core. |
Configuring the test
The configuration form shows only the fields that apply to the plan and variants you selected. You can fill in the form or edit the JSON directly.
Test information
Makes any URLs used in the test unique to you. Set it to something unique, such as your company name, using only URL-safe characters.
Optional free text to identify the run. It is public if you publish the results.
Leave as “No”. You can publish later (see Publishing the results). “Everything” also publishes your configuration, including any secrets in it.
AuthZEN (the pdp object)
pdp.policy_decision_pointall plans
Required for every plan. The base HTTPS URL of your PDP, as defined in spec section 9.1.1. It must use https, have a host, and contain no query or fragment. Do not add a trailing slash unless your PDP’s identifier has one.
With the Discovery plan or the discovery variant, the suite derives the metadata URL from it by inserting /.well-known/authzen-configuration between the host and any path:
| PDP Identifier | Metadata URL |
|---|---|
https://pdp.example.com | https://pdp.example.com/.well-known/authzen-configuration |
https://pdp.example.com/tenant1 | https://pdp.example.com/.well-known/authzen-configuration/tenant1 |
The policy_decision_point in the returned metadata must equal this value exactly (spec section 9.2.3), or the test stops.
| Field | Key | Used by |
|---|---|---|
| Access Evaluation API Endpoint | pdp.access_evaluation_endpoint | Evaluation plan |
| Access Evaluations API Endpoint | pdp.access_evaluations_endpoint | Batch evaluations plan |
| Subject Entity Search API Endpoint | pdp.search_subject_endpoint | Subject Search plan |
| Resource Entity Search API Endpoint | pdp.search_resource_endpoint | Resource Search plan |
| Action Entity Search API Endpoint | pdp.search_action_endpoint | Action Search plan |
If you leave an endpoint field empty, the suite uses the default path from spec section 10.1 under the PDP Identifier:
<PDP Identifier>/access/v1/evaluation
<PDP Identifier>/access/v1/evaluations
<PDP Identifier>/access/v1/search/subject
<PDP Identifier>/access/v1/search/resource
<PDP Identifier>/access/v1/search/action
The suite warns if an endpoint path does not contain a version segment such as /v1 (spec section 4).
pdp.jwksDiscovery plan and discovery variant
Required when your PDP’s metadata contains signed_metadata (spec section 9.1.3); leave it empty otherwise. A JSON Web Key Set, starting with { "keys": [, holding the public key the suite uses to verify the signed_metadata JWT. Do not include private asymmetric keys.
If your PDP signs with an HMAC algorithm such as HS256, the set holds the shared secret as an "oct" key. Anyone who can see your configuration can then sign metadata as your PDP, so use a secret that is only for testing and do not publish “Everything”.
pdp.metadata_issuerDiscovery plan and discovery variant, optional
The expected iss claim of the signed_metadata JWT. Set it only when a party other than the PDP signs the metadata. When empty, iss must equal the PDP Identifier.
Client credentials (pdp_auth_type=client_secret_basic)
client.client_id
The client identifier your PDP issued to the PEP.
client.client_secret
The matching secret.
The suite sends Authorization: Basic base64(client_id:client_secret). This is the OAuth 2.0 client_secret_basic method; the AuthZEN specification recommends OAuth 2.0 support (spec section 2).
As RFC 6749 section 2.3.1 requires, the id and secret are each form-URL-encoded before they are joined and base64-encoded. Letters, digits and - . _ * are unchanged, a space becomes +, and any other character is percent-encoded: a+b/c= is sent as a%2Bb%2Fc%3D. Your PDP must decode the credentials the same way. A PDP that decodes them as plain HTTP Basic credentials (RFC 7617) rejects an id or secret containing such characters with 401, so for that PDP use an id and secret made only of letters, digits and - . _ *. RFC 6749 allows only printable US-ASCII in a client id or secret (Appendix A.1, A.2).
API key (pdp_auth_type=api_key)
client.api_key
The API key your PDP expects. The suite sends it as Authorization: Bearer <key>.
Mutual TLS (pdp_auth_type=mtls)
| Key | Value |
|---|---|
mtls.cert | The TLS client certificate, in PEM format or as base64-encoded DER without headers. Your PDP must trust it. |
mtls.key | The matching unencrypted private key, in PEM format or as base64-encoded DER without headers. |
mtls.ca | Optional. Certificates for the client certificate’s chain, in the same formats. Omit it for a self-signed certificate. |
The certificate is presented in the TLS handshake of every API call. The missing-credentials and invalid-credentials 401 tests do not run with mtls.
Example configurations
Static metadata, no authentication:
{
"alias": "my-company",
"description": "Example PDP, static endpoints",
"pdp": {
"policy_decision_point": "https://pdp.example.com",
"access_evaluation_endpoint": "https://pdp.example.com/access/v1/evaluation",
"access_evaluations_endpoint": "https://pdp.example.com/access/v1/evaluations",
"search_subject_endpoint": "https://pdp.example.com/access/v1/search/subject",
"search_resource_endpoint": "https://pdp.example.com/access/v1/search/resource",
"search_action_endpoint": "https://pdp.example.com/access/v1/search/action"
}
}
Discovery plan, with signed metadata:
{
"alias": "my-company",
"description": "Example PDP, metadata",
"pdp": {
"policy_decision_point": "https://pdp.example.com",
"jwks": {
"keys": [
{ "kty": "EC", "crv": "P-256", "kid": "metadata-key",
"x": "...", "y": "..." }
]
}
}
}
Discovery variant on an API plan, with signed metadata and an API key:
{
"alias": "my-company",
"description": "Example PDP, discovery",
"pdp": {
"policy_decision_point": "https://pdp.example.com",
"jwks": {
"keys": [
{ "kty": "EC", "crv": "P-256", "kid": "metadata-key",
"x": "...", "y": "..." }
]
}
},
"client": {
"api_key": "test-api-key"
}
}
Running the tests
In the test plan overview you are presented with a list of individual tests. Click Run Test to execute each one. Each test calls your PDP and finishes on its own; none of them needs any action in a browser.
Status codes for rejected requests
Several tests send a request your PDP should reject. These statuses follow the AuthZEN working group’s guidance:
| Request | Expected status |
|---|---|
| Wrong method (GET or PUT; the API uses POST, spec section 10.1) | 405 Method Not Allowed with an Allow: POST header (preferred; RFC 7231 section 6.5.5 requires the Allow header on any 405), or 400. A 404 is reported as a warning: it is only expected when your framework routes on method and path, so check it is not a routing problem. A PDP that accepts the request is also reported as a warning. |
| Wrong or missing Content-Type (spec section 10.1, certification profile section 5.4.3) | 400 or 415 Unsupported Media Type. Any other status fails, including 404, as an unsupported media type is a problem with the content, not a missing route. |
| Malformed request (missing or wrongly typed fields, malformed JSON, an empty body) | 400 (spec section 10.1.1). A batch in which an evaluation has no subject, action or resource at any level may instead get 200, with that evaluation returned as "decision": false. |
| Missing or invalid credentials, when your PDP authenticates callers | 401 (spec section 11.3); a WWW-Authenticate header is recommended. |
| Result | Meaning |
|---|---|
| Passed | Every check succeeded. |
| Warning | The PDP did something the specification recommends against (a SHOULD), or the suite could not check something; review it. |
| Failed | The PDP violated a requirement. Open the test’s log to see the request the suite sent, the response, and the requirement that failed, with a link to the specification section. |
| Skipped | The test could not be exercised; its log says why. Tests that do not apply to the variants you chose are not listed in the plan. |
Run every test in the plan. For certification, all tests must have finished without failures.
Publishing the results
In the test plan overview, click Publish everything to make the results visible to anyone, not just logged-in users. Use Publish summary if your configuration contains secrets (client_secret, API key, mTLS key, or an HMAC signing key) that must stay private.
To apply for certification, follow the OpenID Foundation certification process at https://openid.net/certification/ with the published results and the certification profile name for each plan.
Getting help
Do not hesitate to register issues in the conformance suite project if you see unexpected or incorrect behaviour. You can also contact certification@oidf.org.