Swiftpack.co - Package - mattpolzin/jsonapi-openapi-test-server

JSON:API/OpenAPI Test Server



The test server requires a Postgres database.


The test server and commandline tool will execute tests against an OpenAPI document. Out of box, you get errors for unprocessable OpenAPI documentation, warnings for certain unhandled types of APIs, and a test that every OpenAPI Example parses under the corresponding request/response body schema.

You can additionally create tests that make API calls and verify that the actual responses from your server are parseable under the corresponding response schema. You do this with the x-tests Specification Extension on the OpenAPI Media Type Object within a Response Object (e.g. responses/'200'/content/'application/json'/x-tests). x-tests has the following structure:

    "test_name": {
        "test_host": "url", (optional, if omitted then default server for API will be used.
        "skip_example": true | false, (optional, defaults to false)
        "ignore_missing_parameter_warnings": true | false, (optional, defaults to false)
        "parameters": {
            "path_param_name": "value",
            "header_param_name": "value" (must be a string, even if the parameter type is Int or other)
        "query_parameters": [
                "name": "param_name",
                "value": "param_value"

The Commandline Tool

Against a URL

You can point the test tool at a URL serving up OpenAPI documentation. The URL can either require HTTP Basic Authentication or no authentication.

The unauthenticated version only requires the API_TEST_IN_URL environment variable.

docker run --rm --entrypoint ./APITest --env 'API_TEST_IN_URL=https://website.com/api/documentation' mattpolzin2/api-test-server

The authenticated version additionally requires the API_TEST_USERNAME and API_TEST_PASSWORD environment variables.

docker run --rm --entrypoint ./APITest --env 'API_TEST_IN_URL=https://website.com/api/documentation' --env 'API_TEST_USERNAME=username' --env 'API_TEST_PASSWORD=password' mattpolzin2/api-test-server ./APITest

Against a local file

You can point the test tool at a local file if you mount that file into the docker container and specify the mount destination with the API_TEST_IN_FILE environment variable or the --openapi-file option for the test command.

# command option
docker run --rm --entrypoint ./APITest -v '/full/path/to/openapi.json:/api/openapi.json' mattpolzin2/api-test-server test --openapi-file /api/openapi.json

# ENV var
docker run --rm --entrypoint ./APITest --env 'API_TEST_IN_FILE=/api/openapi.json' -v '/full/path/to/openapi.json:/api/openapi.json' mattpolzin2/api-test-server

Note that you cannot use relative paths with bind mounts but if, for example, your openapi.json file is in the current working directory then you could invoke as:

docker run --rm --entrypoint ./APITest --env 'API_TEST_IN_FILE=/api/openapi.json' -v "$(pwd)/openapi.json:/api/openapi.json" mattpolzin2/api-test-server

API Host Override

You can specify an override test server URL if you want to make API test requests against a different URL than is specified bu the OpenAPI documentation. You use the test command's --override-server option for this.

docker run --rm --entrypoint ./APITest -v '/full/path/to/openapi.json:/api/openapi.json' mattpolzin2/api-test-server test --openapi-file /api/openapi.json --override-server https://test.server.com

Dumping test files

The test tool works by generating Swift code to parse examples and test responses. These test files include JSON:API models that could be used as a basis for client implementations. You can dump the test files with the --dump-files argument to the ./APITest test command. You must also mount the output directory so you can access the generated file from outside of the container.

docker run --rm --env 'API_TEST_IN_URL=https://website.com/api/documentation' -v "$(pwd)/out:/app/out" mattpolzin2/api-test-server ./APITest test --dump-files

You will find the dumped files at ./out/api_test_files.zip.

The Test Server

You can run an API Test server that accepts requests to run tests at HTTP endpoints. This requires the same input file or URL environment variables explained in the above section but you also must provide a Postgres database for the server to use as its persistence layer. You specify this database using a Postgres URL in the API_TEST_DATABASE_URL environment variable.

First you need to run the migrator against your Postgres database.

docker run --env 'API_TEST_IN_URL=https://website.com/api/documentation' --env 'API_TEST_DATABASE_URL=postgres://user:password@host:port/databasename' -p '8080:80' mattpolzin2/api-test-server migrate --yes

Then you can start the server.

docker run --env 'API_TEST_IN_URL=https://website.com/api/documentation' --env 'API_TEST_DATABASE_URL=postgres://user:password@host:port/databasename' -p '8080:80' mattpolzin2/api-test-server

NOTE We must explicitly expose the port to the host device. In this example, http://localhost:8080 will point to the server which is listening on port 80 in the container.

Visit the /docs API endpoint to see what endpoints the server provides.


Note that Vapor 4 (and therefore this server) requires Swift 5.2.

Running and Testing

As of this writing, you need to run swift package generate-xcodeproj and then open that project in Xcode. Using Xcode's built-in Swift Package Manager support is currently broken for libraries like swift-syntax that require dynamic libraries from the Swift toolchain. swift build, swift test, etc. from the command line will work fine, though.

Generating API Documentation

To generate API documentation, run the GenAPIDocumenation target and save the output to Public/openapi.yml. This file is not committed to the repository but it will be picked up by the ReDoc UI served at the /docs API endpoint.

Documentation is generated as part of the Docker image build so you do not need to perform it as a separate step if you are building the Docker image anyway.

Building Docker Image

From the root folder of the repository, run

docker build -t api-test-server:latest .

Once done, you will have the api-test-server:latest Docker image.


Stars: 2


Used By

Total: 0


- 2020-07-04 03:10:55

  • Log info messages.
  • Improve warnings for JSON:API resource swift gen failures.
  • Add test names to output of context for tests when available.
  • Fix numerous edge cases around API interpretation by using new resolved (canonical) types from OpenAPIKit.

Fix bugs with source and properties creation - 2020-06-10 03:27:40

Fix bugs where source and property creation resulted in duplicate database records even though there's no reason to have dupes. Fix bug with database field for URL being a TEXT whereas Fluent expected a JSON field.

Add options to command line tool. - 2020-06-07 23:26:09

Add --openapi-file and --override-server command line options to APITest test command. These allow you to specify the input file (alternative to API_TEST_IN_FILE environment variable) and override the server specified by the OpenAPI documentation for test requests.

Update for JSONAPI v4 release. - 2020-06-01 07:10:06

Error logging bug fix - 2020-05-27 06:59:56

Was not logging request errors.

Breaking Changes to API models - 2020-05-27 02:55:11

Add --ignore-warnings flag - 2020-05-06 01:45:44

Add --ignore-warnings flag to APITest test command.

Add --fail-hard flag to `APITest test` command. - 2020-05-04 02:55:14

When running the test service as a terminal utility instead of a server, you now have an additional command line flag --fail-hard which tells the utility to fail with exit code 1 (when there is at least 1 failing test). Otherwise, the script will exit with code 0 even when there are failing tests (which is what it always has done in past versions).

Adds Test Properties - 2020-05-03 03:37:09

Adds a new resource type: test properties.

⚠️ Breaking Changes ⚠️ Test Properties were added to Test Descriptors and now Test Sources are owned by Test Properties instead of directly being referenced by Test Descriptors. It was decided to simply break SQL migrations given the early stage of the project.

Updates to some warning messages. - 2020-04-27 06:20:16

Improve test result context output. - 2020-04-27 03:03:49

Test results now report the context (API path, endpoint, http status) in a much more human friendly way thats they used to.

Fix bug with positional parameter type for request tests. - 2020-04-26 08:06:52

Minor bug fixes and added ignore missing parameter warning option. - 2020-04-26 03:37:01

New "raw logs" route and bug fixes - 2020-04-25 18:01:27

  • Add /api_tests/{id}/logs route to retrieve raw (XCTest) logs from a particular test run.
  • Fix a bug preventing status code checks on response tests.

Add test messages to watch mode - 2020-04-23 02:05:37

Update OpenAPIKit version - 2020-04-22 07:29:46

Add Websocket support for watching test results. - 2020-04-19 22:36:12

The ws route at /watch will push JSON:API api_test_descriptor single-resource documents for each new or modified test descriptor. These updates occur for every test state: pending, building, running, passed, failed.

Update Vapor, Update OpenAPIKit, change docker name - 2020-03-09 05:53:52

⚠️ Breaking Changes ⚠️

  • Updated Vapor 4. Now requires Swift 5.2
  • Updated OpenAPIKit
  • Moved to mattpolzin2/api-test-server on Docker Hub.

Archived test files and multiple runs simultaneously - 2019-12-05 16:53:27

Add support for dumping test files in a zipped format from the APITest executable (APITest test --dump-files).

Add support for retrieving archived test files from server with new api_test/{id}/files endpoint.

Add support for running more than one test simultaneously (or rather coincidentally) to the server. Previously you could request tests in rapid fire, but test files were all generated in the same folder so this was not really supported. Even now, the number of concurrently running tests is 1, but async operations from preparing or tearing down tests will not step on the toes of other test runs.

Improve error messages around expected response body - 2019-11-07 01:22:05

When a test case that checks for an expected JSON:API response body fails, the errors are now much more concise and clear.

Initial pre-release - 2019-11-03 01:41:38

Notes: Currently using beta versions of Vapor 4.