This is the abridged developer documentation for Axx # axx > Axx (axxeptance) is a human-readable acceptance testing framework for the agentic era. # Why Axx > Axx is a human-readable acceptance testing framework for the agentic era - acceptance criteria that people can read and coding agents can write, run against your real service. Axx is a human-readable acceptance testing framework for the agentic era. Coding agents now write a growing share of code and tests. That makes one question matter more: what exactly was checked? With Axx, the answer is the acceptance criteria themselves. Each scenario states a behavior in plain language, so the people who own that behavior can read and review it. Agents can find the steps, write the scenario, run it and fix what fails. Axx runs every scenario against your service from the outside, through the same interfaces your users and neighboring services use: its HTTP API, its database, its event streams and the dependencies it calls. When a scenario passes, the behavior it describes works in the running system, not in a mock of it or in one unit on its own. ## One scenario covers the whole flow [Section titled “One scenario covers the whole flow”](#one-scenario-covers-the-whole-flow) ```gherkin Feature: Register parcels Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | And the mocked addresses service with the following properties: | url | http://localhost:8081 | And a parcels-db database with the following properties: | url | postgres://localhost:5432/parcels | | user | parcels | | password | parcels | Scenario: A shop registers a parcel Given a POST request to /api/parcels And a request payload using an application/json content example And the request payload properties are: | reference | PX-REG-1001 | | recipient.postcode | "53111" | When the request is executed Then the response status code is 201 And the mocked GET request to /v1/postcodes/DE/53111 named postcode-check was received by addresses And a selection of rows is retrieved from the parcels.parcels table where: | reference | PX-REG-1001 | And the 1st row details property for the selection json properties are: | source | api | | zone | DE-1 | ``` This one scenario: * builds a request from the service’s own OpenAPI example and sends it; * checks the request and the response against that OpenAPI document; * proves the service asked its downstream address service about the postcode; * checks what the service stored in PostgreSQL, down to a JSON column. Every line is a step from one of Axx’s packs, so there is no step code to write. ## Test the service, not the code [Section titled “Test the service, not the code”](#test-the-service-not-the-code) Axx never loads your code. The service can be written in any language, and the tests keep passing through refactors and rewrites because they only depend on what the service does. The people who wrote the acceptance criteria can read them too. See [Black-box testing](/explanations/black-box-testing/). These are built in: | Area | What you get | | ------------------- | --------------------------------------------------------------------------------------------- | | REST | Requests and responses validated against your OpenAPI document | | Mocked dependencies | Calls to them verified through WireMock | | SQL | PostgreSQL, MySQL, SQLite and SQL Server: seeds, selections, JSON columns, locks and triggers | | MongoDB | Seeding and querying | | Kafka | Publishing and consuming events, with Avro and Schema Registry | ## One binary with nothing to wire up [Section titled “One binary with nothing to wire up”](#one-binary-with-nothing-to-wire-up) Axx needs no JVM, no build plugin and no test-runner glue. `axx run` does the whole job: 1. It starts the apps listed in `axx.yaml` and waits until they are healthy. 2. It runs the scenarios in parallel. 3. It stops everything and cleans up, even after a crash or Ctrl-C. `axx up` keeps the system running between runs, so each edit-and-run cycle takes seconds. It is fast. The [example suite](https://github.com/nimbusxr/axx/tree/main/examples/parcels) has 32 scenarios and 316 steps, covering REST, WireMock, PostgreSQL, MongoDB and Kafka. Against a running stack, it finishes in about a second. ## Axx’s steps handle the hard parts [Section titled “Axx’s steps handle the hard parts”](#axxs-steps-handle-the-hard-parts) The steps of Axx’s packs do the difficult work: * OpenAPI validation * database seeds and JSONB queries * Avro with Schema Registry * waiting for events to arrive * checking calls to mocked services All of them compose the same way. You build something (a request, an event, a selection), act on it, and assert on what came back. Any step can name the service it targets (`on parcels-db`). Ordinals such as `the 2nd selection` refer back to earlier ones. Once you know the pattern, a pack you have never used reads the same. Step text is public API, and it never changes, so a feature file keeps working across every release. ## Extend it your way [Section titled “Extend it your way”](#extend-it-your-way) Steps come in **packs**. A project uses only the packs it needs (`axx pack add rest sql`). When Axx doesn’t cover something, write a pack of your own in Go. Your steps work on the same scenario context as Axx’s: * the services the scenario registered * its requests and responses * its database selections * its events Use custom packs to reach a system Axx doesn’t cover, or to write scenarios in your team’s own words. `axx pack new ./steps` creates a pack. The first run builds it into Axx and caches the result. After that, your steps appear in `axx steps search`, `axx validate` and the agent skills like any other step. See [Write custom steps](/guides/write-custom-steps/). ## Built for coding agents [Section titled “Built for coding agents”](#built-for-coding-agents) Axx is designed so coding agents never have to guess. They search the real step text instead of inventing it, check every line without starting anything, get failures as data with the expected and actual values, and read docs generated from the binary they are running. [Axx for agents](/explanations/axx-for-agents/) explains how. ## Test data that doesn’t collide [Section titled “Test data that doesn’t collide”](#test-data-that-doesnt-collide) Parallel scenarios only stay reliable if each one uses its own data. `axx lint` finds colliding ids and keys before they cause flaky failures. `axx fixtures` generates test data checked against your schemas and reports when it drifts. See [Isolate test data](/guides/isolate-test-data/) and [Fixture factories](/guides/fixture-factories/). ## Where to go next [Section titled “Where to go next”](#where-to-go-next) | If you want to | Go to | | ------------------------------------------- | --------------------------------------------- | | Get a passing scenario in about ten minutes | [Quickstart](/tutorials/quickstart/) | | Get a task done | [Guides](/guides/install/) | | Look up a step, command or setting | [References](/references/steps/) | | Understand the design | [How Axx works](/explanations/how-axx-works/) | Axx is pre-release (`v0.x`): interfaces may change before `v1.0.0`. # Your first suite > Build an acceptance suite for a realistic service step by step - its API and OpenAPI contract, a dependency it calls, and its database. In this tutorial, we’ll write an acceptance suite for Parcels, an example service that comes with Axx. Parcels is a parcel-delivery service with a PostgreSQL database. Shops ask it for price quotes and register their parcels with it, and it checks every address with an address service, which the example replaces with a mock. Step by step, our suite will: * call the service’s API and check its answers; * check every request and response against the API’s OpenAPI document; * check the call the service makes to the address service; * check what the service writes to its database. You’ll need Docker with Compose v2, Git, and Axx (see the [Quickstart](/tutorials/quickstart/)). Plan on about 30 minutes, including a few minutes for Docker to build images the first time. ## Get the example [Section titled “Get the example”](#get-the-example) ```sh git clone https://github.com/nimbusxr/axx.git cd axx/examples/parcels ``` The `infra` directory holds a Docker Compose file that runs Parcels with everything it needs. ## Create the suite [Section titled “Create the suite”](#create-the-suite) Make a directory for our suite: ```sh mkdir my-suite cd my-suite ``` Create `axx.yaml` in it: axx.yaml ```yaml version: 1 apps: parcels: dir: ../infra command: docker compose up --build ready: http: url: http://localhost:8400/health timeout: 10m cleanup: docker compose down -v --remove-orphans ``` This tells Axx to start Parcels with Docker Compose, to wait until its health check answers, and to remove the containers when it stops. Next, choose the packs of steps the suite uses. We’ll send requests to Parcels with `rest`, check what it asked the address service’s mock with `mock`, and look in its database with `sql`: ```sh axx pack add rest mock sql ``` ```console created axx-packs.yaml added rest, mock, sql ``` ## Start Parcels [Section titled “Start Parcels”](#start-parcels) ```sh axx up ``` ```console axx: starting apps in the background (logs: .axx/logs) up: parcels (stop with `axx down`) ``` The first time, Docker builds the images, so this takes a few minutes. Parcels now keeps running in the background, and every run in this tutorial takes a fraction of a second. ## Ask for a price [Section titled “Ask for a price”](#ask-for-a-price) Create a `features` directory: ```sh mkdir features ``` Then create `features/quotes.feature`: features/quotes.feature ```gherkin Feature: Quotes Background: Given the parcels service with the following properties: | url | http://localhost:8400 | Scenario: A shop asks for a price Given a POST request to /api/quotes And a request payload using an application/json empty content template And the request payload properties are: | weightGrams | 1200 | | recipient | {"postcode": "10115", "country": "DE"} | When the request is executed Then the response status code is 200 And the response payload property priceCents is '690' ``` The scenario builds a request body from an empty JSON object: a 1.2 kg parcel to a Berlin postcode. Run it: ```sh axx run ``` ```console axx: preparing rest, mock, sql (once; cached for later runs) axx: ready in 31s axx: reusing apps started by `axx up`: parcels Feature: Quotes Scenario: A shop asks for a price # features/quotes.feature:7 ✓ Given the parcels service with the following properties: (background) | url | http://localhost:8400 | ✓ Given a POST request to /api/quotes ✓ And a request payload using an application/json empty content template ✓ And the request payload properties are: | weightGrams | 1200 | | recipient | {"postcode": "10115", "country": "DE"} | ✓ When the request is executed (213ms) log: POST http://localhost:8400/api/quotes -> 200 OK (213ms, 112 bytes) attachment: request body (application/json, 68 B) {"weightGrams":1200,"recipient":{"postcode":"10115","country":"DE"}} attachment: response body (application/json, 112 B) {"zone":"DE-1","serviceLevel":"STANDARD","weightGrams":1200,"priceCents":690,"currency":"EUR","deliveryDays":2} ✓ Then the response status code is 200 ✓ And the response payload property priceCents is '690' 1 scenario (1 passed) 7 steps (7 passed) Finished in 213ms ``` Notice the first lines. Axx prepared itself with the suite’s packs, which it does only once. And it didn’t start anything this time: it used the Parcels service that `axx up` started. The price is 6.90 EUR, in cents. ## Check the API contract [Section titled “Check the API contract”](#check-the-api-contract) Parcels publishes an OpenAPI document that describes its API. Add it to the service in the `Background`: features/quotes.feature ```gherkin Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | ``` Run `axx run` again. The scenario still passes. Axx now checks every request and response against that document, so the suite notices when Parcels breaks its contract. We’ll see the check at work in a moment. ## Start from the contract’s example [Section titled “Start from the contract’s example”](#start-from-the-contracts-example) The OpenAPI document also gives an example request for each operation. Add a second scenario at the end of `features/quotes.feature`: features/quotes.feature ```gherkin Scenario: Heavier parcels cost more Given a POST request to /api/quotes And a request payload using an application/json content example And the request payload property weightGrams is '4500' When the request is executed Then the response status code is 200 And the response payload properties are: | weightGrams | 4500 | | priceCents | 690 | ``` Run `axx run`. Both scenarios pass. Look at the new scenario’s output: ```console Scenario: Heavier parcels cost more # features/quotes.feature:18 ✓ Given the parcels service with the following properties: (background) | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | ✓ Given a POST request to /api/quotes ✓ And a request payload using an application/json content example ✓ And the request payload property weightGrams is '4500' ✓ When the request is executed log: POST http://localhost:8400/api/quotes -> 200 OK (3ms, 112 bytes) attachment: request body (application/json, 94 B) {"weightGrams":4500,"serviceLevel":"STANDARD","recipient":{"postcode":"10115","country":"DE"}} attachment: response body (application/json, 112 B) {"zone":"DE-1","serviceLevel":"STANDARD","weightGrams":4500,"priceCents":690,"currency":"EUR","deliveryDays":2} ✓ Then the response status code is 200 ✓ And the response payload properties are: | weightGrams | 4500 | | priceCents | 690 | ``` Notice the request body. We only wrote the weight. Axx started from the example request in the OpenAPI document, which also has a service level and a recipient, and changed the weight. ## Break the contract on purpose [Section titled “Break the contract on purpose”](#break-the-contract-on-purpose) In the new scenario, change the weight to a string: ```gherkin And the request payload property weightGrams is '"heavy"' ``` Run `axx run`: ```console ✗ When the request is executed log: POST http://localhost:8400/api/quotes -> 400 Bad Request (1ms, 125 bytes) attachment: request body (application/json, 97 B) {"weightGrams":"heavy","serviceLevel":"STANDARD","recipient":{"postcode":"10115","country":"DE"}} attachment: response body (application/problem+json, 125 B) {"detail":"weightGrams must be an integer","instance":"/api/quotes","status":400,"title":"Bad Request","type":"about:blank"} OpenAPI validation failed for POST http://localhost:8400/api/quotes (status 400): - validation.request.body.schema.type: $.weightGrams: got string, want integer (POST request body for '/api/quotes' failed to validate schema) To relax a check, set its key (or a parent key) to WARN, INFO or IGNORE with "Given the OpenAPI validation levels are:" or openapi.levels in axx.yaml. ↷ Then the response status code is 200 ↷ And the response payload properties are: | weightGrams | 4500 | | priceCents | 690 | ``` The double quotes make the value a string. The contract says `weightGrams` is an integer, and Axx fails the request step with the reason: `$.weightGrams: got string, want integer`. The steps after it are skipped. Change the value back to `'4500'` and run `axx run` again. Both scenarios pass. ## Check what Parcels asked the address service [Section titled “Check what Parcels asked the address service”](#check-what-parcels-asked-the-address-service) To price a parcel, Parcels asks the address service which delivery zone serves the recipient’s postcode. In this example, the address service is a mock running on port 8081. Let’s check that the call happened. Register the mock in the `Background`, and add a last step to the first scenario. Your file now looks like this: features/quotes.feature ```gherkin Feature: Quotes Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | And the mocked addresses service with the following properties: | url | http://localhost:8081 | Scenario: A shop asks for a price Given a POST request to /api/quotes And a request payload using an application/json empty content template And the request payload properties are: | weightGrams | 1200 | | recipient | {"postcode": "10115", "country": "DE"} | When the request is executed Then the response status code is 200 And the response payload property priceCents is '690' And the mocked GET request to /v1/postcodes/DE/10115 named zone-lookup was received by addresses Scenario: Heavier parcels cost more Given a POST request to /api/quotes And a request payload using an application/json content example And the request payload property weightGrams is '4500' When the request is executed Then the response status code is 200 And the response payload properties are: | weightGrams | 4500 | | priceCents | 690 | ``` Run `axx run`. Both scenarios pass: ```console ✓ Then the response status code is 200 ✓ And the response payload property priceCents is '690' ✓ And the mocked GET request to /v1/postcodes/DE/10115 named zone-lookup was received by addresses ``` Axx asked the mock which requests it had received, and found Parcels’ `GET /v1/postcodes/DE/10115`. ## Check the database [Section titled “Check the database”](#check-the-database) Now we’ll register a parcel through the API and check that Parcels stored it. Create `features/parcels.feature`: features/parcels.feature ```gherkin Feature: Parcels Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | And a parcels-db database with the following properties: | url | postgres://localhost:5432/parcels | | user | parcels | | password | parcels | Scenario: A registered parcel is stored Given a POST request to /api/parcels And a request payload using an application/json content example And the request payload properties are: | reference | PX-TUTORIAL-1 | | sender | my-shop | When the request is executed Then the response status code is 201 And a selection of rows is retrieved from the parcels.parcels table where: | reference | PX-TUTORIAL-1 | | sender | my-shop | And the selection has 1 row ``` The `Background` now also registers Parcels’ database. Run `axx run`, and look at the end of the output: ```console ✓ Then the response status code is 201 ✓ And a selection of rows is retrieved from the parcels.parcels table where: | reference | PX-TUTORIAL-1 | | sender | my-shop | ✓ And the selection has 1 row 3 scenarios (3 passed) 26 steps (26 passed) Finished in 22ms ``` The new scenario registered a parcel through the API. Then it selected the rows of the `parcels.parcels` table with that reference and sender, and found exactly one: Parcels stored what it was sent. Run `axx run` once more. This time the new scenario fails: ```console log: POST http://localhost:8400/api/parcels -> 409 Conflict (1ms, 135 bytes) {"detail":"parcel PX-TUTORIAL-1 is already registered","instance":"/api/parcels","status":409,"title":"Conflict","type":"about:blank"} ✗ Then the response status code is 201 ``` The parcel from the first run is still in the database, and references are unique. Acceptance tests run against the service’s real data, so data a scenario creates stays behind. [Isolate test data](/guides/isolate-test-data/) shows how suites deal with that. For now, stopping Parcels clears everything. ## Stop Parcels [Section titled “Stop Parcels”](#stop-parcels) ```sh axx down ``` ```console stopped: parcels ``` Axx stopped Parcels and ran its cleanup, which removed the containers and their data. The next `axx up` starts from empty databases. ## What you’ve done [Section titled “What you’ve done”](#what-youve-done) You wrote a suite for a service with a database and a dependency, and checked four things from the outside: * the answers of its API; * its requests and responses against its OpenAPI contract; * the call it makes to another service; * what it writes to its database. You didn’t write any test code: every line is a step Axx provides. The complete suite for Parcels, with events, a document store and more, is in `examples/parcels/acceptance`. Next, in [Test with an agent](/tutorials/with-an-agent/), we’ll hand an acceptance criterion to a coding agent and review the scenario it writes. # Quickstart > Install Axx and run your first passing scenario against a small web server on your machine, in about ten minutes. In this tutorial, we’ll install Axx, create a project, and write a scenario that tests a small web server running on your machine. We’ll run it and watch it pass, then break it on purpose to see what a failure looks like. It takes about ten minutes. You’ll need a terminal, [Go](https://go.dev/dl/) 1.27 or newer to install Axx, and Python 3, which we’ll use as the web server. ## Install Axx [Section titled “Install Axx”](#install-axx) Install Axx with Go: ```sh go install github.com/nimbusxr/axx/cmd/axx@latest ``` Check that your shell finds it: ```sh axx version ``` Axx prints its version. If your shell says `command not found`, add Go’s `bin` directory to your `PATH` (`export PATH="$PATH:$(go env GOPATH)/bin"`) and try again. ## Create a project [Section titled “Create a project”](#create-a-project) Make a directory for the project and let Axx set it up: ```sh mkdir hello-axx cd hello-axx axx init ``` ```console create axx.yaml create axx-packs.yaml create features/smoke.feature create .github/workflows/acceptance.yml create AGENTS.md create .gitignore next: edit apps in axx.yaml, then `axx doctor` and `axx run` ``` Axx created a configuration file, `axx.yaml`, and a first feature file, `features/smoke.feature`. We’ll change both. It also created `axx-packs.yaml`, the list of the packs of steps the project uses. It lists `rest`, whose steps send HTTP requests, and that’s all we need. We won’t need the other files in this tutorial. ## Give the web server something to serve [Section titled “Give the web server something to serve”](#give-the-web-server-something-to-serve) Our web server will serve the files in this directory. Create a small JSON file for it: ```sh echo '{"message": "Hello, axx"}' > hello.json ``` ## Tell Axx how to start the server [Section titled “Tell Axx how to start the server”](#tell-axx-how-to-start-the-server) Open `axx.yaml`. At the bottom, replace the whole `apps:` section with this one: axx.yaml ```yaml apps: hello-axx: command: python3 -m http.server 8000 ready: http: url: http://${sys:local.host}:8000/ ``` Axx now knows how to start the server, and how to tell when it’s ready: when `http://localhost:8000/` answers. ## Write the scenario [Section titled “Write the scenario”](#write-the-scenario) Replace everything in `features/smoke.feature` with: features/smoke.feature ```gherkin Feature: Hello axx Background: Given the hello-axx service with the following properties: | url | http://${sys:local.host}:8000 | Scenario: The service says hello Given a GET request to /hello.json When the request is executed Then the response status code is 200 And the response payload property message is 'Hello, axx' ``` Every line is a step Axx already knows. The `Background` registers the server as a service named `hello-axx`. The scenario sends it a request and checks the answer. ## Check the scenario [Section titled “Check the scenario”](#check-the-scenario) Before running anything, ask Axx to check every line: ```sh axx validate ``` ```console axx: preparing rest (once; cached for later runs) axx: ready in 31s 1 file, 1 scenario, 5 steps: ok ``` Axx matched each line to one of its steps, without starting the server. The first time, it also prepared itself with the project’s packs. It won’t need to do that again. ## Run it [Section titled “Run it”](#run-it) ```sh axx run ``` ```console axx: starting hello-axx (logs: .axx/logs/apps.log) Feature: Hello axx Scenario: The service says hello # features/smoke.feature:7 ✓ Given the hello-axx service with the following properties: (background) | url | http://${sys:local.host}:8000 | ✓ Given a GET request to /hello.json ✓ When the request is executed log: GET http://localhost:8000/hello.json -> 200 OK (4ms, 26 bytes) attachment: response body (application/json, 26 B) {"message": "Hello, axx"} ✓ Then the response status code is 200 ✓ And the response payload property message is 'Hello, axx' 1 scenario (1 passed) 5 steps (5 passed) Finished in 4ms ``` Notice what happened. Axx started the web server, waited until it answered, ran the scenario, and stopped the server again. Under the request step, the `log:` line shows the request Axx sent, and the attachment shows the response it got back. ## Break it on purpose [Section titled “Break it on purpose”](#break-it-on-purpose) Change the last line of `features/smoke.feature` so it expects a different message: ```gherkin And the response payload property message is 'Hello, world' ``` Run it again: ```sh axx run ``` ```console axx: starting hello-axx (logs: .axx/logs/apps.log) Feature: Hello axx Scenario: The service says hello # features/smoke.feature:7 ✓ Given the hello-axx service with the following properties: (background) | url | http://${sys:local.host}:8000 | ✓ Given a GET request to /hello.json ✓ When the request is executed log: GET http://localhost:8000/hello.json -> 200 OK (2ms, 26 bytes) attachment: response body (application/json, 26 B) {"message": "Hello, axx"} ✓ Then the response status code is 200 ✗ And the response payload property message is 'Hello, world' Response payload property message is not Hello, world expected: "Hello, world" actual: "Hello, axx" Failed scenarios: ✗ The service says hello # features/smoke.feature:7 rerun: axx run features/smoke.feature:7 1 scenario (1 failed) 5 steps (1 failed, 4 passed) Finished in 3ms ``` Axx marks the step that failed and shows what it expected next to what it got. It also prints a command that reruns only this scenario. Change the line back to `'Hello, axx'` and run `axx run` once more. The scenario passes again. ## What you’ve done [Section titled “What you’ve done”](#what-youve-done) You installed Axx, told it how to start a service, wrote a scenario, ran it, and read a failure. That’s the loop you’ll use with every Axx suite. Next, in [Your first suite](/tutorials/first-suite/), we’ll test a real service with a database and a dependency it calls. # Test with an agent > Hand an acceptance criterion to a coding agent, watch it write and run the scenario with Axx, and review what it wrote. In this tutorial, we’ll give a coding agent an acceptance criterion in plain words and watch it write a scenario, check it and run it with Axx. Then we’ll review what it wrote. We’ll use [Claude Code](https://claude.com/claude-code) and the `hello-axx` project from the [Quickstart](/tutorials/quickstart/). You’ll need both. ## Teach the agent Axx [Section titled “Teach the agent Axx”](#teach-the-agent-axx) In the `hello-axx` directory, install Axx’s skills: ```sh axx skills install ``` ```console installed 4 skills to .agents/skills (22 files updated) linked for Claude Code in .claude/skills ``` The skills teach an agent how to find Axx’s steps, write a scenario, run it and read a failure. The `AGENTS.md` that `axx init` wrote in the Quickstart points agents at the same rules. ## Ask for a test [Section titled “Ask for a test”](#ask-for-a-test) Start Claude Code in the project: ```sh claude ``` Give it the criterion, not step text: ```text Add an acceptance test for this: asking for a file that doesn't exist returns 404. Use axx, and run it. ``` ## Watch what it does [Section titled “Watch what it does”](#watch-what-it-does) The agent’s wording will differ from run to run, but you’ll see it work through the same loop. Claude Code asks before each command, so you can follow along. Notice that it: 1. checks the project with `axx doctor`; 2. searches for real steps with `axx steps search "response status code"` instead of guessing their text; 3. checks a line it’s unsure of with `axx explain`; 4. writes a feature file; 5. checks it with `axx validate`, starts the server with `axx up`, runs the scenarios with `axx run --compact`, and stops the server with `axx down`. ## Review what it wrote [Section titled “Review what it wrote”](#review-what-it-wrote) Open the feature file the agent created. In our run it was `features/missing-file.feature`: features/missing-file.feature ```gherkin Feature: Missing files Asking for a file the service does not have is answered with Not Found. Background: Given the hello-axx service with the following properties: | url | http://${sys:local.host}:8000 | Scenario: Asking for a file that does not exist returns 404 Given a GET request to /no-such-file-404.json When the request is executed Then the response status code is 404 ``` Read it against the criterion you gave. Each line says what it checks, so you can tell at a glance whether the agent tested what you asked for. That review is the part that stays yours. ## Run it yourself [Section titled “Run it yourself”](#run-it-yourself) ```sh axx run ``` The end of the output shows both scenarios, the Quickstart’s and the agent’s: ```console 2 scenarios (2 passed) 9 steps (9 passed) ``` ## What you’ve done [Section titled “What you’ve done”](#what-youve-done) You gave an agent a criterion in plain words. It found the steps, wrote the scenario, checked it and ran it, and you confirmed it by reading it. [Set up agents](/guides/set-up-agents/) connects other agents and Axx’s MCP server, and [Axx for agents](/explanations/axx-for-agents/) explains why Axx works this way. # Axx for agents > The design choices that make Axx usable by coding agents - discoverable steps, validation without side effects, compact structured output, stable contracts, and docs served from the binary. Coding agents are now a primary user of test tools: they write most new tests, run them, and act on the result. Axx is designed for that user alongside people. The principle: an agent should never have to guess. ## Never guess a step [Section titled “Never guess a step”](#never-guess-a-step) Agents invent plausible step text, and Gherkin punishes near-misses. Axx makes the real steps cheap to find and cheap to check: * `axx steps search ""` and the `steps_search` MCP tool return real expressions with docs and complete examples. * The skills ship a one-line-per-step index generated from the project, including custom steps. * `axx validate` and `axx explain` check every line without starting anything, and suggest the closest real steps for a near-miss. * Step text is public API: it never changes, so what an agent learned stays true. ## Never guess what happened [Section titled “Never guess what happened”](#never-guess-what-happened) * **Exit codes** classify the outcome: a failed scenario (`1`), a broken setup (`2`), an invented step (`3`), an app that did not start (`4`). * **Error codes** (`AXX-Exxxx`) are stable and come with a hint and a link. * **Failures are data.** Assertion failures carry expected and actual values, the matched step definition, pack context such as the last HTTP exchange, and the exact command that reruns the scenario. * **`--json` everywhere**, in one envelope, with a frozen schema. ## Spend tokens on the problem [Section titled “Spend tokens on the problem”](#spend-tokens-on-the-problem) When Axx detects an agent and its output is not a terminal, it switches to compact output: failures and one summary line. A green run of a thousand scenarios costs one line of context. Details are one call away (`axx run --json`, or the `failure_context` MCP tool). ## Stay in sync with the installed version [Section titled “Stay in sync with the installed version”](#stay-in-sync-with-the-installed-version) Documentation drifts; binaries do not. The MCP server, `axx steps`, the skills and the reference pages on this site are all generated from the binary (ADR 0005), so an agent that asks the tool gets answers for the version in the repository, including that project’s custom packs. ## A fast, safe loop [Section titled “A fast, safe loop”](#a-fast-safe-loop) `axx up` keeps the system under test running between runs, so an agent’s edit-run loop costs seconds instead of minutes. `axx run features/x.feature:LINE` reruns exactly one scenario. Validation, explanation and step search have no side effects and need no running apps. ## Guardrails in the instructions [Section titled “Guardrails in the instructions”](#guardrails-in-the-instructions) The skills and the AGENTS.md section encode the rules that keep agent-written tests honest: * one scenario per acceptance criterion, named after the behavior; * unique data in every scenario; * assert on observable outcomes, never loosen an assertion to go green; * no fixed sleeps, use readiness checks and polling steps. ## Docs for machines [Section titled “Docs for machines”](#docs-for-machines) Every page on this site has a Markdown twin (append `.md`), advertised with ``. [`/llms.txt`](/llms.txt) indexes them, and [`/.well-known/agent-skills/index.json`](/.well-known/agent-skills/index.json) lists the skills. Each page also has *Copy as Markdown* and *Open in Claude* buttons. Set it up with [Set up agents](/guides/set-up-agents/), and see it work in [Test with an agent](/tutorials/with-an-agent/). # Black-box acceptance testing > Why Axx tests services from the outside, through the same interfaces their clients use, and what that buys you over in-process tests. An **acceptance test** checks that the system does what was agreed: the acceptance criteria of a story, written so the people who agreed on them can read the test. A **black-box** test checks it the way a client would, through the system’s public interfaces, with no knowledge of its insides. Axx does both. A scenario sends real requests to a running service, puts real rows in its real database, publishes real events to its broker, and checks what comes back and what the service did to the world. ## Why from the outside [Section titled “Why from the outside”](#why-from-the-outside) * **The test survives refactoring.** Rename a class, swap a framework, rewrite the service in another language: the scenarios still describe the same behavior and still run. * **It tests what ships.** The artifact under test is the one you deploy (a container, a binary), started the way you start it, configured the way you configure it. Serialization, framework wiring, database constraints and migrations are all in the loop. * **It needs no access to the code.** The service does not depend on Axx or on any test library. A team can write acceptance tests for a service in any language, and an agent can write them without reading the implementation. * **It reads like the requirement.** A scenario is the acceptance criterion in structured English. When it fails, the name says which promise broke. ## What it costs, and how Axx pays it [Section titled “What it costs, and how Axx pays it”](#what-it-costs-and-how-axx-pays-it) Black-box suites have a reputation for being slow and flaky. The causes are specific, and so are the fixes: | Cause | What Axx does | | ---------------------------------------- | -------------------------------------------------------------------------- | | Starting the system for every test class | starts apps once per run, or once per session with `axx up` | | Tests run one at a time | runs scenarios in parallel by default | | Tests share and corrupt data | makes data isolation checkable with `axx lint` and fixture identities | | Waiting with fixed sleeps | readiness checks for apps, polling steps for asynchronous results | | Brittle hand-written payloads | payloads from OpenAPI examples and fixture factories | | Opaque failures | expected and actual values, the last request and response, a rerun command | ## Where it fits [Section titled “Where it fits”](#where-it-fits) Black-box acceptance tests sit on top of unit and integration tests; they do not replace them. Use them for the behavior a client or another team relies on: the API contract, the events you publish, the data you persist, the calls you make to dependencies. Keep edge cases of pure logic in unit tests, where they are cheaper. ## Contracts in both directions [Section titled “Contracts in both directions”](#contracts-in-both-directions) A service has two contracts: the API it provides and the APIs it consumes. Axx checks both from the outside: * **Provided**: every request and response is validated against your OpenAPI document ([OpenAPI as contract](/explanations/openapi-contract/)). * **Consumed**: dependencies are replaced by WireMock mocks, which can themselves be validated against the provider’s OpenAPI document ([Mock dependencies](/guides/mock-dependencies/)). # How Axx works > What happens between typing axx run and reading the result - configuration, step matching, app lifecycle, parallel execution, packs, the world and reporting. Axx is one Go binary with its own Cucumber executor. It reads Gherkin, matches every step to a definition, starts the system under test, runs the scenarios against it from the outside, and reports. ```text axx.yaml ─┐ features ─┼─▶ load & validate ─▶ match steps ─▶ start apps ─▶ run scenarios ─▶ stop apps ─▶ reporters packs ─┘ (registry) (lifecycle) (workers) (cleanup) (pretty, junit, json…) ``` ## 1. Load [Section titled “1. Load”](#1-load) Axx finds `axx.yaml` by walking up from the working directory, applies the selected profile and `axx.local.yaml`, expands `${env:...}` and `${sys:...}`, and validates the result against the [schema](/references/config/). It then parses the feature files with the official Cucumber Gherkin parser and applies the path, line, tag and name filters. ## 2. Match [Section titled “2. Match”](#2-match) Every step line is matched against the **registry**: the steps of the packs the project loads. Steps are [Cucumber Expressions](https://github.com/cucumber/cucumber-expressions) with custom parameter types such as `{ordinal}` and `{service}`. Optional segments like `[[ on {service}]]` register every variant of a step. A line that matches nothing is *undefined*; a line that matches two definitions is *ambiguous*. Both are found before anything runs: `axx validate` and `axx explain` are this phase alone, so they never start an app. ## 3. Start the apps [Section titled “3. Start the apps”](#3-start-the-apps) The lifecycle manager starts `apps` in dependency order, independent apps in parallel, each in its own process group. It polls every `ready` check until they pass or time out. With `active.enabled`, only apps whose tags match the selected scenarios start. With `axx up`, a background supervisor keeps them running and later runs skip this step. ## 4. Run [Section titled “4. Run”](#4-run) Scenarios run in parallel on `run.workers` workers; steps within a scenario run in order. Scenarios tagged with a `run.exclusive` tag run one at a time after the parallel phase. Each scenario gets a fresh **world**: its registered services and the state each pack keeps (requests built and responses received, selections, events). Nothing in the world is shared between scenarios, which is why parallel runs are safe as long as the *external* data is unique. See [Scenario isolation](/explanations/scenario-isolation/). ## 5. Packs [Section titled “5. Packs”](#5-packs) A **pack** is a bundle of steps, parameter types and hooks, plus the per-scenario context its steps share. The packs are `rest`, `mock`, `sql`, `mongo`, `kafka`, `logs`, and `core` for shared parameter types. ## 6. Stop and report [Section titled “6. Stop and report”](#6-stop-and-report) After the last scenario, apps are stopped (signal, grace period, kill) and every `cleanup` runs, including after a crash or an interrupt. Reporters receive the run as [Cucumber Messages](https://github.com/cucumber/messages) events and render them: pretty, progress or compact on the console; JUnit, HTML, Cucumber JSON, NDJSON messages and the JSON agent report to files. The exit code summarizes the worst outcome ([exit codes](/references/error-codes/#exit-codes)). ## What Axx is not [Section titled “What Axx is not”](#what-axx-is-not) * **Not a unit test framework.** Axx tests a running system through its public interfaces. It never loads your code. * **Not a load-testing tool.** Parallelism is for speed, not for generating traffic. # OpenAPI as the contract > Why Axx treats your OpenAPI document as the single source of truth for an API, validates every exchange against it, and builds payloads from its examples. An API has one contract, and it should live in one place. In Axx that place is the OpenAPI document. Scenarios do not restate the contract; they rely on it. ## The document is the source of truth [Section titled “The document is the source of truth”](#the-document-is-the-source-of-truth) Some contract-testing approaches make the tests the source of truth and generate stubs or documents from them. Then the published OpenAPI document and the tested behavior can disagree, and nobody notices until a client breaks. Axx goes the other way: the document the service publishes is the contract, and the acceptance suite proves the service honors it. When a REST service is registered with an `openapi` property, every request a scenario sends and every response it receives is validated against that document. A scenario that checks only the status code still catches a missing required field, a wrong type or an undocumented status, because the validator checks everything the scenario did not. ## Examples are test data [Section titled “Examples are test data”](#examples-are-test-data) Documents already carry examples for their operations. Axx uses them as the starting payload (`a request payload using an application/json content example`), so a scenario lists only the values it is about. Two good effects follow: * The examples in your published documentation are exercised on every run, so they stay correct. * When a schema gains a field, you update the example once instead of every scenario. ## Strict by default, relaxed on purpose [Section titled “Strict by default, relaxed on purpose”](#strict-by-default-relaxed-on-purpose) Every validation rule starts at `ERROR`. A negative test, which sends an invalid request on purpose to check that the service rejects it, relaxes exactly the rule it breaks, in that scenario only: ```gherkin Given the OpenAPI validation levels are: | validation.request.body.schema.maximum | IGNORE | ``` That line comes from a scenario that sends a 31 kg parcel to check that the service refuses parcels over 30 kg: the request breaks the document’s `maximum`, and nothing else is relaxed. The relaxation is visible in the feature file, next to the behavior that needs it. Suite-wide relaxations in `axx.yaml` exist for documents you do not control; using them for your own contract defeats the point. ## Both sides of a dependency [Section titled “Both sides of a dependency”](#both-sides-of-a-dependency) A contract has two sides, and Axx checks both. Your service’s own document is checked by the REST steps: the provider side, where your service keeps its promise to its clients. The documents of the services your service calls are checked by the WireMock image that mocks them: the consumer side, where your service calls its dependencies correctly and the mocks answer as the real services would, so a stub cannot promise something the real API never does. The two sides have separate settings. Relaxing a rule of your own contract for a negative test says nothing about a dependency’s contract, and a stub that answers off-contract on purpose says nothing about yours. The provider validates its implementation against its document; the consumer validates its calls and mocks against the same document. Neither side needs the other’s tests. See [Validate against OpenAPI](/guides/validate-openapi/) for the steps and settings, and [Mock dependencies](/guides/mock-dependencies/) for the consumer side. # Scenario isolation and test data > Why Axx shares infrastructure between scenarios but never data, how that makes parallel runs safe, and the tools that enforce it. A fast black-box suite shares infrastructure: one database, one broker, one WireMock, one running service for hundreds of scenarios running in parallel. It stays correct only if scenarios never share **data**. Axx is built around that rule. ## What is isolated for you [Section titled “What is isolated for you”](#what-is-isolated-for-you) Each scenario has its own **world**: the services it registered, the requests it built and the responses it received, its selections and its events. Nothing in the world leaks into another scenario, and a scenario always runs on a single worker. ## What you isolate [Section titled “What you isolate”](#what-you-isolate) Everything outside Axx persists: rows in the database, documents in MongoDB, events on topics, requests in the WireMock journal. Those are visible to every scenario, including ones running at the same moment and ones in the next run. So: 1. **Every scenario owns its data.** References, ids, keys and emails belong to one scenario: `PX-REG-1004`, not `test`. A scenario that registers a parcel for the sender `shop-example` and then counts that sender’s parcels will pass alone and fail beside any other scenario that does the same. 2. **Assert on your own data only.** Select rows by your ids; name mock request patterns by URLs that contain your ids; consume events by your keys. 3. **Do not rely on cleanup.** Data from earlier runs, failed and interrupted ones included, is still there. A scenario must pass beside everyone else’s data. A scenario that inserts a fixed id (a seed, a registration with a fixed reference) needs that id to be free, so it runs again only on a fresh environment: `axx run` starts from empty databases, and against `axx up`, restart with `axx down` and `axx up`. ## Enforcing it [Section titled “Enforcing it”](#enforcing-it) Conventions drift, so Axx makes them checkable: * **`axx lint`** extracts ids from seeds, payloads and features and fails when a value appears where it must be unique ([Isolate test data](/guides/isolate-test-data/)). * **Fixture identities**: an `identity:` in a factory spec derives unique values for every fixture and generates the matching lint rule ([Fixture factories](/guides/fixture-factories/)). * **Randomized order**: `axx run --order random` exposes scenarios that only pass after another one ran. ## When sharing is unavoidable [Section titled “When sharing is unavoidable”](#when-sharing-is-unavoidable) Some scenarios change the environment for everyone: a database trigger that makes inserts fail, a global feature flag, a dependency that is stopped on purpose. Tag them with a tag listed in `run.exclusive` (for example `@isolated`). Axx runs them one at a time after the parallel phase, so they never overlap with another scenario ([Run in parallel](/guides/parallel-runs/)). ## Why not reset the database between scenarios [Section titled “Why not reset the database between scenarios”](#why-not-reset-the-database-between-scenarios) Truncating tables or restoring snapshots makes scenarios serial (a reset in one breaks another that is running), slow, and blind to problems that only appear with realistic data volumes. Unique data costs a naming convention and gives you parallel runs against a long-lived environment, including `axx up` sessions that last all day. # Step design > Why Axx steps are shaped the way they are - variants, parameter types, tables versus arguments - and why step text never changes. The steps of Axx’s packs follow a few rules, so once you know one pack you can guess the others, and an agent can too. ## One step, several variants [Section titled “One step, several variants”](#one-step-several-variants) Most steps come in up to four forms: the plain step, a form that names the service, a form with an ordinal for the 2nd or 3rd request, selection or event, and both together. The plain form covers the common case, a scenario with one service and one request, so most scenarios read like the acceptance criterion. The other forms appear only when a scenario needs them, and they read the same way in every pack ([How steps read](/references/steps/) has the grammar): ```gherkin Scenario: A reference can only be registered once Given a 1st ordered POST request to /api/parcels And a request payload using an application/json content example for 1st ordered request And the request payload property reference is 'PX-REG-1004' for 1st ordered request And a 2nd ordered POST request to /api/parcels And a request payload using an application/json content example named 'Express parcel' for 2nd ordered request And the request payload property reference is 'PX-REG-1004' for 2nd ordered request When the 1st ordered request is executed And the 2nd ordered request is executed Then the 1st ordered response status code is 201 And the 2nd ordered response status code is 409 ``` ## Parameter types [Section titled “Parameter types”](#parameter-types) Besides Cucumber’s built-in types (`{int}`, `{word}`, `{string}`, …), Axx adds types that let steps read naturally: `a 2nd selection`, `within 5s`, `on parcels`. `{filepath}` marks a value that names a file, such as a seed or a schema, so editors can link it to the file. They are listed in [How steps read](/references/steps/#parameter-types). ## Tables for many values, arguments for one [Section titled “Tables for many values, arguments for one”](#tables-for-many-values-arguments-for-one) Steps that set or check one value take it inline (`the request header Accept is 'application/json'`); their plural twins take a two-column table (`the request headers are:`). Tables use JSONPath keys for payloads, so nested and array values need no extra steps. ## Step text is public API [Section titled “Step text is public API”](#step-text-is-public-api) Once a step is released, its text never changes. New behavior gets new steps; old steps can be deprecated with a pointer to the replacement, never renamed. That is what keeps feature files working across releases, and what lets an agent trust `axx steps search` today and next year. Every built-in step is checked against a frozen catalog of step text on every build. ## Designing your own steps [Section titled “Designing your own steps”](#designing-your-own-steps) The same rules make [custom steps](/guides/write-custom-steps/) easy to use: * Give every step a stable `id`, a doc string and a complete example line. * Add an `on {service}` variant when a step talks to a service that can appear twice. * Report failed expectations with expected and actual values, not just a message. # Check logs > Prove that something did not happen by asserting the log entry that says so, reading logs from files or receiving them over UDP, TCP or HTTP. It is hard to prove that something did not happen: waiting and seeing nothing proves only that nothing arrived yet. Flip it to a positive. Have your service log its decision (“registration refused; not announced”, “line ML-KES-0413-1 rejected: duplicate reference”) and assert that entry. The `logs` steps read what your services log, whether it lands in a file or is sent over the network. ## Register a log [Section titled “Register a log”](#register-a-log) A log is where a service’s lines are. Register it in the `Background`, with a `url`: ```gherkin Background: Given the parcels log with the following properties: | url | udp://0.0.0.0:5140 | And the console log with the following properties: | url | file://.axx/logs/apps.log | ``` | url | Axx | | ------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `file:///var/log/app.log`, `file://logs/app.log` | reads what is appended to the file; a relative path is relative to `axx.yaml` | | `udp://0.0.0.0:5140` | listens; each datagram is one or more lines | | `tcp://0.0.0.0:5150` | listens; newline-delimited or octet-counted syslog (RFC 6587) messages | | `http://0.0.0.0:5160/logs`, `https://0.0.0.0:5161/logs` | listens; the body of each POST or PUT to that path (https uses a self-signed certificate) | A log only counts what it receives after the scenario registers it. Scenarios run in parallel and share logs, so match on data unique to the scenario: a reference, an order id. For the network schemes Axx is the log service: your service, or the thing that ships its logs, sends lines to Axx. Axx opens those listeners before it starts the apps, for the log steps of the scenarios in the run, so nothing sent during startup is lost; `axx up` keeps them open between runs. Containers reach them at `host.docker.internal` (on Linux, add `extra_hosts: ["host.docker.internal:host-gateway"]` to the service). ## Point your logs at Axx [Section titled “Point your logs at Axx”](#point-your-logs-at-axx) **The console of the apps Axx starts** needs nothing: Axx writes it to `.axx/logs/apps.log`, every compose container’s lines prefixed with its name, so `file://.axx/logs/apps.log` reads it. **A log file** a service writes, for example to a mounted volume: infra/compose.yaml ```yaml app: volumes: - ./logs:/var/log/parcels ``` ```gherkin Given the parcels log with the following properties: | url | file://../infra/logs/app.log | ``` **Syslog over UDP or TCP.** An app’s syslog handler (Python’s `SysLogHandler`, logback’s `SyslogAppender`) can send to `host.docker.internal:5140`, and so can Docker for any container’s console, without changing the app: infra/compose.yaml ```yaml app: logging: driver: syslog options: syslog-address: tcp://host.docker.internal:5150 ``` The example’s service sends every log line as a UDP datagram when `PARCELS_LOG_UDP` is set, which is what its `parcels` log receives. **A log forwarder over HTTP**, such as Fluent Bit or Vector posting to Axx: infra/fluent-bit.conf ```ini [OUTPUT] Name http Match * Host host.docker.internal Port 5160 URI /logs Format json_lines ``` ## Assert entries [Section titled “Assert entries”](#assert-entries) Patterns are regular expressions (Java syntax), searched in the log’s text rather than matched against whole lines. Every step waits for its entries: 10 seconds, or the time `within` gives. ```gherkin Then the parcels log has an entry matching 'msg="registration refused; not announced" reference=PX-EVT-4003' Then within 30s the parcels log has an entry matching 'depot notified reference=PX-DSP-2001' ``` Several entries in one log; each row needs a match of its own, so the same pattern twice needs two matches: ```gherkin Then the console log has entries matching: | msg="manifest line processed" line=ML-FJORD-0101-1 status=REJECTED reason="weight exceeds 30000 g" | | msg="manifest line processed" line=ML-FJORD-0101-2 status=REJECTED reason="unknown service level" | ``` A number of matches, for example one per retry; more than that fails: ```gherkin Then the parcels log has 2 entries matching 'msg="storing parcel failed, retrying" reference=PX-DBF-3002' ``` Entries in different logs, for example your service’s decision and the call its dependency received: ```gherkin Then the logs have entries matching: | parcels | msg="registration refused; not announced" reference=PX-ADR-1104 | | console | Request received:\n.*GET /v1/postcodes/DE/00012 | ``` ## Multi-line entries [Section titled “Multi-line entries”](#multi-line-entries) Patterns match across the whole text, so an entry can span lines: a stack trace, or WireMock’s request log above. `^` and `$` match at the start and end of each line, `\n` matches a line break, and `(?s)` lets `.` match line breaks too: ```gherkin Then the parcels log has an entry matching '(?s)registration failed reference=PX-7.*IllegalStateException: address service unavailable' ``` In a table cell, Gherkin turns `\n` into a real line break, which matches a line break just the same. When a step fails, it shows what the log received since the scenario registered it, so you can see what your service said instead. # Configure services > Register the REST APIs, mocks, databases and brokers a scenario talks to, name them, and keep URLs and credentials out of feature files. A **service** is something a scenario talks to: your REST API, a WireMock server, a SQL or MongoDB database, a Kafka cluster. Scenarios register services by name, usually in a `Background`, and later steps use them. ## Register services in a Background [Section titled “Register services in a Background”](#register-services-in-a-background) ```gherkin Feature: Register parcels Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | And the mocked addresses service with the following properties: | url | http://localhost:8081 | And a parcels-db database with the following properties: | url | postgres://localhost:5432/parcels | | user | parcels | | password | parcels | And a tracking-db mongo database with the following properties: | url | mongodb://localhost:27017/parcels?authSource=admin | | user | parcels | | password | parcels | And the events kafka service with the following properties: | brokers | localhost:9092 | ``` Each kind of service has its own registration step and properties, listed on its pack’s page: [REST](/references/steps/rest/) (an `openapi` property turns on [validation](/guides/validate-openapi/)), [Mocks](/references/steps/mock/), [SQL](/references/steps/sql/), [MongoDB](/references/steps/mongo/) and [Kafka](/references/steps/kafka/). ## The default service and named services [Section titled “The default service and named services”](#the-default-service-and-named-services) The first service of a type registered in a scenario is the default. Steps without a service name use it: ```gherkin Given a GET request to /api/parcels/PX-REG-1001 When the request is executed Then the response status code is 200 ``` When a scenario uses two services of the same type, name the one you mean with the `on ` form of the step: ```gherkin Given a GET request to /api/parcels/PX-REG-1001 on parcels When the request is executed on parcels Then the response status code is 200 on parcels ``` Most steps that address a service have both forms (a few mock steps always name it); see [How steps read](/references/steps/#services). ## Keep values out of feature files [Section titled “Keep values out of feature files”](#keep-values-out-of-feature-files) Values in service tables and in `axx.yaml` are interpolated ([syntax](/references/config/#interpolation)). Put what differs between machines, and every secret, in properties and environment variables: axx.yaml ```yaml properties: local.host: localhost db.password: ${env:PARCELS_DB_PASSWORD:-parcels} ``` ```gherkin Given a parcels-db database with the following properties: | url | postgres://${sys:local.host}:5432/parcels | | user | parcels | | password | ${sys:db.password} | ``` Override a property for one run with `axx run -D local.host=docker`. ## Differences between environments [Section titled “Differences between environments”](#differences-between-environments) Use a **profile** for settings that differ in CI or on another machine: axx.yaml ```yaml profiles: ci: properties: local.host: docker ``` Select it with `axx run --profile ci` or `AXX_PROFILE=ci`. A profile can also live in its own file, and personal overrides in an `axx.local.yaml`; [Finding and merging files](/references/config/#finding-and-merging-files) has the order. ## Resource paths [Section titled “Resource paths”](#resource-paths) Steps that take a file (`seeds/manifest-kestrel.yaml`, `kafka/scan-delivered.json`, `schemas/depot-scan.avsc`) resolve it against the `resources` directories in order, then the directory of `axx.yaml`: axx.yaml ```yaml resources: [acceptance, ../shared-fixtures] ``` If a file is not found, the step fails with [`AXX-E0301`](/references/error-codes/#axx-e0301) and lists the directories it searched. # Debug failures > Go from a red axx run to the cause - read the exit code, rerun one scenario, explain a step, read expected and actual values, find app logs, and fix flaky parallel scenarios. ## Start from the exit code [Section titled “Start from the exit code”](#start-from-the-exit-code) | Code | What happened | First move | | ----- | ---------------------------- | --------------------------------------------------------------------------------- | | `1` | a scenario failed | read the failing step’s expected and actual values | | `2` | usage or configuration error | read the `hint`; config errors point at `axx.yaml:line:col`; run `axx doctor` | | `3` | undefined or ambiguous step | `axx validate`, then `axx explain ""` | | `4` | an app did not start or stop | the error shows the app’s last output lines; the full log is `.axx/logs/apps.log` | | `130` | interrupted | nothing completed; run again | Every error also has a stable code (`AXX-E0408`, say) that links to its entry in the [error code reference](/references/error-codes/). ## Read the failure [Section titled “Read the failure”](#read-the-failure) ```console $ axx run ... Scenario: The service says hello # features/smoke.feature:7 ✓ Given the hello-axx service with the following properties: (background) | url | http://${sys:local.host}:8000 | ✓ Given a GET request to /hello.json ✓ When the request is executed log: GET http://localhost:8000/hello.json -> 200 OK (2ms, 26 bytes) attachment: response body (application/json, 26 B) {"message": "Hello, axx"} ✓ Then the response status code is 200 ✗ And the response payload property message is 'Hello, world' Response payload property message is not Hello, world expected: "Hello, world" actual: "Hello, axx" Failed scenarios: ✗ The service says hello # features/smoke.feature:7 rerun: axx run features/smoke.feature:7 ``` Each failure names the step, the expected and actual values, and the command that reruns only that scenario. Steps that talk to a service attach what they sent and received. For the same failure as data, with the last request and response, use `axx run --json` ([the run report](/references/json-output/#the-run-report)). ## Rerun one scenario [Section titled “Rerun one scenario”](#rerun-one-scenario) ```sh axx up # keep the apps running while you iterate axx run features/register-parcels.feature:15 # the scenario on (or containing) line 15 ``` The line of any step inside the scenario works too. ## Undefined and ambiguous steps [Section titled “Undefined and ambiguous steps”](#undefined-and-ambiguous-steps) ```console $ axx explain 'the mocked request named postcode-check was received 2 times' undefined: no step matches did you mean: the mocked request named {word} was received exactly {int} time(s) (mock.count.exactly) the mocked request named {word} was received at least {int} time(s) (mock.count.atLeast) the mocked request named {word} was received at most {int} time(s) (mock.count.atMost) ... search all steps with `axx steps search ` ``` * **Undefined**: the text differs from every step. Use the suggestion or `axx steps search`. Look for extra spaces, `a` versus `an`, singular versus plural, and missing quotes around `{string}` values. Never change a step definition to match your text. * **Ambiguous**: two steps match. Make the line more specific, usually by naming the service with `on `. ## Common causes [Section titled “Common causes”](#common-causes) * **Expected X, got Y, and the product looks right.** Another scenario running at the same time may have used the same ids. Check that the data is unique (`axx lint`), then check the value’s type: the step’s entry in the [step reference](/references/steps/) says how it reads values (`'5'` and `'"5"'` differ). * **OpenAPI validation error on `the request is executed`.** The request or the response violates the document; the message names the rule, such as `validation.response.body.schema.required`. Fix the payload or the service. For a deliberate negative test, relax that rule in the scenario ([Validate against OpenAPI](/guides/validate-openapi/#validation-levels)). * **Timeout.** A step exceeded `run.timeouts.step`. For asynchronous behavior use a polling step (`within 10s a selection of at least 1 row ...`) instead of a sleep or a longer timeout. * **Passes alone, fails in the full run.** Shared data or shared state. Make the data unique; if the scenario really must run alone, tag it for `run.exclusive` ([Run in parallel](/guides/parallel-runs/)). * **The app never becomes ready.** Check `apps..ready` (URL, port, timeout). Run `axx up`, then `curl` the health URL yourself, and read `.axx/logs/apps.log`. ## Debug the service itself [Section titled “Debug the service itself”](#debug-the-service-itself) Set a breakpoint in your service under test and run the scenario against it: ```sh axx run --attach parcels features/register-parcels.feature:15 # you start parcels from your IDE; axx waits for it axx run --debug=parcels features/register-parcels.feature:15 # axx starts parcels with its debug command ``` [Manage the app lifecycle](/guides/manage-app-lifecycle/#debug-an-app) shows how to configure `apps..debug`. ## Stop in step code [Section titled “Stop in step code”](#stop-in-step-code) To see exactly what a step does, why it fails, or what your service sent back, set breakpoints in the step’s own Go code, whether it comes from one of Axx’s packs or from your own, and step through it: ```sh axx run --debug-steps features/register-parcels.feature:15 ``` Axx builds itself with debug information (the first time; later runs reuse the build), starts under [Delve](https://github.com/go-delve/delve), Go’s debugger, and waits for a debugger on port 2345 (`--debug-steps=` picks another). The scenario starts when one attaches. Step timeouts are off while you debug, and the exit code is the run’s as usual. * **GoLand, or IntelliJ IDEA with the Go plugin:** click the gutter icon of a scenario and choose *Debug*. The axx plugin attaches the Go debugger for you. From a terminal run, start the *Debugger: axx-steps* configuration (written by `axx ide intellij`). * **VS Code with the Go extension:** use the debug button of a scenario in the gutter or the Testing view. To attach to a run you started in a terminal, use *axx: attach to steps* (written by `axx ide vscode`). * **Anything else:** `dlv connect 127.0.0.1:2345`. Jump from a step in a feature file to its code with go-to-definition ([Set up your editor](/guides/set-up-your-editor/)). Steps of Axx’s packs open in axx’s source, which the debug build is compiled from, so breakpoints set there hold. It needs Go, to build, and Delve: `go install github.com/go-delve/delve/cmd/dlv@latest`. ## Logs [Section titled “Logs”](#logs) | File | Contents | | -------------------- | ------------------------------- | | `.axx/logs/apps.log` | output of every app Axx started | Add `-v` or `-vv` to any command for more detail from Axx itself. ## Don’t [Section titled “Don’t”](#dont) * Don’t loosen an assertion to make a test pass. Find out whether the product is wrong first. * Don’t add fixed sleeps. Use readiness checks and polling steps. * Don’t edit generated files (`axx-lint.generated.yaml`, generated fixtures); change their source and regenerate. # Fixture factories > Generate schema-valid seeds, event payloads and mock bodies from a compact factory spec with axx fixtures, and catch drift when a schema changes. Suites accumulate dozens of fixture files that share one schema: Kafka payloads for one Avro record, seed datasets for one set of tables, mock bodies for one API. Most of each file is boilerplate, and the day the schema gains a required field, every file needs the same edit. A **factory** keeps the shared shape once and each fixture as only its differences. `axx fixtures` expands them into ordinary files. ## The files [Section titled “The files”](#the-files) Everything lives next to the fixtures it produces: ```text kafka/ depot-scans.factory.yaml # family, schema, identities depot-scans.prototype.yaml # the shared shape scan-delivered.fixture.yaml # one fixture: only its differences scan-out-for-delivery.fixture.yaml scan-delivered.json # generated scan-out-for-delivery.json # generated ``` kafka/depot-scans.factory.yaml ```yaml # yaml-language-server: $schema=https://axx.nimbusxr.us/schemas/v0/axx-factory.schema.json factory: family: avro schema: ../schemas/depot-scan.avsc # relative to this file identity: - path: scanId # unique across every fixture ``` kafka/depot-scans.prototype.yaml ```yaml data: parcelRef: PX-EXAMPLE-1 location: Leipzig scannedAt: "2026-05-06T10:15:00Z" ``` kafka/scan-delivered.fixture.yaml ```yaml data: scanId: SC-FIXTURE-DELIVERED status: DELIVERED ``` The fixture’s file name is its name, and `scan-delivered.json` is what features reference, exactly as if you had written it by hand: kafka/scan-delivered.json (generated) ```json { "scanId": "SC-FIXTURE-DELIVERED", "parcelRef": "PX-EXAMPLE-1", "status": "DELIVERED", "location": "Leipzig", "scannedAt": "2026-05-06T10:15:00Z" } ``` ## Generate and check [Section titled “Generate and check”](#generate-and-check) ```sh axx fixtures generate # write the fixture files, the manifest and lint rules axx fixtures check # CI: regenerate in memory and compare, without writing ``` * **Generation is a development-time step.** Nothing is generated during `axx run`; features cannot tell a generated fixture from a hand-written one. * **Output is deterministic.** The same spec always produces the same bytes, and every output is validated against the schema (the Avro schema here, so a `status` that is not one of its symbols fails) before it is written. * **The tool only touches files it owns.** `axx-fixtures.manifest.yaml` records them with their checksums. A managed file that was edited by hand is refused, never overwritten: move the change into the spec. * **Identities become lint rules.** Each `identity:` entry produces a rule in `axx-lint.generated.yaml`, which `lint.include` pulls into [`axx lint`](/guides/isolate-test-data/). A fixture that omits an identity value gets one derived from its name, so it is unique by construction. When the schema gains a required field, generation fails and names every fixture that lacks it. Add it once to the prototype (or a `defaults:` entry) and regenerate. ## Families [Section titled “Families”](#families) | Family | Produces | Schema | | ---------- | ----------------------------------------------------- | -------------------------------------------------------------------------- | | `avro` | Kafka payload JSON | an `.avsc` file | | `json` | any JSON: mock bodies, request payloads | JSON Schema, or an OpenAPI component (`api.yaml#/components/schemas/Name`) | | `yaml` | record-shaped YAML | the same as `json` | | `xml` | XML documents | an XSD | | `protobuf` | canonical proto-JSON | a descriptor set (`orders.desc#pkg.Message`) | | `dataset` | SQL seeds: YAML datasets, flat XML or CSV directories | optional SQL DDL (a file or a directory of migrations) | For `dataset`, the prototype is a row template per table, and identities are `table.column` paths enforced per row. `options: { format: xml }` or `{ format: csv }` in the factory writes flat XML or a CSV directory instead of YAML. The parcels example generates its rejected manifest lines as flat XML: seeds/manifests/xml/manifests-xml.factory.yaml ```yaml factory: family: dataset options: { format: xml } schema: ../../../../infra/postgres/init/01-schema.sql ``` ## Project settings [Section titled “Project settings”](#project-settings) axx.yaml ```yaml fixtures: sources: ["../infra/wiremock"] # extra roots, e.g. mock bodies served by a container output: { ignored: true } # generated files are gitignored, not committed conformance: # lint hand-written files against a schema too - name: seeds match the database schema filePatterns: ["seeds/*.yaml"] schemaType: dataset schemaRef: ../infra/postgres/init/01-schema.sql ``` With `output: { ignored: true }`, generated files are derived on demand instead of committed: Axx maintains an exact `.gitignore` in each output directory, and a fresh clone runs `axx fixtures generate` once before `axx run`. The specs, the manifest and the generated lint rules stay committed; they are what reviewers read. `conformance` rules check files that no factory owns, so hand-written seeds are held to the same schema. ## Adopt existing files [Section titled “Adopt existing files”](#adopt-existing-files) You do not need to write specs for fixtures you already have. Adoption reads existing files, writes the most common value of each field into the prototype, keeps each file’s differences in its own `*.fixture.yaml`, and proposes identity candidates for you to review. It only writes anything if regenerating from the new spec reproduces the originals exactly. # Install Axx > Install the Axx CLI with Homebrew, the install script, go install or the container image, verify the download and keep it up to date. Axx is a single static binary for Linux, macOS and Windows on amd64 and arm64. It has no runtime dependencies. Docker is only needed if the apps you test start with Docker Compose. Pre-release Axx is in beta. Until `v0.1.0` is published, install from source with `go install`. The other channels below go live with that release. Every `0.x` release is marked as a pre-release on GitHub. ## Choose a channel [Section titled “Choose a channel”](#choose-a-channel) ### Homebrew (macOS, Linux) [Section titled “Homebrew (macOS, Linux)”](#homebrew-macos-linux) ```sh brew install nimbusxr/tap/axx ``` ### Install script (Linux, macOS, CI) [Section titled “Install script (Linux, macOS, CI)”](#install-script-linux-macos-ci) ```sh curl -fsSL https://axx.nimbusxr.us/install.sh | sh ``` The script picks the newest release (pre-releases included, since every `0.x` release is one), downloads the archive for your OS and architecture, and checks it against `checksums.txt`. ### Go [Section titled “Go”](#go) ```sh go install github.com/nimbusxr/axx/cmd/axx@latest ``` Needs Go 1.27 or newer. The binary lands in `$(go env GOPATH)/bin`. ### Container image [Section titled “Container image”](#container-image) ```sh docker run --rm -v axx-cache:/home/nonroot -v "$PWD:/work" -w /work ghcr.io/nimbusxr/axx validate ``` The image contains only the `axx` binary (on a distroless base). The `axx-cache` volume keeps the packs Axx prepares for the project ([Choose packs](/guides/use-packs/)); without it, every run prepares them again. It suits commands that do not start apps (`validate`, `steps`, `explain`, `schema`) and runs against services that are already up (`axx run --no-start`). It cannot run `docker compose` for you. ### Release archives [Section titled “Release archives”](#release-archives) Download `axx___.tar.gz` (`.zip` on Windows) from [GitHub releases](https://github.com/nimbusxr/axx/releases), extract `axx` and put it on your `PATH`. A rolling `nightly` pre-release is built from `main` every day. ## Verify the download [Section titled “Verify the download”](#verify-the-download) Release checksums are signed with Sigstore (keyless), and archives carry an SBOM and build provenance. ```sh cosign verify-blob checksums.txt \ --bundle checksums.txt.sigstore.json \ --certificate-identity-regexp '^https://github.com/nimbusxr/axx/' \ --certificate-oidc-issuer https://token.actions.githubusercontent.com sha256sum --ignore-missing -c checksums.txt gh attestation verify axx_0.1.0_linux_amd64.tar.gz --repo nimbusxr/axx ``` ## Check the installation [Section titled “Check the installation”](#check-the-installation) ```sh axx version axx doctor ``` `axx doctor` checks the binary, `axx.yaml`, the step packs, your feature files and the commands your apps need. It exits `0` when nothing failed (warnings are allowed) and `4` otherwise. ## Shell completion [Section titled “Shell completion”](#shell-completion) ```sh axx completion zsh > "${fpath[1]}/_axx" # zsh axx completion bash > /etc/bash_completion.d/axx # bash axx completion fish > ~/.config/fish/completions/axx.fish ``` ## Editors [Section titled “Editors”](#editors) To see Axx’s steps in feature files as you write them, set up IntelliJ IDEA, VS Code or another editor with [Set up your editor](/guides/set-up-your-editor/). ## Upgrade [Section titled “Upgrade”](#upgrade) Upgrade with the channel you installed from (`brew upgrade`, rerun the script, or `go install ...@latest`). Step text never changes between versions, so feature files keep working. Before `v1.0.0`, a minor version bump (`0.1` to `0.2`) may change a flag or a configuration key; read the [changelog](https://github.com/nimbusxr/axx/blob/main/CHANGELOG.md) first. After upgrading, refresh the agent skills so their step index matches the new binary: ```sh axx skills install ``` # Isolate test data > Use axx lint to guarantee that ids, keys and names in seeds, payloads and features never collide, so scenarios can share infrastructure and run in parallel. Scenarios run in parallel against one database, one broker and one set of mocks. They stay independent only if each one owns its data. Two seed files that both insert the parcel `PX-KES-1001`, or two scenarios that both publish events keyed `PX-TRK-3002`, produce failures that come and go with scheduling. `axx lint` finds those collisions before a run does. ## Write rules [Section titled “Write rules”](#write-rules) Rules live under `lint:` in `axx.yaml`. Each one extracts values from files and says how unique they must be: axx.yaml ```yaml lint: config: baseDir: . # patterns are relative to this directory mode: error # error fails the lint; warn only reports rules: - name: Parcel references in seeds filePatterns: ["seeds/*.yaml"] regex: '^\s+-?\s*reference:\s*"([^"]+)"' description: Every seeded parcel and manifest line has its own reference validation: cross-file-unique - name: Manifest line ids filePatterns: ["seeds/*.yaml"] regex: '^\s+-?\s*id:\s*"([^"]+)"' validation: cross-file-unique - name: Depot scan ids filePatterns: ["seeds/*.json"] type: jsonpath jsonPath: "scans[*].scanId" validation: cross-file-unique ``` Each rule key is described in the [configuration reference](/references/config/#lint). Generated rules can be merged in with `include`: axx.yaml ```yaml lint: include: [axx-lint.generated.yaml] # written by `axx fixtures generate` ``` ## Run it [Section titled “Run it”](#run-it) ```sh axx lint ``` ```console FAIL Parcel references in seeds (cross-file-unique, 11 files): 1 duplicate value Every seeded parcel and manifest line has its own reference value "PX-KES-1001" appears in 2 files (cross-file-unique) [AXX-E0820] seeds/manifest-kestrel-resend.yaml:4:17 reference: "PX-KES-1001" seeds/manifest-kestrel.yaml:4:17 reference: "PX-KES-1001" ok Manifest line ids (cross-file-unique, 11 files) ok Depot scan ids (cross-file-unique, 1 file) ok kafka/depot-scans.factory.yaml: scanId uniqueness (cross-file-unique, 2 files) ok SQL selection and trigger ordinals (8 files) axx lint: 5 rules, 22 files: 1 error, 0 warnings ``` A new seed file reused a reference that `seeds/manifest-kestrel.yaml` already inserts. The rule from `axx-lint.generated.yaml` and the built-in check of SQL ordinals in features run too. `axx lint` exits with `3` when an `error`-mode rule finds a duplicate, like `axx validate` does for undefined steps. Run it in CI next to `axx validate`, and start new rules in `warn` mode while you clean up existing data. ## Tips [Section titled “Tips”](#tips) * **Prove a rule bites.** After writing a rule, plant a duplicate value and check that `axx lint` fails. A pattern that matches no files still passes (its line only says `no files matched`). * **Name values after the scenario.** `PX-DBF-3001` tells a reader which feature owns it; `test` tells nobody anything. * **Declare identities once.** With [fixture factories](/guides/fixture-factories/), an `identity:` in the factory spec generates the matching lint rule for you. * **Some scenarios cannot share.** A scenario that changes a table’s triggers affects everyone. Tag it with a tag from `run.exclusive` so it runs alone ([Run in parallel](/guides/parallel-runs/)). [Scenario isolation](/explanations/scenario-isolation/) explains the reasoning behind shared infrastructure with isolated data. # Manage the app lifecycle > Declare how Axx starts, waits for, stops and cleans up the apps under test, keep them running with axx up, attach apps you run yourself, and start only the apps a run needs. `apps:` in `axx.yaml` describes the system under test: your service and anything it needs, such as databases, brokers and mocks. `axx run` starts them, waits until they are ready, runs the scenarios, stops them and runs their cleanup. ## Declare an app [Section titled “Declare an app”](#declare-an-app) axx.yaml ```yaml apps: infra: dir: ./infra # relative to axx.yaml command: docker compose up --wait postgres kafka ready: tcp: localhost:5432 cleanup: docker compose down -v --remove-orphans api: dependsOn: [infra] command: ./gradlew bootRun # or npm start, go run ./cmd/api, ... env: SPRING_PROFILES_ACTIVE: acceptance ready: http: url: http://localhost:8080/actuator/health timeout: 120s interval: 1s stop: signal: SIGTERM grace: 20s ``` Each key is described in the [configuration reference](/references/config/#apps). A command that exits `0` before the app is ready is fine, which is what `docker compose up -d` or `--wait` does: Axx keeps polling the `ready` checks. App output goes to `.axx/logs/apps.log`. When an app fails to start, the error shows the last lines of its output and a stable code from [`AXX-E0400` to `AXX-E0414`](/references/error-codes/#app-lifecycle). ## Keep apps running between runs [Section titled “Keep apps running between runs”](#keep-apps-running-between-runs) ```sh axx up # start every enabled app (or: axx up api) and wait until ready axx run # reuses the running apps: no start, no stop axx run --tags @smoke axx down # stop the apps and run their cleanup ``` This is the fastest local loop, and the one agents should use. `axx down` also stops apps left behind by an interrupted run. ## Run an app yourself [Section titled “Run an app yourself”](#run-an-app-yourself) To run the service from your IDE (with breakpoints, hot reload, a profiler), tell Axx not to start it. Axx still starts everything else and waits for your app’s readiness checks: ```sh axx run --attach api ``` `--no-start` skips starting, stopping and cleaning up every app; use it when the whole system is already running somewhere. ## Debug an app [Section titled “Debug an app”](#debug-an-app) Give the app a debug command and a debugger to wait for: axx.yaml ```yaml apps: api: command: ./gradlew bootRun debug: command: ./gradlew bootRun -PappJvmArgs=-agentlib:jdwp=transport=dt_socket,server=n,address=localhost:5005,suspend=n debugger: type: java # java, go, nodejs or python port: 5005 mode: ide-listens # the IDE listens, the app connects (default for java) onUnavailable: retry # retry (default), fail, or fallback to the normal command retry: {attempts: 10, delay: 3s} ``` ```sh axx run --debug # every app with a debug block axx run --debug=api # only api axx up --debug=api ``` With `mode: ide-listens`, start a listening debugger in your IDE first (in IntelliJ IDEA: a *Remote JVM Debug* configuration in *Listen to remote JVM* mode on port 5005), then run Axx. With `mode: app-listens` (delve, `--inspect`, debugpy), the app opens the port and you attach; the [parcels example](https://github.com/nimbusxr/axx/tree/main/examples/parcels) runs its Go service under Delve this way. If no debugger is listening, `onUnavailable` decides whether to wait, fail with [`AXX-E0409`](/references/error-codes/#axx-e0409), or run without debugging. ## Start only what a run needs [Section titled “Start only what a run needs”](#start-only-what-a-run-needs) In a repository with several services, start only the apps the selected scenarios need: axx.yaml ```yaml active: enabled: true onNoTags: fallback # scenarios without tags: start every enabled app (or: error) apps: orders: command: ./gradlew :orders:bootRun active: {tags: ["@orders"]} billing: command: ./gradlew :billing:bootRun active: {tags: ["@billing", "@payments"]} wiremock: command: docker compose up wiremock # no active.tags: always starts ``` An app with `active.tags` starts when any selected scenario carries one of its tags. Apps without `active.tags` always start. `axx run --tags @billing` starts `billing` and `wiremock`, not `orders`. # Mock dependencies > Replace the services your service calls with WireMock, verify the requests it sent, and check both sides of each dependency's OpenAPI contract with the axx WireMock image. Your service calls other services. In an acceptance test you replace them with [WireMock](https://wiremock.org/) mocks, which gives you two things: the responses are under your control, and you can check exactly what your service sent. ## Define the stubs [Section titled “Define the stubs”](#define-the-stubs) Stubs are ordinary WireMock mapping files. Axx does not create stubs; it verifies traffic. Mount the mappings into a WireMock container: infra/wiremock/mappings/postcode-undeliverable.json ```json { "name": "postcode-undeliverable", "priority": 1, "request": { "method": "GET", "urlPathPattern": "/v1/postcodes/[A-Z]{2}/999[^/]*" }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": "{\"postcode\": \"{{request.pathSegments.[3]}}\", \"country\": \"{{request.pathSegments.[2]}}\", \"deliverable\": false, \"reason\": \"no delivery to this postcode\"}", "transformers": ["response-template"] } } ``` infra/compose.yaml (excerpt) ```yaml services: address-service: image: wiremock/wiremock:3.13.0 ports: ["8081:8080"] volumes: ["./wiremock:/home/wiremock"] ``` Configure your service to call `http://localhost:8081` (or the compose service name) instead of the real dependency. ## Verify what your service sent [Section titled “Verify what your service sent”](#verify-what-your-service-sent) Register the mock in the scenario, trigger the behavior, then check the requests WireMock received: ```gherkin Feature: Address check Background: Given the parcels service with the following properties: | url | http://localhost:8400 | And the mocked addresses service with the following properties: | url | http://localhost:8081 | Scenario: The address service is asked with the API key Given a POST request to /api/parcels And a request payload using an application/json content example And the request payload properties are: | reference | PX-ADR-1101 | | recipient.postcode | "53111" | When the request is executed Then the response status code is 201 And the mocked GET request to /v1/postcodes/DE/53111 named postcode-check was received by addresses And the mocked request named postcode-check was received exactly 1 time And the header X-Api-Key for mocked request named postcode-check on addresses is 'example-address-key' ``` `the mocked GET request to /v1/postcodes/DE/53111 named postcode-check was received by addresses` registers a request pattern (method and exact URL, including the query string) under a name you choose, and checks it was received at least once. Later steps refer to it by that name; the [mock step reference](/references/steps/mock/) has the ones that check counts, absence and headers. ```gherkin Scenario: Invalid registrations never reach the address service Given the OpenAPI validation levels are: | validation.request.body.schema.minimum | IGNORE | And a POST request to /api/parcels And a request payload using an application/json content example And the request payload properties are: | reference | PX-ADR-1103 | | weightGrams | 0 | | recipient.postcode | "12489" | When the request is executed Then the response status code is 400 And the mocked GET request to /v1/postcodes/DE/12489 named skipped-check was not received ``` Journals persist WireMock keeps its request journal between scenarios, and between runs for as long as it keeps running, and scenarios run in parallel. Match on something unique to the scenario (a postcode or an id in the URL, a header) so one scenario never counts another scenario’s requests. See [Isolate test data](/guides/isolate-test-data/). ## Check the dependency’s contract [Section titled “Check the dependency’s contract”](#check-the-dependencys-contract) A mock that answers something the real API never would makes a test pass for the wrong reason, and a service that calls its dependency wrongly only finds out in production. The `ghcr.io/nimbusxr/axx-wiremock` image is WireMock with Axx’s OpenAPI validation extension. It checks every call to the mock against the dependency’s OpenAPI document: your service’s request, and the stub’s response. infra/compose.yaml (excerpt) ```yaml services: address-service: image: ghcr.io/nimbusxr/axx-wiremock:0.1 ports: ["8081:8080"] environment: OPENAPI_SPEC_SOURCE: /var/openapi/address-service.yaml OPENAPI_VALIDATION_MODE: report volumes: - ./wiremock:/home/wiremock - ./openapi:/var/openapi:ro ``` `OPENAPI_SPEC_SOURCE` is the dependency’s document (a path inside the container or a URL). A stub can name another one with `"metadata": {"openApiSpecSource": "..."}`, or opt out with `"openApiValidation": false`. In `report` mode your service gets the stub’s answer unchanged, as it would from the real dependency, and Axx reports what broke the contract: * A mock step that checks the call fails, and names the rule: ```console ✗ And the mocked GET request to /v1/postcodes/DE/44444 named zone-lookup was received by addresses The GET /v1/postcodes/DE/44444 request named zone-lookup broke the contract of the mocked addresses service (/var/openapi/address-service.yaml): - validation.response.body.schema.additionalProperties (response): property 'surcharge' is not defined in the schema and the schema does not allow additional properties ``` `(response)` means the stub broke the contract; `(request)` means your service did. * A call that breaks the contract and that no scenario checks fails the run, after the scenarios: ```console Run failures: ✗ Calls to the mocked addresses service (http://localhost:8081) broke its contract, and no scenario checked them: (mock) GET /v1/postcodes/DE/55555 (/var/openapi/address-service.yaml) - validation.response.body.schema.additionalProperties (response): property 'surcharge' is not defined in the schema and the schema does not allow additional properties ``` Axx only reads WireMock’s request journal, so scenarios running in parallel cannot affect each other’s checks. It reads the journals of the mocks the run’s scenarios register: register the mocked service in the `Background` of every feature whose scenarios make the calls, so Axx knows to look. ### Relax a check [Section titled “Relax a check”](#relax-a-check) Every rule is an `ERROR` until you relax it. Relaxing is how you tell Axx a deviation is known: `WARN` logs the finding on the checking step instead of failing it, `INFO` and `IGNORE` drop it. The rules have the same keys as your own service’s contract (`validation.request.body.schema.required`, `validation.response.status.unknown`, …), and a key also covers the keys below it. The settings are separate: [`the OpenAPI validation levels are:`](/guides/validate-openapi/) never relaxes a dependency’s contract. Relax where the deviation lives: | Where | Relaxes | Use it for | | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `"openApiValidationLevels": {"validation.response.body": "WARN"}` in a stub’s metadata | the calls that stub answers, in every scenario and in the end-of-run check | a stub whose answer is off-contract wherever it is used | | `OPENAPI_VALIDATION_LEVELS` on the container, e.g. `validation.response.body.schema.additionalProperties=WARN` | every call to that mock | a known gap between the dependency’s document and its behavior | | the scenario step below | the calls that scenario’s mock steps check | a scenario that exercises an off-contract call on purpose, next to the behavior it checks | In the example, the address service starts sending a field its document does not list yet, and quoting must keep working. The relaxation sits in that scenario, which checks the call: ```gherkin Scenario: Quotes keep working when the address service sends a field it has not documented Given the OpenAPI validation levels for the mocked addresses service are: | validation.response.body.schema.additionalProperties | WARN | And a POST request to /api/quotes And a request payload using an application/json content example named 'Domestic parcel' And the request payload property recipient.postcode is '"80995"' When the request is executed Then the response status code is 200 And the response payload property zone is 'DE-8' And the mocked GET request to /v1/postcodes/DE/80995 named zone-lookup was received by addresses ``` The scenario step only reaches calls the scenario checks with a mock step: the end-of-run check cannot tell which scenario made a call. When a scenario relaxes a mocked service but checks none of its calls, Axx warns at the end of that scenario, and if one of its calls breaks the rule anyway, the run report names the scenario that meant to relax it. The more specific place wins. Without the extension’s `report` mode (`fail`, its default), a call that breaks the contract also gets an HTTP 500 listing the findings; Axx reports it the same way. All settings, the admin endpoints and how the image is versioned are in the [extension’s README](https://github.com/nimbusxr/axx/tree/main/extensions/wiremock-openapi). Keep mock response bodies schema-valid as the provider’s API evolves with [fixture factories](/guides/fixture-factories/): the `json` family can use an OpenAPI component as its schema, which is how the example generates `postcode-remote.json` from `infra/wiremock/__files/postcodes.factory.yaml`. # Run in parallel > Control how many scenarios run at once, keep scenarios that cannot share infrastructure on their own with run.exclusive, and randomize order to flush out hidden dependencies. Axx runs scenarios in parallel by default, one per CPU. Parallelism is what keeps a suite of hundreds of black-box scenarios fast, and it only works if scenarios do not step on each other. ## Workers [Section titled “Workers”](#workers) axx.yaml ```yaml run: workers: auto # default: the number of CPUs; or a number ``` ```sh axx run --workers 4 axx run -w 1 # one at a time, for debugging an ordering problem ``` Each scenario runs start to finish on one worker. Steps inside a scenario always run in order. ## Scenarios that must run alone [Section titled “Scenarios that must run alone”](#scenarios-that-must-run-alone) Some scenarios change shared state in a way no other scenario can tolerate: they install database triggers, change a feature flag for the whole service, or restart a dependency. Tag them and list the tag in `run.exclusive`: axx.yaml ```yaml run: exclusive: ["@isolated"] ``` ```gherkin @isolated Feature: Database failures Background: Given a parcels-db database with the following properties: | url | postgres://localhost:5432/parcels | | user | parcels | | password | parcels | Scenario: A brief database failure does not fail the registration Given a before insert trigger on the parcels.parcels table will raise a 40001 exception 1 time where: | reference | PX-DBF-3001 | ``` Axx runs every other scenario in parallel first, then the exclusive ones one at a time. Tags are inherited, so tagging the feature covers every scenario in it. ## Order [Section titled “Order”](#order) ```sh axx run --order random # a new random order each run axx run --order random:4242 # repeat a specific order ``` `run.order` in `axx.yaml` sets the default (`defined` unless you change it). A suite that passes in `defined` order but fails in a random one has scenarios that depend on each other. ## Timeouts [Section titled “Timeouts”](#timeouts) axx.yaml ```yaml run: timeouts: step: 60s # default 10m scenario: 5m # default: none hook: 2m # default 2m ``` Durations are strings such as `90s` or `5m`, or a number of seconds. A step that runs over is cancelled and reported with `error.kind: timeout`. For behavior that takes time, use a polling step (`within 10s ...`) instead of a longer timeout. ## When parallel runs fail [Section titled “When parallel runs fail”](#when-parallel-runs-fail) A scenario that passes alone (`axx run features/x.feature:12`) but fails in a full run almost always shares data with another scenario. In order of preference: 1. Give it unique data, and enforce that with [`axx lint`](/guides/isolate-test-data/). 2. Assert on its own data only (filter selections by its ids, name mock request patterns by its URLs). 3. If it truly cannot share, tag it for `run.exclusive`. Lowering `--workers` hides the problem instead of fixing it. # Reports > Choose console output, write JUnit, HTML, Cucumber JSON, Cucumber Messages and agent reports, and get machine-readable results with --json. ## Console output [Section titled “Console output”](#console-output) | Format | What you see | When | | ---------- | ---------------------------------------------------------- | ----------------------------------------- | | `pretty` | every scenario and step, tables, logs, failures, a summary | the default in a terminal | | `progress` | one character per scenario, then failures and a summary | long suites | | `compact` | failures only, and one summary line | automatic for coding agents (`--compact`) | ```console $ axx run --format progress features/register-parcels.feature ...F.... Feature: Register parcels Scenario: A reference can only be registered once # features/register-parcels.feature:67 ... Failed scenarios: x A reference can only be registered once # features/register-parcels.feature:67 rerun: axx run features/register-parcels.feature:67 8 scenarios (1 failed, 7 passed) 96 steps (1 failed, 1 skipped, 94 passed) Finished in 0.6s ``` Problems that belong to no single scenario are listed under `Run failures:` after the failed scenarios, and fail the run. The one Axx reports today is a call to a [mocked dependency](/guides/mock-dependencies/#check-the-dependencys-contract) that broke its contract when no scenario checked the call. JUnit reports them as an `axx run` test suite, the agent report as `runErrors`, and Cucumber Messages in `TestRunFinished`. `--compact` is turned on automatically when an agent is detected (`CLAUDECODE`, `CODEX_SANDBOX`, `GEMINI_CLI`, `CURSOR_AGENT`, `AGENT` or `AI_AGENT` is set) and stdout is not a terminal. `--no-color` or `NO_COLOR` turns colors off. ## Report files [Section titled “Report files”](#report-files) Add `--format NAME:FILE` once per report: ```sh axx run \ --format pretty \ --format junit:build/axx/junit.xml \ --format html:build/axx/report.html ``` | Name | Output | Use it for | | --------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------- | | `junit` | JUnit XML, one test case per scenario | CI test tabs (GitHub, GitLab, Jenkins) | | `html` | a single self-contained HTML report | humans, as a CI artifact | | `cucumber-json` | the classic Cucumber JSON format | tools that read Cucumber reports | | `messages` | [Cucumber Messages](https://github.com/cucumber/messages) as NDJSON | the Cucumber ecosystem, custom tooling | | `agent` | the compact JSON report below | agents, scripts | | `teamcity` | TeamCity service messages: a live tree of features, scenarios and steps | IDE test runners (the axx IntelliJ plugin and VS Code extension run it) | The same list can be the default in `axx.yaml`: axx.yaml ```yaml run: reporters: - pretty - junit: build/axx/junit.xml - html: build/axx/report.html ``` An unknown name fails with [`AXX-E0600`](/references/error-codes/#axx-e0600). ## Machine-readable results [Section titled “Machine-readable results”](#machine-readable-results) `axx run --json` prints one [JSON envelope](/references/json-output/) whose `data` is the run report. `--format agent:FILE` writes the same report to a file while the console shows another format. Only failures are listed, so the report stays small however large the suite is. [JSON output](/references/json-output/#the-run-report) describes every field. The exit code says what kind of result it was without parsing any output; see [exit codes](/references/error-codes/#exit-codes). # Run in CI > Run Axx in GitHub Actions with setup-axx, in GitLab CI with the install script, or anywhere with the container image, and publish JUnit and HTML reports. In CI, Axx does the same thing it does on a laptop: start the apps, wait until they are ready, run the scenarios, stop everything. The only extra work is installing Axx and keeping the reports. ## GitHub Actions [Section titled “GitHub Actions”](#github-actions) `axx init` writes this workflow to `.github/workflows/acceptance.yml`: .github/workflows/acceptance.yml ```yaml name: acceptance on: pull_request: push: branches: [main] permissions: contents: read jobs: acceptance: runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 - uses: nimbusxr/setup-axx@v1 - run: axx run --format junit:build/axx/junit.xml --format html:build/axx/report.html - uses: actions/upload-artifact@v4 if: always() with: name: axx-report path: build/axx/ ``` GitHub-hosted Ubuntu runners include Docker and Compose, so apps that start with `docker compose up` work as they do locally. `nimbusxr/setup-axx` installs the latest release (pre-releases included) and adds it to `PATH`. ## GitLab CI [Section titled “GitLab CI”](#gitlab-ci) Install Axx with the install script. When the apps start with Docker Compose, run the job with Docker-in-Docker and point the tests at the `docker` host through a profile: .gitlab-ci.yml ```yaml acceptance: image: docker:27 services: - docker:27-dind variables: DOCKER_TLS_CERTDIR: "/certs" before_script: - apk add --no-cache curl - curl -fsSL https://axx.nimbusxr.us/install.sh | sh - export PATH="$HOME/.local/bin:$PATH" script: - axx run --profile ci --format junit:build/axx/junit.xml --format html:build/axx/report.html artifacts: when: always paths: [build/axx/] reports: junit: build/axx/junit.xml ``` axx.yaml ```yaml profiles: ci: properties: local.host: docker # containers publish their ports on the dind host ``` This works when your features and `axx.yaml` use `${sys:local.host}` instead of a hard-coded `localhost`. ## Any other CI [Section titled “Any other CI”](#any-other-ci) Use the install script, or the container image for commands that do not start apps: ```sh docker run --rm -v axx-cache:/home/nonroot -v "$PWD:/work" -w /work ghcr.io/nimbusxr/axx validate ``` ## Keep Axx’s cache [Section titled “Keep Axx’s cache”](#keep-axxs-cache) The first command in a project prepares Axx with the packs in `axx-packs.yaml` ([Choose packs](/guides/use-packs/)). On a fresh runner, that happens on every run. Keep the prepared builds between runs to skip it. In GitHub Actions, add this step before `axx run`: ```yaml - uses: actions/cache@v4 with: path: ~/.cache/axx/builds key: axx-${{ runner.os }}-${{ hashFiles('**/axx-packs.yaml', '**/axx-packs.lock') }} restore-keys: axx-${{ runner.os }}- ``` GitLab caches only paths inside the project, so move Axx’s cache there: .gitlab-ci.yml ```yaml acceptance: variables: XDG_CACHE_HOME: "$CI_PROJECT_DIR/.cache" cache: key: files: [axx-packs.yaml] paths: [.cache/axx/builds] ``` A stale cache is harmless: when the packs or Axx’s version change, Axx prepares again. ## Catch problems early [Section titled “Catch problems early”](#catch-problems-early) The exit code says what kind of failure it was, without parsing output: `1` is a failed scenario, `3` an undefined step, `4` an app that did not start ([exit codes](/references/error-codes/#exit-codes)). Two cheap guards catch the last two before any app starts: ```sh axx validate # exit 3 on undefined steps, in seconds axx doctor # exit 4 if an app's command is missing ``` Use `--fail-fast` to stop scheduling new scenarios after the first failure when you prefer a quick red over a full report. ## Reports [Section titled “Reports”](#reports) The workflows above write JUnit and HTML reports with `--format`. [Reports](/guides/reports/) lists every format, and `run.reporters` in `axx.yaml` can make them the default. `--rerun-file build/axx/rerun.txt` also records the failed scenarios for a follow-up run ([Rerun what failed](/guides/tags-and-filtering/#rerun-what-failed)). ## Run scenarios in parallel safely [Section titled “Run scenarios in parallel safely”](#run-scenarios-in-parallel-safely) CI runners have fewer CPUs than laptops, and `run.workers` defaults to the number of CPUs. If scenarios that pass locally fail in CI, check that they use unique data ([Isolate test data](/guides/isolate-test-data/)) before lowering `--workers`. # Seed and query SQL > Load datasets into PostgreSQL (and MySQL, SQLite or SQL Server), assert on rows and JSON columns, wait for asynchronous writes, and inject database faults with triggers. The SQL steps put a database in a known state before a scenario and check what your service wrote afterwards. PostgreSQL is first-class; MySQL, SQLite and SQL Server are supported. ## Register the database [Section titled “Register the database”](#register-the-database) ```gherkin Background: Given a parcels-db database with the following properties: | url | postgres://localhost:5432/parcels | | user | parcels | | password | parcels | ``` Native URLs (`postgres://`, `mysql://`, `sqlserver://`, `sqlite:`) and JDBC-style URLs (`jdbc:postgresql://`, `jdbc:mysql://`, `jdbc:mariadb://`, `jdbc:sqlserver://`, `jdbc:sqlite:`) are accepted; an optional `schema` property sets the default schema. The drivers are pure Go and built into `axx`. JSONB containment and triggers need PostgreSQL; seeds, selections, row counts and locks work on every database. ## Seed [Section titled “Seed”](#seed) ```gherkin Given a seeds/manifest-kestrel.yaml db seed Given a seeds/dispatching.yaml db seed on parcels-db ``` A seed is a dataset keyed by table. YAML is the usual format: seeds/manifest-kestrel.yaml ```yaml parcels.manifest_lines: - id: "ML-KES-0412-1" manifest_id: "M-KESTREL-0412" reference: "PX-KES-1001" sender: "kestrel-books" weight_grams: 850 service_level: "STANDARD" recipient: '{"name": "Emmy Noether", "street": "Bunsenstrasse 3", "city": "Goettingen", "postcode": "37073", "country": "DE"}' - id: "ML-KES-0412-2" manifest_id: "M-KESTREL-0412" reference: "PX-KES-1002" sender: "kestrel-books" weight_grams: 2300 service_level: "EXPRESS" recipient: '{"name": "Lise Meitner", "city": "Hamburg", "postcode": "20095", "country": "DE"}' ``` Flat XML (`.xml`), JSON (`.json`), Excel (`.xlsx`, one sheet per table) and CSV dataset directories (one `.csv` per table plus `table-ordering.txt`) work the same way. The `[null]`, `[DAY,NOW]`, `[DAY,PLUS,1]` and `[UNIX_TIMESTAMP]` placeholders are supported. Values are escaped, so text containing `'` is safe. Scenarios share the database and run in parallel, so give every seeded row an id that belongs to one scenario. [Isolate test data](/guides/isolate-test-data/) shows how to enforce that with `axx lint`. A seed inserts its rows in file order, in one transaction. It never updates or deletes existing rows, so a row whose key already exists fails the step and nothing from that file is written. ## Select rows and assert [Section titled “Select rows and assert”](#select-rows-and-assert) A selection is a query with equality conditions. Later steps check its size and contents: ```gherkin Then a selection of rows is retrieved from the parcels.manifest_lines table where: | manifest_id | M-KESTREL-0412 | And the selection has 2 rows And the selection has more than 1 row And the selection has fewer than 5 rows ``` A scenario can hold several selections, addressed by ordinal (`2nd`, `3rd`, …). Leaving the ordinal out means the first: ```gherkin Then a selection of rows is retrieved from the parcels.manifest_lines table where: | manifest_id | M-KESTREL-0412 | And a 2nd selection of rows is retrieved from the parcels.manifest_lines table where: | id | ML-KES-0412-1 | And the 2nd selection has 1 row ``` ### JSON columns [Section titled “JSON columns”](#json-columns) Assert on properties inside a JSON or JSONB column of a selected row: ```gherkin Then a selection of rows is retrieved from the parcels.parcels table where: | reference | PX-REG-1001 | And the 1st row details property for the selection json properties are: | source | api | | zone | DE-1 | And the 1st row details property for the selection json properties match: | zone | ^DE-.*$ | ``` Or select by JSONB containment (PostgreSQL): ```gherkin Then a selection of rows is retrieved from the parcels.parcels table where the details jsonb column contains: | manifestId | M-KESTREL-0412 | And the selection has 2 rows ``` Paths are JSONPaths such as `manifestId` or `items[0].sku`, and values are compared as text ([the rules](/references/steps/sql/#sqljsonare)). ## Wait for asynchronous writes [Section titled “Wait for asynchronous writes”](#wait-for-asynchronous-writes) When the service writes in the background (importing a manifest, say), poll instead of sleeping: ```gherkin Then within 10s a selection of at least 2 rows is retrieved from the parcels.manifest_lines table where: | manifest_id | M-KESTREL-0412 | | status | IMPORTED | ``` The step retries until the selection has at least that many rows or the time runs out. It passes as soon as the rows appear. ## Inject faults with triggers [Section titled “Inject faults with triggers”](#inject-faults-with-triggers) To test how your service handles database errors, install a trigger that raises an SQL state for matching inserts: ```gherkin @isolated Scenario: A brief database failure does not fail the registration Given a before insert trigger on the parcels.parcels table will raise a 40001 exception 1 time where: | reference | PX-DBF-3001 | And a POST request to /api/parcels And a request payload using an application/json content example And the request payload property reference is 'PX-DBF-3001' When the request is executed Then the response status code is 201 And the before insert trigger on the parcels.parcels table was raised 1 time ``` The trigger raises a serialization failure (`40001`) once; the service retries and the second insert succeeds. Without `1 time` the trigger raises on every matching insert. Another variant inserts the row and then raises (`... will insert and raise a 23505 exception where:`). Triggers change a shared table, so tag those scenarios with a tag in `run.exclusive` (for example `@isolated`) to run them alone after the parallel phase. See [Run in parallel](/guides/parallel-runs/). ## Row locks [Section titled “Row locks”](#row-locks) ```gherkin Scenario: A parcel can be changed again once the depot lets go of it Given a seeds/dispatching.yaml db seed And the rows in the parcels.parcels table are locked where: | reference | PX-DSP-2001 | And a 1st ordered PATCH request to /api/parcels/PX-DSP-2001 And a request payload using an application/json content example named 'Heavier' for 1st ordered request And a 2nd ordered PATCH request to /api/parcels/PX-DSP-2001 And a request payload using an application/json content example named 'Heavier' for 2nd ordered request When the 1st ordered request is executed Then the 1st ordered response status code is 409 When the row locks are released And the 2nd ordered request is executed Then the 2nd ordered response status code is 200 ``` Lock rows to test timeouts and contention in your service. The lock holds until `the row locks are released` or the end of the scenario. ## Named databases [Section titled “Named databases”](#named-databases) Every step has an `on ` form for scenarios with more than one database, for example `a selection of rows is retrieved from the parcels.parcels table on parcels-db where:` and `the selection on parcels-db has 1 row`. # Seed and query MongoDB > Register a MongoDB database, insert documents from JSON seed files, and assert on the documents your service wrote. The MongoDB steps put documents in place before your service reads them, and check the documents it writes. ## Register the database [Section titled “Register the database”](#register-the-database) ```gherkin Background: Given a tracking-db mongo database with the following properties: | url | mongodb://localhost:27017/parcels?authSource=admin | | user | parcels | | password | parcels | ``` All three properties are required and support `${env:...}` and `${sys:...}`. The URL must include the database name; `authSource` defaults to it. The first MongoDB database registered in a scenario is the default. ## Write a seed file [Section titled “Write a seed file”](#write-a-seed-file) A seed is a JSON object that maps collection names to arrays of documents. Extended JSON (`{"$oid": "..."}`, `{"$date": "..."}`) is supported: seeds/scans-in-transit.json ```json { "scans": [ { "scanId": "SC-3001-2", "parcelRef": "PX-TRK-3001", "status": "IN_TRANSIT", "location": "Hamburg hub", "scannedAt": {"$date": "2026-05-05T06:40:00Z"} }, { "scanId": "SC-3001-1", "parcelRef": "PX-TRK-3001", "status": "PICKED_UP", "location": "Berlin depot", "scannedAt": {"$date": "2026-05-04T16:10:00Z"} } ] } ``` ## Seed and use it [Section titled “Seed and use it”](#seed-and-use-it) ```gherkin Scenario: Tracking shows the latest scan, however the scans arrive Given a seeds/scans-in-transit.json mongo db seed And within 10s a selection of at least 1 document is retrieved from the tracking collection where: | _id | PX-TRK-3001 | | scanCount | 2 | When a GET request to /api/parcels/PX-TRK-3001/tracking And the request is executed Then the response status code is 200 And the response payload properties are: | status | IN_TRANSIT | | lastLocation | Hamburg hub | | scanCount | 2 | ``` The service builds its tracking summaries from the scans in the background, so the scenario [waits for the summary](#wait-for-asynchronous-writes) before it calls the API. To seed a specific database when the scenario registers more than one: ```gherkin Given a seeds/scans-in-transit.json MongoDB seed for tracking-db ``` `a seeds/scans-in-transit.json mongo db seed for tracking-db` is the same step with the other spelling. The file path resolves against the `resources` directories in `axx.yaml`. See the [mongo step reference](/references/steps/mongo/). ## Query and assert [Section titled “Query and assert”](#query-and-assert) A document selection finds documents with equality conditions. Dotted field paths reach into nested documents, and [the step reference](/references/steps/mongo/#mongofind) says how values are read: ```gherkin Then a selection of documents is retrieved from the scans collection where: | parcelRef | PX-TRK-3001 | | status | PICKED_UP | And the selection has 1 document And the 1st document for the selection properties are: | scanId | SC-3001-1 | | location | Berlin depot | | scannedAt | 2026-05-04T16:10:00.000Z | ``` Properties are JSONPaths (`location`, `history[0].status`), compared as text; `null` means null and `undefined` means the field is absent ([the rules](/references/steps/mongo/#mongodocare)). `the 1st document for the selection properties match:` takes Java regular expressions instead. Like the SQL selections, they are numbered in the order they are retrieved (`the 2nd selection has 3 documents`), and `has more than` / `has fewer than` compare counts. ### Wait for asynchronous writes [Section titled “Wait for asynchronous writes”](#wait-for-asynchronous-writes) ```gherkin Then within 10s a selection of at least 1 document is retrieved from the tracking collection where: | _id | PX-TRK-3001 | | scanCount | 2 | ``` The step polls every 500 ms until enough documents match or the time runs out. Values that parse as JSON are typed, so `2` matches the number 2. Every query step has an `on ` form for scenarios with more than one MongoDB database. ## Keep documents apart [Section titled “Keep documents apart”](#keep-documents-apart) Seeded documents stay in the database after the scenario, and scenarios run in parallel. Give each scenario’s documents their own keys (`PX-TRK-3001` above) and query by them. [Scenario isolation](/explanations/scenario-isolation/) explains the approach. # Set up agents > Install the Axx skills, connect the Axx MCP server to Claude Code, Cursor, Codex or VS Code, and keep the AGENTS.md section current. Three pieces give a coding agent everything it needs. Install all three; they reinforce each other. [Test with an agent](/tutorials/with-an-agent/) shows them in use. ## Skills [Section titled “Skills”](#skills) ```sh axx skills install # this repository axx skills install --scope user # your home directory, for every repository axx skills list ``` `install` writes the skills to `.agents/skills/` (read by Codex, Cursor, Gemini CLI and Copilot) and links them into `.claude/skills/` for Claude Code (`--no-claude` skips the link). Commit them so every contributor and every CI agent gets them. | Skill | Teaches | | ---------------------- | ----------------------------------------------------------------------------- | | `axx-acceptance-tests` | the write, validate, run loop; rules that keep tests reliable; the step index | | `axx-setup` | `axx init`, apps and readiness, CI, agent integration | | `axx-custom-steps` | custom steps as Go packs | | `axx-debugging` | reading failures, exit codes and logs | The step references inside the skills are generated from *your* project, including your custom packs. Rerun `axx skills install` after adding steps or upgrading Axx; files you edited are kept unless you pass `--force`. The same skills are published at [`/.well-known/agent-skills/index.json`](/.well-known/agent-skills/index.json). ## MCP server [Section titled “MCP server”](#mcp-server) `axx mcp` serves Axx over the [Model Context Protocol](https://modelcontextprotocol.io) on stdio. Because it is the installed binary, its answers match your Axx version and your project’s steps. | Tool | Does | | ------------------ | ----------------------------------------------------------------------------- | | `steps_search` | find steps by intent, with docs and examples | | `step_explain` | how one line matches, or the closest steps | | `feature_validate` | check feature files or feature text without running | | `scenarios_run` | run scenarios (paths, tags, names); returns failures with expected and actual | | `failure_context` | logs, attachments and the last request and response of one failure | | `env` | `up`, `down` or `status` of the apps | | `config_show` | the effective `axx.yaml`, with secrets redacted | | `scaffold` | starter contents for a feature or an `axx.yaml` | It also serves the `axx.yaml` JSON Schema as a resource and a `write-acceptance-tests` prompt. Pass `--profile ci` (or any profile) to apply it to every tool call. ### Claude Code [Section titled “Claude Code”](#claude-code) .mcp.json ```json { "mcpServers": { "axx": { "command": "axx", "args": ["mcp"] } } } ``` The same from the command line: `claude mcp add axx -- axx mcp`. Or install the skills and the server together as a plugin: ```text /plugin marketplace add nimbusxr/axx /plugin install axx@nimbusxr ``` ### Cursor [Section titled “Cursor”](#cursor) .cursor/mcp.json ```json { "mcpServers": { "axx": { "command": "axx", "args": ["mcp"] } } } ``` ### Codex [Section titled “Codex”](#codex) \~/.codex/config.toml ```toml [mcp_servers.axx] command = "axx" args = ["mcp"] ``` ### VS Code (GitHub Copilot) [Section titled “VS Code (GitHub Copilot)”](#vs-code-github-copilot) .vscode/mcp.json ```json { "servers": { "axx": { "type": "stdio", "command": "axx", "args": ["mcp"] } } } ``` ## AGENTS.md [Section titled “AGENTS.md”](#agentsmd) `axx init` adds a managed section to `AGENTS.md`, the file most agents read first. Running `axx init` again updates the section in place and leaves the rest of the file alone: AGENTS.md ```markdown ## Acceptance tests (axx) axx (github.com/nimbusxr/axx, "axxeptance") is a human-readable acceptance testing framework. - Feature files are acceptance criteria a person can read: one scenario per criterion, in plain language, using the steps exactly as written. No programming constructs in Gherkin. - Find steps before writing: `axx steps search ""`; never invent step text. - Steps come from the packs in `axx-packs.yaml`; `axx pack list` shows the others, `axx pack add ` adds one. - Check without running: `axx validate`. Explain one line: `axx explain ""`. - Check test data: `axx lint` reports ids and keys that collide across seed and fixture files. - Run: `axx up` once (keeps apps running), then `axx run --compact`; `axx down` when done. - Every scenario uses unique data (IDs, names, keys): scenarios run in parallel and data persists. - Features live in `features/`; configuration in `axx.yaml` (schema: `axx schema`). - Diagnose failures from the report: `axx run --json` includes expected/actual and a rerun command. ``` ## Docs for agents [Section titled “Docs for agents”](#docs-for-agents) * Every page on this site has a Markdown twin: append `.md` to the path (`/guides/set-up-agents.md`). Pages advertise it with ``. * [`/llms.txt`](/llms.txt) indexes the site; [`/llms-full.txt`](/llms-full.txt) and [`/llms-small.txt`](/llms-small.txt) contain it in one file. * The `axx.yaml` schema is at [`/schemas/v0/axx.schema.json`](/schemas/v0/axx.schema.json); `axx schema` prints the same thing offline. # Set up your editor > See Axx's steps in feature files as you write them - problems flagged as you type, step completion, docs on hover and highlighted parameters - in IntelliJ IDEA, VS Code or any editor with LSP support. Axx’s steps are defined inside the `axx` binary, not in your project, so an editor cannot find them on its own. `axx lsp`, a language server built into the binary, tells the editor about them. The IntelliJ plugin and the VS Code extension start it for you. In feature files you get: * undefined, ambiguous and misused steps, and Gherkin syntax errors, flagged as you type, with the closest real steps; * completion of step text, with the step’s parameters as placeholders; * the step’s documentation, and the values of its parameters, on hover; * go to a step’s definition: the Go code that defines it when that code is on your machine (your custom packs, or axx built from source), otherwise the step’s entry in a reference page; * the files that steps name: Ctrl/Cmd+click a seed, a payload, a schema, an OpenAPI document or a log’s `file://` url to open the file, get its path completed as you type, and a warning when it does not exist (with a pointer to `axx fixtures generate` for a fixture that is not generated yet). These are the steps’ `{filepath}` parameters and file properties, and the Examples cells that fill them in; the file is found the way the step finds it, in the `resources` directories, next to `axx.yaml` or at an absolute path. A log’s file may appear only during the run, so it is never flagged; * highlighted parameter values and Scenario Outline ``. The steps are the ones your project loads, custom packs included ([Choose packs](/guides/use-packs/)). The server finds the project’s `axx.yaml` itself, even when it is in a subdirectory of the folder you opened. Pre-release The IntelliJ plugin and the VS Code extension are published with the first release. Until then, build them from the repository: `./gradlew buildPlugin` in `ide/intellij`, `npm ci && npm run package` in `ide/vscode`. ## IntelliJ IDEA [Section titled “IntelliJ IDEA”](#intellij-idea) Needs IntelliJ IDEA 2025.3 or later (or another JetBrains IDE of that version). 1. Install the **axx** plugin: *Settings | Plugins | Marketplace*, or *Install Plugin from Disk* with the zip from `ide/intellij/build/distributions`. 2. Optionally install the **Gherkin** plugin for keyword highlighting. It looks for step definitions in code and would mark every Axx step as undefined, so the axx plugin turns that inspection off in Axx projects. 3. Open a `.feature` file. The server appears in the **Language Services** widget in the status bar, where you can restart it. If `axx` is not on your `PATH`, set **axx executable** in *Settings | Tools | axx*. ## VS Code [Section titled “VS Code”](#vs-code) 1. Install the **axx** extension: from the Marketplace, or *Extensions: Install from VSIX* with the file `npm run package` writes. 2. Disable the official Cucumber extension in Axx workspaces. It looks for step definitions in code and marks every Axx step as undefined. 3. Open a `.feature` file. The extension highlights Gherkin itself and starts the server once the workspace is trusted. If `axx` is not on your `PATH`, set `axx.path` to its location. The **axx** output channel shows the server’s log. ## Other editors [Section titled “Other editors”](#other-editors) Any editor with LSP support can run `axx lsp` for `*.feature` files. It talks over stdio and finds the project from its working directory. In Neovim 0.11 or later, which gives `.feature` files the `cucumber` file type: init.lua ```lua vim.lsp.config('axx', { cmd = { 'axx', 'lsp' }, filetypes = { 'cucumber' }, root_markers = { 'axx.yaml', '.git' }, }) vim.lsp.enable('axx') ``` ## Run and debug scenarios [Section titled “Run and debug scenarios”](#run-and-debug-scenarios) Both plugins run scenarios from the editor, with the results in the IDE’s test view: features, scenarios and steps, each one linked to its line, with expected and actual values for failed assertions. * **IntelliJ IDEA:** click the run icon in the gutter of a Feature, Rule, Scenario or Scenario Outline line, or of an Examples row, or choose *Run* on a feature file or directory. *Rerun Failed Tests* is in the test view’s toolbar. * **VS Code:** use the run buttons in the gutter or the Testing view. *Debug* instead of *Run* stops at breakpoints in the Go code of the steps ([Stop in step code](/guides/debug-failures/#stop-in-step-code)). ## After changing packs [Section titled “After changing packs”](#after-changing-packs) The server starts with the project’s packs. After you change a pack of your own or `axx-packs.yaml`, restart the server: from the Language Services widget in IntelliJ, with **axx: Restart language server** in VS Code, or by restarting the editor. # Tags and filtering > Select which scenarios run by path, line, tag expression or name, and set defaults in axx.yaml. Every way of selecting scenarios combines: paths narrow the files, then tags and names filter the scenarios in them. ## By path and line [Section titled “By path and line”](#by-path-and-line) ```sh axx run # run.paths from axx.yaml (default: features) axx run features/quotes.feature # one file axx run features/rest # a directory axx run features/quotes.feature:14 # the scenario on line 14 axx run features/a.feature:14 features/b.feature:9 # several ``` A line number selects the scenario that starts on that line or contains it, so the line of any step works. That is why the `rerun` command in a failure report is always a `file:line`. ## By tag [Section titled “By tag”](#by-tag) Tag features, rules, scenarios and examples: ```gherkin @parcels Feature: Register parcels @smoke Scenario: A shop lists its own parcels Given a GET request to /api/parcels?sender=lark-ceramics When the request is executed Then the response status code is 200 @wip Scenario: A cancelled parcel is gone Given a DELETE request to /api/parcels/PX-TAG-0001 When the request is executed Then the response status code is 204 ``` Select with a [tag expression](https://cucumber.io/docs/cucumber/api/#tag-expressions): ```sh axx run --tags @smoke axx run --tags "@parcels and not @wip" axx run -t "(@smoke or @critical) and not @slow" ``` Tags are inherited: a scenario has its own tags plus those of its feature, rule and examples. ## By name [Section titled “By name”](#by-name) ```sh axx run --name "registered once" # a regular expression on the scenario name axx run -n "^A shop" -n "cancelled" # repeatable: matches either ``` ## Defaults in axx.yaml [Section titled “Defaults in axx.yaml”](#defaults-in-axxyaml) axx.yaml ```yaml run: paths: [features] tags: "not @wip and not @ignore" ``` `--tags` on the command line replaces `run.tags`. ## Tags that change behavior [Section titled “Tags that change behavior”](#tags-that-change-behavior) * Tags listed in `run.exclusive` make scenarios run alone, after the parallel phase ([Run in parallel](/guides/parallel-runs/)). * With `active.enabled`, tags decide which apps start ([Manage the app lifecycle](/guides/manage-app-lifecycle/#start-only-what-a-run-needs)). ## Rerun what failed [Section titled “Rerun what failed”](#rerun-what-failed) ```sh axx run --rerun-file build/axx/rerun.txt # writes file:line of each failed scenario axx run $(cat build/axx/rerun.txt) # run only those ``` ## Preview a selection [Section titled “Preview a selection”](#preview-a-selection) `--dry-run` matches every selected step without starting apps or executing anything, which shows what a filter selects: ```sh axx run --tags @smoke --dry-run ``` # Test cloud services > Check what your services do with S3, SQS, SNS, EventBridge, DynamoDB, Cloud Storage, Pub/Sub, BigQuery, Firestore, Blob Storage and Service Bus, against a cloud account or local emulators. Services often do their work through the cloud: a file lands in a bucket and a job picks it up, results go to a warehouse, and events go to a topic. The cloud packs let a scenario do what the outside world does, such as uploading the file or publishing the event. The scenario then checks what your service did: the row, the object, the message. ```gherkin Scenario: A claim within the limit is settled when its photo arrives When the evidence/crushed-box.png file is uploaded to the claim-evidence s3 bucket as claims/CLM-4101/crushed-box.png Then within 30s the claims dynamodb table has an item where: | id | CLM-4101 | | status | APPROVED | And the refund-requests sqs queue has a message where: | claim | CLM-4101 | | attribute reason | DAMAGED | ``` ## The packs [Section titled “The packs”](#the-packs) | Cloud | Packs | | ------------ | -------------------------------------------------------------------------------- | | AWS | `aws-s3`, `aws-sqs`, `aws-sns`, `aws-eventbridge`, `aws-dynamodb`, on `aws-core` | | Google Cloud | `gcp-storage`, `gcp-pubsub`, `gcp-bigquery`, `gcp-firestore`, on `gcp-core` | | Azure | `azure-blob`, `azure-servicebus` | Each pack talks to its service through the cloud’s official SDK. The [step reference](/references/steps/) lists the steps of each one, grouped by cloud. With an `axx-packs.yaml` ([Choose packs](/guides/use-packs/)), list the packs you use. An AWS or Google Cloud pack brings its core with it. ## Connect as your service does [Section titled “Connect as your service does”](#connect-as-your-service-does) Register the account, project or resource once, usually in the `Background`. A scenario then uses it everywhere: ```gherkin Background: Given the parcels aws account with the following properties: | region | eu-west-1 | | endpoint | http://${sys:local.host}:4566 | | access key id | ${env:AWS_ACCESS_KEY_ID:-local} | | secret access key | ${env:AWS_SECRET_ACCESS_KEY:-local} | ``` * **AWS:** `the {word} aws account` takes a `region`, and optionally an `endpoint`, a `profile` or static keys. Without keys, the SDK’s default credential chain applies. * **Google Cloud:** `the {word} gcp project` takes a `project`, and optionally an `endpoint` or a `credentials` key file. Without them, Application Default Credentials apply. * **Azure:** each service connects to its own resource. `the {word} azure storage account` takes a `connection string` or a `url`. `the {word} service bus namespace` takes a `connection string` or a `namespace`, plus a `management endpoint` for emulators. An `endpoint` points every service of the account or project at an emulator. Leave it out, and the same features run against the cloud. ## Run the clouds locally [Section titled “Run the clouds locally”](#run-the-clouds-locally) Emulators such as [floci](https://floci.io) run AWS, Google Cloud and Azure on your machine: one container per cloud, free, with no account needed. Start one next to your service in the Compose file your [app definition](/guides/manage-app-lifecycle/) runs, and point your service at it the way its SDK expects: * `AWS_ENDPOINT_URL` for AWS; * `STORAGE_EMULATOR_HOST`, `PUBSUB_EMULATOR_HOST` and `FIRESTORE_EMULATOR_HOST` for Google Cloud; * connection strings for Azure. compose.yaml ```yaml services: aws: image: floci/floci:latest ports: ['4566:4566'] environment: FLOCI_HOSTNAME: aws claims: build: ../app environment: AWS_REGION: eu-west-1 AWS_ENDPOINT_URL: http://aws:4566 ``` Create the buckets, queues, topics and tables the way your infrastructure code does in a real account. Scenarios bring the data: files, seeds and messages. The examples run a one-shot `provision` command before the service starts. ## What the checks do [Section titled “What the checks do”](#what-the-checks-do) * **They wait.** Services act asynchronously, so every check waits: 10 seconds, or the time `within {duration}` gives. * **They see the scenario’s messages only.** A message check counts only what arrived since its scenario started. Scenarios run in parallel, so match on data unique to the scenario, such as a claim or an invoice ID. * **Topics and buses cost nobody a message.** For the topics and buses a run checks, axx subscribes a listener of its own before the scenarios start, and removes it when the run ends: * an SQS queue for an SNS topic; * a subscription for a Pub/Sub topic; * a rule and a queue for an EventBridge bus; * a subscription for a Service Bus topic. * **Checking a queue takes its messages.** On an SQS or Service Bus queue, axx consumes each message like any other consumer would. So check the queues your service writes to. Check a queue it consumes by what the service does with the messages. * **Conditions compare text.** A `| field | value |` row compares the value as text, with a dotted path into nested data. `null` means null and `undefined` means absent. On messages, `attribute ` or `property ` rows check the values sent with the message. ## Examples [Section titled “Examples”](#examples) Each of these examples tests a service built on one cloud, with that cloud running in floci: * [`parcel-claims`](https://github.com/nimbusxr/axx/tree/main/examples/parcel-claims), on AWS. Shops claim for damaged or lost parcels. Evidence photos in S3 trigger the decision; refunds, decisions and events go out over SQS, SNS and EventBridge. * [`carrier-billing`](https://github.com/nimbusxr/axx/tree/main/examples/carrier-billing), on Google Cloud. Carriers’ invoices uploaded to Cloud Storage are priced from BigQuery and Firestore, and disputes are published on Pub/Sub. * [`customs-clearance`](https://github.com/nimbusxr/axx/tree/main/examples/customs-clearance), on Azure. Declarations filed on Service Bus are cleared from invoices in Blob Storage. # Test Kafka with Avro > Configure Kafka topic clients, build and publish Avro events with a Schema Registry, verify consumed events, and address several events or clusters in one scenario. The Kafka steps publish the events your service consumes and check the events it produces, with Avro and a Schema Registry or as plain text. ## Register the cluster and a topic client [Section titled “Register the cluster and a topic client”](#register-the-cluster-and-a-topic-client) ```gherkin Background: Given the events kafka service with the following properties: | brokers | ${sys:local.host}:9092 | And a depot-scans kafka topic client with the following properties: | producer.value.serializer | io.confluent.kafka.serializers.KafkaAvroSerializer | | producer.schema.registry.url | http://${sys:local.host}:9081 | And a parcel-events kafka topic client with the following properties: | consumer.value.deserializer | io.confluent.kafka.serializers.KafkaAvroDeserializer | | consumer.schema.registry.url | http://${sys:local.host}:9081 | ``` The service consumes depot scans, so the suite publishes to `depot-scans`; it announces registered parcels on `parcel-events`, so the suite reads that topic. A topic the suite both publishes to and reads takes both sets of properties. The topic client’s properties use the Kafka client property names, prefixed with `producer.` or `consumer.`. Axx translates them to its own Go client, so you do not need a JVM; the [Kafka step reference](/references/steps/kafka/) lists every property it understands. Consumer group settings (`group.id`, `enable.auto.commit`) have no effect, because Axx reads topics without a group. For a TLS-secured cluster: ```gherkin Given a secure-events kafka topic client with the following properties: | producer.security.protocol | SASL_SSL | | producer.sasl.mechanism | SCRAM-SHA-512 | | producer.sasl.jaas.config | org.apache.kafka.common.security.scram.ScramLoginModule required username="app" password="${env:KAFKA_PASSWORD}"; | | producer.ssl.truststore.location | certs/truststore.p12 | | producer.ssl.truststore.password | ${env:TRUSTSTORE_PASSWORD} | ``` The consumer takes the same settings with the `consumer.` prefix. For a topic on a different cluster, register that cluster and the client on it explicitly: ```gherkin Given the depots kafka service with the following properties: | brokers | depots-kafka:9092 | And a depot-scans kafka topic client on the depots kafka service with the following properties: | producer.value.serializer | io.confluent.kafka.serializers.KafkaAvroSerializer | | producer.schema.registry.url | http://depots-registry:8081 | ``` ## Build an event [Section titled “Build an event”](#build-an-event) ```gherkin Given a depot-scans kafka event And the depot-scans kafka event key is PX-TRK-3002 And the depot-scans kafka event payload is a kafka/scan-delivered.json resource And the depot-scans kafka event payload properties are: | $.scanId | SC-3002-2 | | $.parcelRef | PX-TRK-3002 | And the depot-scans kafka event headers are: | X-Scanner-Id | leipzig-dock-4 | ``` * The payload starts from a file in Avro’s JSON encoding. Union values are written as `{"string": "..."}` or `null`; set `packs.kafka.lenientUnions: true` in `axx.yaml` to accept bare values when only one branch fits. * `the depot-scans kafka event payload properties are:` sets JSONPath properties of that topic’s event; `the kafka event payload properties are:` uses the first topic client’s. Values are always set as **strings** (an empty cell sets null), and each property must already exist in the payload, so give numbers and objects their values in the file. * `the depot-scans kafka event payload property is null` sets a property to JSON null (for an Avro union with `null`). * Keep payload files schema-valid with [fixture factories](/guides/fixture-factories/) (the `avro` family); `kafka/scan-delivered.json` is generated that way. ## Publish [Section titled “Publish”](#publish) ```gherkin When the depot-scans kafka event is published using schema schemas/depot-scan.avsc ``` The payload is read with the Avro schema (`.avsc`, resolved against `resources`), the schema is registered under `-value` (or looked up, with `producer.auto.register.schemas=false`), and the record is sent in the Confluent wire format with the key and headers you set. A payload that does not fit the schema fails with the path of the mismatch, such as `$.location: expected string, got number 5`. To publish without Avro (plain text or JSON with the default `StringSerializer`): ```gherkin Given a delivery-notifications kafka event And the delivery-notifications kafka event payload is a kafka/delivery-notification.json resource When the delivery-notifications kafka event is published ``` ## Verify consumed events [Section titled “Verify consumed events”](#verify-consumed-events) Check the events your service publishes. After a scenario registers parcel `PX-EVT-4001` through the API: ```gherkin Then the parcel-events kafka event named registered key is PX-EVT-4001 And the parcel-events kafka event named registered payload properties are: | $.reference | PX-EVT-4001 | | $.weightGrams | 1200 | | $.zone | DE-1 | | $.source | api | And the parcel-events kafka event named registered headers are: | X-Event-Type | ParcelRegistered | And the parcel-events kafka event named registered headers match: | X-Event-Type | ^Parcel.*$ | ``` `named registered` names an expectation. Each step adds its checks to the name and passes when one record of the topic meets all of them at once, so the four steps above check that a single record has that key, those properties and those headers. Records are read from the start of the topic (with `consumer.auto.offset.reset=latest`, only those produced after the step starts), and a step waits up to 30 seconds (`packs.kafka.timeout`). Expected payload values are typed, so `1200` does not equal `1200.0` ([the rules](/references/steps/kafka/#kafkaconsumedproperties)). `headers are` needs each header exactly once with that value; `headers match` takes regular expressions. When a step fails, the report lists the name’s checks and the latest records of the topic with the reason each one did not match. ## Test what your service does with an event [Section titled “Test what your service does with an event”](#test-what-your-service-does-with-an-event) Publishing and then consuming on the same topic only checks the wiring. An acceptance test publishes the event your service consumes and checks what the service does: a row in its database, a document, a response from its API, or an event on another topic. The parcels service keeps a tracking summary in MongoDB for every parcel the depots scan: ```gherkin Scenario: A delivery scan marks the parcel delivered Given a depot-scans kafka event And the depot-scans kafka event key is PX-TRK-3003 And the depot-scans kafka event payload is a kafka/scan-delivered.json resource And the depot-scans kafka event payload properties are: | $.scanId | SC-3003-1 | | $.parcelRef | PX-TRK-3003 | When the depot-scans kafka event is published using schema schemas/depot-scan.avsc Then within 20s a selection of at least 1 document is retrieved from the tracking collection where: | _id | PX-TRK-3003 | | delivered | true | And the 1st document for the selection properties are: | status | DELIVERED | | lastLocation | Leipzig | ``` The `Background` registers the Kafka service, `the depot-scans kafka topic client` and the MongoDB database. The service handles the event in the background, so the `Then` step polls until the summary appears instead of sleeping. ## Several events in one scenario [Section titled “Several events in one scenario”](#several-events-in-one-scenario) Address events by ordinal when a scenario builds more than one. `a 2nd ordered ... kafka event` must follow the 1st; steps without an ordinal use the first event: ```gherkin Given a 1st ordered depot-scans kafka event And the 1st ordered depot-scans kafka event key is PX-TRK-3002 And the 1st ordered depot-scans kafka event payload is a kafka/scan-out-for-delivery.json resource And the 1st ordered depot-scans kafka event payload properties are: | $.scanId | SC-3002-1 | | $.parcelRef | PX-TRK-3002 | And a 2nd ordered depot-scans kafka event And the 2nd ordered depot-scans kafka event key is PX-TRK-3002 And the 2nd ordered depot-scans kafka event payload is a kafka/scan-delivered.json resource And the 2nd ordered depot-scans kafka event payload properties are: | $.scanId | SC-3002-2 | | $.parcelRef | PX-TRK-3002 | When the 1st ordered depot-scans kafka event is published using schema schemas/depot-scan.avsc And the 2nd ordered depot-scans kafka event is published using schema schemas/depot-scan.avsc ``` ## Several clusters [Section titled “Several clusters”](#several-clusters) Every step has a form that names the Kafka service, such as `the depot-scans kafka event key is PX-TRK-3002 on the depots kafka service`. See the [step index](/references/step-index/) for the full list. ## Keep events apart [Section titled “Keep events apart”](#keep-events-apart) Topics keep their events across runs, and scenarios run in parallel. Use a key and identifiers that belong to one scenario (a parcel reference, a scan id), and assert on them. For fixture files, a generated `axx lint` rule can enforce that every event file owns a unique id, such as the `scanId` of every depot scan ([Isolate test data](/guides/isolate-test-data/)). # Choose packs > Pick the packs a project uses with axx-packs.yaml and axx pack add, remove, list and update. A **pack** is a set of steps. Axx publishes `rest`, `mock`, `sql`, `mongo`, `kafka` and `logs`. It also publishes packs for cloud services, one per service, named after its cloud ([Test cloud services](/guides/test-cloud-services/)): * **AWS:** `aws-s3`, `aws-sqs`, `aws-sns`, `aws-eventbridge` and `aws-dynamodb`, which build on `aws-core`. * **Google Cloud:** `gcp-storage`, `gcp-pubsub`, `gcp-bigquery` and `gcp-firestore`, which build on `gcp-core`. * **Azure:** `azure-blob` and `azure-servicebus`. A project can write packs of its own, or use packs other teams publish. Every pack, Axx’s or anyone else’s, is used the same way: the project lists it in `axx-packs.yaml`, next to `axx.yaml`. ## Choose the packs a project uses [Section titled “Choose the packs a project uses”](#choose-the-packs-a-project-uses) ```console $ axx pack add rest sql created axx-packs.yaml added rest, sql $ axx pack list rest axx 46 steps sql axx 17 steps mock axx not used mongo axx not used ... ``` axx-packs.yaml ```yaml packs: - rest - sql ``` Only the listed packs are loaded, so their steps are the only ones `axx validate`, `axx steps` and the skills know about. Listing an AWS or Google Cloud pack loads its core too. `axx pack remove` takes packs off the list. `axx init` starts the file with `rest`, which its first feature uses. Commit `axx-packs.yaml`. The first `axx` command that needs the project’s steps prepares Axx with its packs: ```console $ axx validate axx: preparing rest, sql (once; cached for later runs) axx: ready in 31s 2 files, 6 scenarios, 41 steps: ok ``` This happens once for each list of packs. Later runs start immediately. The first time, Axx downloads what it needs, so it takes longer. In CI, keep Axx’s cache between runs ([Run in CI](/guides/run-in-ci/#keep-axxs-cache)). ## Add your own packs [Section titled “Add your own packs”](#add-your-own-packs) Your own packs are listed the same way, by path or by Go module: ```console $ axx pack new ./steps # create a pack in this repository $ axx pack add github.com/team/axx-grpc@v1.2.0 # use a pack published as a Go module ``` [Write custom steps](/guides/write-custom-steps/) shows what goes in a pack. When you change a pack in the repository, the next `axx` command prepares Axx with the new version. Versions of module packs are pinned in `axx-packs.lock` (commit it). `axx pack update` moves them to the latest version, or to the version written in `axx-packs.yaml`, and prepares Axx again. # Validate against OpenAPI > Check every request and response against your OpenAPI document, build payloads from its examples, and relax individual rules for negative tests. Give a REST service an `openapi` property and Axx validates every request it sends and every response it receives against that document. A violation fails the `When the request is executed` step and names the rule that broke. Your OpenAPI document stays the contract, and the acceptance suite proves the service honors it. ## Turn it on [Section titled “Turn it on”](#turn-it-on) ```gherkin Background: Given the parcels service with the following properties: | url | http://localhost:8400 | | openapi | http://localhost:8400/openapi.json | ``` `openapi` is a URL or a file path (resolved against `resources`), in JSON or YAML. OpenAPI 3.0 and 3.1 are supported. Point it at the document your service serves (`/openapi.json`, `/v3/api-docs`) or at the file in your repository. Do not point it at a third party’s live URL: if it is unreachable, your tests fail for reasons that have nothing to do with your service. Mock third parties instead: the WireMock image that mocks them checks their contracts, with settings of their own ([Mock dependencies](/guides/mock-dependencies/#check-the-dependencys-contract)). ## Build payloads from examples [Section titled “Build payloads from examples”](#build-payloads-from-examples) Payloads in Gherkin tables get long. Start from an example in the OpenAPI document and change only what the scenario is about: openapi.yaml (excerpt) ```yaml paths: /api/parcels: post: requestBody: content: application/json: examples: Standard parcel: value: reference: PX-EXAMPLE-1 sender: shop-example weightGrams: 1200 serviceLevel: STANDARD recipient: name: Ada Lovelace street: Invalidenstrasse 116 city: Berlin postcode: "10115" country: DE ``` ```gherkin Scenario: A shop registers a parcel Given a POST request to /api/parcels And a request payload using an application/json content example named 'Standard parcel' And the request payload properties are: | reference | PX-REG-1001 | | weightGrams | 2500 | When the request is executed Then the response status code is 201 ``` * `a request payload using an application/json content example` takes the operation’s first example; add `named ''` to pick one. * `a request payload using an application/json empty content template` starts from an empty body instead. * Examples can use `externalValue` to keep large payloads in their own files. ### Setting values in a payload [Section titled “Setting values in a payload”](#setting-values-in-a-payload) Keys are JSONPath expressions (`reference`, `recipient.postcode`, `items[0].sku`). A value keeps the type of the property it replaces, double quotes make it a string, `null` sets JSON null and `undefined` removes the property. The [step reference](/references/steps/rest/#restrequestproperties) has the exact rules. Double-quote values that look like numbers but are strings in the schema, such as postcodes (`"50667"`) or long numeric identifiers, so they stay exact strings. ## Validation levels [Section titled “Validation levels”](#validation-levels) Each rule has a level: `ERROR` fails the step, while `WARN`, `INFO` and `IGNORE` let it pass. Everything is `ERROR` by default. `FAIL` is accepted as an alias of `ERROR`. For a negative test, where you send an invalid request on purpose to check that the service rejects it, relax only the rule you are breaking, only in that scenario: ```gherkin Scenario: Parcels over 30 kg are refused Given the OpenAPI validation levels are: | validation.request.body.schema.maximum | IGNORE | And a POST request to /api/parcels And a request payload using an application/json content example And the request payload properties are: | reference | PX-REG-1005 | | weightGrams | 31000 | When the request is executed Then the response status code is 400 And the response payload property detail is 'weightGrams must be at most 30000' ``` The document says `weightGrams` is at most 30000, so without the relaxed level the request step would fail on the contract violation, and the scenario could never check how the service answers. Only the `maximum` rule is relaxed; the rest of the body is still checked. With several REST services, name the one to relax: ```gherkin Given the OpenAPI validation levels on parcels are: | validation.request.body.schema.maximum | IGNORE | ``` Levels apply when the request is executed, so a scenario can relax a rule for one request and restore it for the next by setting `ERROR` again. Common keys: | Key | Checks | | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `validation.request.body` | the request body against the schema | | `validation.request.body.schema.` | one schema keyword in the request body (`maximum`, `pattern`, `required`, …) | | `validation.request.parameter.missing` | required path parameters (query and header parameters have `validation.request.parameter.query.missing` and `validation.request.parameter.header.missing`) | | `validation.request.security.missing` | required security schemes | | `validation.response.body` | the response body against the schema | | `validation.response.body.schema.required` | required properties in the response | | `validation.response.body.schema.type` | property types in the response | The failure message of a violated rule includes its key, so you can copy it into the table. ### Defaults for the whole suite [Section titled “Defaults for the whole suite”](#defaults-for-the-whole-suite) `openapi.levels` in `axx.yaml` sets suite-wide defaults. Use it for a known gap in a document you do not control, not to silence your own contract: axx.yaml ```yaml openapi: levels: validation.request.security.missing: IGNORE ``` ## OpenAPI 3.1 and `nullable` [Section titled “OpenAPI 3.1 and nullable”](#openapi-31-and-nullable) OpenAPI 3.1 replaced `nullable: true` with JSON Schema type unions. In a document that declares `openapi: 3.1.0`, a schema that uses `nullable` does not compile, so every request or response validated against it fails with a `...schema.processingError`, whatever its values: ```yaml # openapi: 3.1.0 nickname: type: [string, "null"] # not: type: string + nullable: true ``` `nullable` is still correct in `openapi: 3.0.x` documents. ## The other side: your dependencies’ contracts [Section titled “The other side: your dependencies’ contracts”](#the-other-side-your-dependencies-contracts) This page covers your service’s own contract. The services your service calls have contracts too: run their WireMock mocks with the OpenAPI validation extension, and every call your service makes and every stubbed answer is checked against the dependency’s document. Those checks have levels of their own; `the OpenAPI validation levels are:` never relaxes them. See [Mock dependencies](/guides/mock-dependencies/#check-the-dependencys-contract). # Write custom steps > Add your own Gherkin steps as a Go pack that shares the context of Axx's packs. Custom steps sit next to the steps of Axx’s packs and are used the same way in feature files. They live in a **pack** of your own: Go code that builds on Axx’s core, like Axx’s packs, and works on the same context their steps use. ## Create a pack [Section titled “Create a pack”](#create-a-pack) Create a pack and add it to the project in one go: ```console $ axx pack new ./steps created the steps pack in ./steps and added it to axx-packs.yaml ``` A pack is a Go package that exports `Pack()`. Its steps receive the scenario, and from it every pack’s context: the services registered in the scenario and their state. A custom step sees exactly what the steps of Axx’s packs set up, and their steps see what it adds. steps/pack.go ```go package steps import ( "github.com/nimbusxr/axx/core" "github.com/nimbusxr/axx/packs/sql" ) func Pack() core.Pack { return pack{} } type pack struct{} func (pack) Manifest() core.Manifest { return core.Manifest{ Name: "steps", Steps: []core.StepDef{{ ID: "steps.rows", Keyword: "Then", Expr: "the {word} table on {dbService} holds {int} row(s)", Doc: "Counts the rows of a table on a registered database.", Examples: []string{"Then the parcels.manifest_lines table on parcels-db holds 3 rows"}, Run: func(sc *core.Scenario, a core.Args) error { db := a.Value(1).(*sql.Service) // {dbService} resolves to the registered service sel, err := db.Query(sc.Context(), a.String(0), "SELECT * FROM "+a.String(0)) if err != nil { return err } if got := len(sel.Rows); got != a.Int(2) { return core.Fail("rows in "+a.String(0), a.Int(2), got) } return nil }, }}, } } ``` The contexts: | Pack | Context | Holds | | ------- | ------------------- | ------------------------------------------------------------------------------------------ | | `rest` | `rest.Context(sc)` | REST services, their requests in order, and each request’s response (`Exchange()`) | | `sql` | `sql.Context(sc)` | databases with their connection pool (`DB`), selections, triggers; `Query`, `AddSelection` | | `mongo` | `mongo.Context(sc)` | MongoDB databases (`DB()`) and their selections | | `mock` | `mock.Context(sc)` | mocked (WireMock) services | | `kafka` | `kafka.Context(sc)` | Kafka services, their topic clients (`Topic(name)`) and the events drafted for each topic | Every context has `Service(name...)` (no name: the default, the first registered), `Services()` and `AddService(...)`, and `sql.Connect`, `mongo.Connect` and `mock.NewService` create services the way the packs’ own steps do. Report failed expectations with `core.Fail(message, expected, actual)` so reports show both values, and keep state of your own in a `core.NewStateKey`: it is per scenario, like the packs’ contexts. A step parameter that names a file is a `{filepath}`, and a step whose `key | value` table has file properties names them in `TableTypes`: `map[string]string{"schema": "filepath"}` for a file the step reads, or `"url"` for a URL whose `file://` form names a file that may appear only during the run. Read the file with `sc.Suite().ResolvePath(a.String(0))`, which looks in the `resources` directories and next to `axx.yaml`: that is where editors look to link the value, complete its path and flag a missing file ([Set up your editor](/guides/set-up-your-editor/)). The first `axx` command that needs the project’s steps prepares Axx with the pack, like any other pack in `axx-packs.yaml` ([Choose packs](/guides/use-packs/)). ## Use the step [Section titled “Use the step”](#use-the-step) Once the pack is listed in `axx-packs.yaml`, its steps are used like any other step: `axx steps search` finds them, `axx validate` checks feature lines against them, and `axx skills install` adds them to the agent step index. # Configuration > The Axx configuration files - axx.yaml (every section, the apps and lint keys, how files and profiles merge, interpolation) and axx-packs.yaml. An acceptance project has up to two configuration files, side by side: `axx.yaml` for the project and `axx-packs.yaml` for the packs it uses. Both are optional. ## axx.yaml [Section titled “axx.yaml”](#axxyaml) Without an `axx.yaml`, Axx runs `features/` with defaults. The complete, versioned definition is the JSON Schema: * online: [`https://axx.nimbusxr.us/schemas/v0/axx.schema.json`](/schemas/v0/axx.schema.json) * offline: `axx schema` (or `axx schema --out axx.schema.json`) Add this first line to get completion and validation in editors that use the YAML language server (VS Code, IntelliJ, Neovim): ```yaml # yaml-language-server: $schema=https://axx.nimbusxr.us/schemas/v0/axx.schema.json ``` ### A complete example [Section titled “A complete example”](#a-complete-example) axx.yaml ```yaml # yaml-language-server: $schema=https://axx.nimbusxr.us/schemas/v0/axx.schema.json version: 1 run: paths: [features] # feature files or directories tags: "not @wip" # default tag expression workers: auto # parallel scenarios: a number or auto (CPUs) exclusive: ["@isolated"] # these run alone, after the parallel phase order: defined # or random, random: timeouts: {step: 60s, scenario: 5m, hook: 2m} reporters: [pretty, {junit: build/axx/junit.xml}] resources: ["."] # where seed, payload and schema paths in steps resolve properties: # ${sys:name}; override with -D name=value local.host: localhost openapi: levels: # suite-wide OpenAPI validation levels validation.request.security.missing: IGNORE active: enabled: false # start only the apps the selected scenarios' tags need onNoTags: fallback apps: api: dir: . command: docker compose up --build ready: http: {url: http://${sys:local.host}:8080/health} timeout: 120s cleanup: docker compose down -v --remove-orphans lint: {} # axx lint rules fixtures: {} # axx fixtures settings profiles: # overlays: --profile ci or AXX_PROFILE=ci ci: properties: {local.host: docker} ``` ### Sections [Section titled “Sections”](#sections) | Section | What it configures | Guide | | ---------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | `version` | the schema version of this file; currently `1` | | | `run` | paths, default tags, workers, exclusive tags, order, timeouts, default reporters | [Run in parallel](/guides/parallel-runs/), [Reports](/guides/reports/) | | `resources` | directories that relative file paths in steps resolve against, in order; the directory of `axx.yaml` is searched last | [Configure services](/guides/configure-services/#resource-paths) | | `properties` | values for `${sys:name}` | [Configure services](/guides/configure-services/#keep-values-out-of-feature-files) | | `openapi.levels` | default OpenAPI validation levels, by rule key | [Validate against OpenAPI](/guides/validate-openapi/) | | `active` | tag-based app startup | [Manage the app lifecycle](/guides/manage-app-lifecycle/#start-only-what-a-run-needs) | | `apps` | the system under test ([keys](#apps)) | [Manage the app lifecycle](/guides/manage-app-lifecycle/) | | `packs` | settings for packs, keyed by pack name | | | `lint` | test-data isolation rules ([keys](#lint)) | [Isolate test data](/guides/isolate-test-data/) | | `fixtures` | fixture factory settings | [Fixture factories](/guides/fixture-factories/) | | `profiles` | named overlays | [below](#finding-and-merging-files) | ### apps [Section titled “apps”](#apps) Each key under `apps:` names one app. | Key | Meaning | | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `command` | How to start the app: a string (split into words, no shell) or an argv list. A command that exits `0` before the app is ready is fine: Axx keeps polling `ready`. | | `shell` | `true` runs `command` and `cleanup` through the shell, for pipes and `&&`. | | `dir`, `env` | Working directory (relative to `axx.yaml`) and extra environment. | | `dependsOn` | Apps that must be ready first. Independent apps start in parallel. | | `enabled` | `false` skips the app. Defaults to `true`. | | `ready` | Checks that must all pass: `http.url` (every URL returns 2xx), `tcp` (`host:port` accepts connections), `exec` (a command exits 0), `log` (a regular expression matches the app’s output). `timeout` defaults to 60s, `interval` to 1s. | | `stop` | `signal` (`SIGTERM` by default, or `SIGINT`) is sent to the app’s process group; after `grace` (default 10s) it is killed. | | `cleanup` | Runs after the app stops, even if it crashed or never became ready. | | `active.tags` | With `active.enabled`, the app starts only when a selected scenario has one of these tags. | | `debug` | `command`, `debugger` (`type`, `port`, `mode`), `onUnavailable` and `retry` for `axx run --debug`. | ### lint [Section titled “lint”](#lint) Rules live under `lint.rules`; `lint.config` sets `baseDir` (patterns are relative to it) and the default `mode` (`error` fails, `warn` only reports); `lint.include` merges rule files such as `axx-lint.generated.yaml`. Each rule has: | Key | Meaning | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name`, `description` | Shown in the report. | | `filePatterns` | Globs (`*`, `**`, `?`, `[abc]`, `{a,b}`), relative to `baseDir`. A leading `../` reaches outside it. | | `excludePatterns` | Globs removed from the match, for example `**/*.fixture.yaml`. | | `type` | `regex` (default) or `jsonpath`. | | `regex` | The first capture group that matched is the value. | | `jsonPath` | A structural path in JSON files: `order.id`, `payments[0].id`, `payments[*].id`, optional `$.` prefix. | | `validation` | `global-unique` (default): every occurrence is unique. `file-unique`: unique within each file. `cross-file-unique`: may repeat inside a file, never across files. | | `mode` | Overrides the default mode for this rule. | | `ignoreValues` | Values that are shared on purpose. | ### Finding and merging files [Section titled “Finding and merging files”](#finding-and-merging-files) 1. With `-c path/to/axx.yaml`, that file. Otherwise Axx looks for `axx.yaml` (or `axx.yml`) in the working directory, then in each parent directory up to the repository root. 2. With `--profile NAME` or `AXX_PROFILE=NAME`, `profiles.NAME` is merged over the file, and then `axx.NAME.yaml` next to it, if either exists. A profile that exists in neither place is an error ([`AXX-E0104`](/references/error-codes/#axx-e0104)). 3. `axx.local.yaml` next to `axx.yaml`, if present, is merged last. Keep it out of version control for personal settings. 4. `-D name=value` overrides `properties`. Paths inside the file (`apps.*.dir`, `resources`) are relative to the directory of `axx.yaml`. ### Interpolation [Section titled “Interpolation”](#interpolation) String values can reference the environment and properties: | Syntax | Value | | ---------------------- | -------------------------------------------------------------------------- | | `${env:NAME}` | environment variable `NAME` | | `${sys:name}` | property `name` | | `${sys:name:-default}` | `default` when `name` is not set; defaults can nest (`${sys:a:-${env:B}}`) | | `$${...}` | a literal `${...}` | The same syntax works in feature files: in service tables and step arguments. ### Errors [Section titled “Errors”](#errors) A syntax error or an invalid value stops every command with exit code `2` and points at the line: [`AXX-E0101`](/references/error-codes/#axx-e0101) for YAML syntax and [`AXX-E0102`](/references/error-codes/#axx-e0102) for values the schema rejects. ## axx-packs.yaml [Section titled “axx-packs.yaml”](#axx-packsyaml) The packs a project uses: `core`, which is always there, and the listed packs. Without the file, a project has no steps. `axx pack add`, `remove` and `new` edit it, and `axx init` creates it. axx-packs.yaml ```yaml packs: - rest # one of Axx's packs, by name - sql - ./steps # a pack in this repository (a Go package) - github.com/team/axx-grpc@v1.2.0 # a pack from a Go module; the version is optional ``` `axx-packs.lock` pins the versions of module packs; commit both files. [Choose packs](/guides/use-packs/) shows the commands. # JSON output > The --json envelope every Axx command prints, the error objects inside it, and the run report that axx run and the agent reporter produce. Every command accepts `--json` and then prints exactly one JSON document on stdout: the **envelope**. It is a frozen contract ([ADR 0003](https://github.com/nimbusxr/axx/blob/main/docs/adr/0003-cli-contract.md)): fields may be added, but removing or retyping a field requires a new `schemaVersion`. ## The envelope [Section titled “The envelope”](#the-envelope) ```json { "schemaVersion": 1, "command": "axx steps search", "ok": true, "data": { }, "errors": [] } ``` | Field | Type | Meaning | | --------------- | ------- | -------------------------------------------------------------------- | | `schemaVersion` | number | Envelope version; currently `1`. | | `command` | string | The command that ran, such as `axx run` or `axx steps search`. | | `ok` | boolean | `true` when the command succeeded (exit code `0`). | | `data` | object | The command’s result. Absent when the command could not produce one. | | `errors` | array | Errors that stopped the command. Absent or empty on success. | The process [exit code](/references/error-codes/#exit-codes) is the same with or without `--json`. ## Errors [Section titled “Errors”](#errors) ```json { "schemaVersion": 1, "command": "axx run", "ok": false, "errors": [ { "code": "AXX-E0202", "message": "invalid tag expression \"@x and\": Tag expression \"@x and\" could not be parsed because of syntax error: Expected operand.", "hint": "use expressions like \"@smoke and not @wip\"", "docs": "https://axx.nimbusxr.us/references/error-codes/#axx-e0202" } ] } ``` | Field | Meaning | | ---------- | --------------------------------------------------------------------------------------------------------- | | `code` | A stable `AXX-Exxxx` code; see [error codes](/references/error-codes/). Branch on this, not on `message`. | | `message` | What went wrong, for humans. | | `location` | Optional `{file, line, column}`, for example a line in `axx.yaml` or a feature file. | | `hint` | Optional: what to do about it. | | `docs` | A link to the code’s entry in the error code reference. | ## The run report [Section titled “The run report”](#the-run-report) `axx run --json` puts the run report in `data`. `--format agent:FILE` writes the same object to a file. Only failures are listed, so the size depends on what broke, not on the size of the suite. ```json { "schemaVersion": 1, "axx": "0.1.0", "result": "failed", "durationMs": 7, "counts": { "scenarios": { "failed": 1, "passed": 1 }, "steps": { "failed": 1, "passed": 2 } }, "notRun": 0, "interrupted": false, "failures": [ { "scenario": "fails", "location": "features/demo.feature:6", "status": "failed", "step": { "keyword": "Then", "text": "the response status code is 201", "location": "features/demo.feature:9", "definition": "rest.response.status" }, "error": { "kind": "assertion", "message": "Expected status code <201> but was <200>.", "expected": 201, "actual": 200 }, "rerun": "axx run features/demo.feature:6" } ], "undefined": [] } ``` | Field | Meaning | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `result` | `passed` or `failed`, summarizing the run | | `durationMs` | wall-clock time of the run | | `counts` | scenarios and steps by status (`passed`, `failed`, `skipped`, `undefined`, `ambiguous`, `pending`); zero counts are omitted | | `notRun` | selected scenarios that did not run, for example after `--fail-fast` or an interrupt | | `interrupted` | `true` when the run was cancelled | | `failures[]` | one entry per scenario that did not pass | | `failures[].location`, `.rerun` | where the scenario is, and the command that reruns only it | | `failures[].step` | the step that failed: keyword, text, location and the id of the matched `definition` | | `failures[].error.kind` | `assertion`, `error`, `timeout`, `panic`, `undefined`, `ambiguous` or `pending` | | `failures[].error.expected`, `.actual` | for assertions: the two values, as JSON | | `failures[].logs`, `.context` | logs attached to the step, and pack context such as the last HTTP request and response | | `undefined[]` | undefined steps with suggestions, for writing them correctly | | `runErrors[]` | problems found after the scenarios, outside any of them (`source`: the pack, `message`); present only when there are some, and they make `result` `failed` | ## Other commands [Section titled “Other commands”](#other-commands) Each command documents its own `data`. The most useful for scripts and agents: | Command | `data` | | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `axx steps search ` | `{query, steps[]}`; each step has `id`, `pack`, `expr`, `variants`, `keyword`, `arg`, `doc`, `examples`, `params` | | `axx validate` | `{files, scenarios, steps, problems[]}`; each problem has `kind` (`undefined`, `ambiguous`, `argument`), `location`, `text`, and `suggestions` | | `axx doctor` | `{version, config, checks[]}`; each check has `name`, `status` (`ok`, `warn`, `fail`), `detail`, `hint` | | `axx docs export` | `{files[]}` | For editors and generated clients, the `axx.yaml` schema is at [`/schemas/v0/axx.schema.json`](/schemas/v0/axx.schema.json). # All steps > Every step Axx understands, one per line: pack, id and expression. Search it before writing a step. Every step Axx understands, one per line: `pack | id | expression`. `{name}` is a parameter; see [parameter types](/references/steps/#parameter-types). Search this file instead of inventing step text. ```text rest | rest.service | the {word} service with the following properties: [+table] rest | rest.openapi.levels | the OpenAPI validation levels are: [+table] rest | rest.openapi.levels | the OpenAPI validation levels on {service} are: [+table] rest | rest.request | a(n) {word} request to {word} rest | rest.request | a(n) {word} request to {word} on {service} rest | rest.request.ordered | a {ordinal} ordered {word} request to {word} rest | rest.request.ordered | a {ordinal} ordered {word} request to {word} on {service} rest | rest.request.header | the request header {word} is {string} rest | rest.request.header | the request header {word} is {string} for {ordinal} ordered request rest | rest.request.header.on | the request header {word} is {string} for request on {service} rest | rest.request.header.on | the request header {word} is {string} for {ordinal} ordered request on {service} rest | rest.request.headers | the request headers are: [+table] rest | rest.request.headers | the request headers for {ordinal} ordered request are: [+table] rest | rest.request.headers.on | the request headers for request on {service} are: [+table] rest | rest.request.headers.on | the request headers for {ordinal} ordered request on {service} are: [+table] rest | rest.request.payload.empty | a request payload using a(n) {mimeType} empty content template rest | rest.request.payload.empty | a request payload using a(n) {mimeType} empty content template for {ordinal} ordered request rest | rest.request.payload.empty.on | a request payload using a(n) {mimeType} empty content template for request on {service} rest | rest.request.payload.empty.on | a request payload using a(n) {mimeType} empty content template for {ordinal} ordered request on {service} rest | rest.request.payload.example | a request payload using a(n) {mimeType} content example rest | rest.request.payload.example | a request payload using a(n) {mimeType} content example named {string} rest | rest.request.payload.example | a request payload using a(n) {mimeType} content example for {ordinal} ordered request rest | rest.request.payload.example | a request payload using a(n) {mimeType} content example named {string} for {ordinal} ordered request rest | rest.request.payload.example.on | a request payload using a(n) {mimeType} content example for request on {service} rest | rest.request.payload.example.on | a request payload using a(n) {mimeType} content example named {string} for request on {service} rest | rest.request.payload.example.on | a request payload using a(n) {mimeType} content example for {ordinal} ordered request on {service} rest | rest.request.payload.example.on | a request payload using a(n) {mimeType} content example named {string} for {ordinal} ordered request on {service} rest | rest.request.property | the request payload property {word} is {string} rest | rest.request.property | the request payload property {word} is {string} for {ordinal} ordered request rest | rest.request.property.on | the request payload property {word} is {string} for request on {service} rest | rest.request.property.on | the request payload property {word} is {string} for {ordinal} ordered request on {service} rest | rest.request.properties | the request payload properties are: [+table] rest | rest.request.properties | the request payload properties for {ordinal} ordered request are: [+table] rest | rest.request.properties.on | the request payload properties for request on {service} are: [+table] rest | rest.request.properties.on | the request payload properties for {ordinal} ordered request on {service} are: [+table] rest | rest.request.property.null | the request payload property {word} is null rest | rest.request.property.null | the request payload property {word} is null for {ordinal} ordered request rest | rest.request.property.null.on | the request payload property {word} is null for request on {service} rest | rest.request.property.null.on | the request payload property {word} is null for {ordinal} ordered request on {service} rest | rest.execute | the request is executed rest | rest.execute | the {ordinal} ordered request is executed rest | rest.execute | the request is executed on {service} rest | rest.execute | the {ordinal} ordered request is executed on {service} rest | rest.response.status | the response status code is {int} rest | rest.response.status | the {ordinal} ordered response status code is {int} rest | rest.response.status | the response status code is {int} on {service} rest | rest.response.status | the {ordinal} ordered response status code is {int} on {service} rest | rest.response.body.contains | the response body contains {string} rest | rest.response.body.contains | the response body contains {string} for {ordinal} ordered response rest | rest.response.body.contains.on | the response body contains {string} for response on {service} rest | rest.response.body.contains.on | the response body contains {string} for {ordinal} ordered response on {service} rest | rest.response.header.is | the response header {word} is {string} rest | rest.response.header.is | the response header {word} is {string} for {ordinal} ordered response rest | rest.response.header.is.on | the response header {word} is {string} for response on {service} rest | rest.response.header.is.on | the response header {word} is {string} for {ordinal} ordered response on {service} rest | rest.response.header.matches | the response header {word} matches {pattern} rest | rest.response.header.matches | the response header {word} matches {pattern} for {ordinal} ordered response rest | rest.response.header.matches.on | the response header {word} matches {pattern} for response on {service} rest | rest.response.header.matches.on | the response header {word} matches {pattern} for {ordinal} ordered response on {service} rest | rest.response.header.missing | the response header {word} is missing rest | rest.response.header.missing | the response header {word} is missing for {ordinal} ordered response rest | rest.response.header.missing.on | the response header {word} is missing for response on {service} rest | rest.response.header.missing.on | the response header {word} is missing for {ordinal} ordered response on {service} rest | rest.response.headers.are | the response headers are: [+table] rest | rest.response.headers.are | the response headers for {ordinal} ordered response are: [+table] rest | rest.response.headers.are.on | the response headers for response on {service} are: [+table] rest | rest.response.headers.are.on | the response headers for {ordinal} ordered response on {service} are: [+table] rest | rest.response.headers.match | the response headers match: [+table] rest | rest.response.headers.match | the response headers for {ordinal} ordered response match: [+table] rest | rest.response.headers.match.on | the response headers for response on {service} match: [+table] rest | rest.response.headers.match.on | the response headers for {ordinal} ordered response on {service} match: [+table] rest | rest.response.headers.missing | the response headers are missing: [+table] rest | rest.response.headers.missing | the response headers for {ordinal} ordered response are missing: [+table] rest | rest.response.headers.missing.on | the response headers for response on {service} are missing: [+table] rest | rest.response.headers.missing.on | the response headers for {ordinal} ordered response on {service} are missing: [+table] rest | rest.response.property.is | the response payload property {word} is {string} rest | rest.response.property.is | the response payload property {word} is {string} for {ordinal} ordered response rest | rest.response.property.is.on | the response payload property {word} is {string} for response on {service} rest | rest.response.property.is.on | the response payload property {word} is {string} for {ordinal} ordered response on {service} rest | rest.response.property.null | the response payload property {word} is null rest | rest.response.property.null | the response payload property {word} is null for {ordinal} ordered response rest | rest.response.property.null.on | the response payload property {word} is null for response on {service} rest | rest.response.property.null.on | the response payload property {word} is null for {ordinal} ordered response on {service} rest | rest.response.property.undefined | the response payload property {word} is undefined rest | rest.response.property.undefined | the response payload property {word} is undefined for {ordinal} ordered response rest | rest.response.property.undefined.on | the response payload property {word} is undefined for response on {service} rest | rest.response.property.undefined.on | the response payload property {word} is undefined for {ordinal} ordered response on {service} rest | rest.response.property.matches | the response payload property {word} matches {pattern} rest | rest.response.property.matches | the response payload property {word} matches {pattern} for {ordinal} ordered response rest | rest.response.property.matches.on | the response payload property {word} matches {pattern} for response on {service} rest | rest.response.property.matches.on | the response payload property {word} matches {pattern} for {ordinal} ordered response on {service} rest | rest.response.properties.are | the response payload properties are: [+table] rest | rest.response.properties.are | the response payload properties for {ordinal} ordered response are: [+table] rest | rest.response.properties.are.on | the response payload properties for response on {service} are: [+table] rest | rest.response.properties.are.on | the response payload properties for {ordinal} ordered response on {service} are: [+table] rest | rest.response.properties.match | the response payload properties match: [+table] rest | rest.response.properties.match | the response payload properties for {ordinal} ordered response match: [+table] rest | rest.response.properties.match.on | the response payload properties for response on {service} match: [+table] rest | rest.response.properties.match.on | the response payload properties for {ordinal} ordered response on {service} match: [+table] mock | mock.service | the mocked {word} service with the following properties: [+table] mock | mock.received | the mocked {word} request to {word} named {word} was received by {mockedService} mock | mock.openapi.levels | the OpenAPI validation levels for the mocked {mockedService} service are: [+table] mock | mock.count.exactly | the mocked request named {word} was received exactly {int} time(s) mock | mock.count.exactly | the mocked request named {word} on {mockedService} was received exactly {int} time(s) mock | mock.count.atLeast | the mocked request named {word} was received at least {int} time(s) mock | mock.count.atLeast | the mocked request named {word} on {mockedService} was received at least {int} time(s) mock | mock.count.atMost | the mocked request named {word} was received at most {int} time(s) mock | mock.count.atMost | the mocked request named {word} on {mockedService} was received at most {int} time(s) mock | mock.notReceived | the mocked {word} request to {word} named {word} was not received mock | mock.notReceived | the mocked {word} request to {word} named {word} on {mockedService} was not received mock | mock.header.is | the header {word} for mocked request named {word} is {string} mock | mock.header.is | the header {word} for mocked request named {word} on {mockedService} is {string} mock | mock.headers.are | the headers for mocked request named {word} on {mockedService} are: [+table] mock | mock.header.matches | the header {word} for mocked request named {word} on {mockedService} matches {pattern} mock | mock.headers.match | the headers for mocked request named {word} on {mockedService} match: [+table] mock | mock.header.missing | the header {word} for mocked request named {word} on {mockedService} is missing mock | mock.headers.missing | the headers for mocked request named {word} on {mockedService} are missing: [+table] sql | sql.service | a(n) {word} database with the following properties: [+table] sql | sql.seed | a {filepath} db seed sql | sql.seed | a {filepath} db seed on {dbService} sql | sql.lock | the rows in the {word} table are locked where: [+table] sql | sql.lock | the rows in the {word} table on {dbService} are locked where: [+table] sql | sql.unlock | the row locks are released sql | sql.unlock | the row locks on {dbService} are released sql | sql.select | a selection of rows is retrieved from the {word} table where: [+table] sql | sql.select | a {ordinal} selection of rows is retrieved from the {word} table where: [+table] sql | sql.select | a selection of rows is retrieved from the {word} table on {dbService} where: [+table] sql | sql.select | a {ordinal} selection of rows is retrieved from the {word} table on {dbService} where: [+table] sql | sql.select.poll | within {duration} a selection of at least {int} row(s) is retrieved from the {word} table where: [+table] sql | sql.select.poll | within {duration} a {ordinal} selection of at least {int} row(s) is retrieved from the {word} table where: [+table] sql | sql.select.poll | within {duration} a selection of at least {int} row(s) is retrieved from the {word} table on {dbService} where: [+table] sql | sql.select.poll | within {duration} a {ordinal} selection of at least {int} row(s) is retrieved from the {word} table on {dbService} where: [+table] sql | sql.select.jsonb | a selection of rows is retrieved from the {word} table where the {word} jsonb column contains: [+table] sql | sql.select.jsonb | a {ordinal} selection of rows is retrieved from the {word} table where the {word} jsonb column contains: [+table] sql | sql.select.jsonb | a selection of rows is retrieved from the {word} table on {dbService} where the {word} jsonb column contains: [+table] sql | sql.select.jsonb | a {ordinal} selection of rows is retrieved from the {word} table on {dbService} where the {word} jsonb column contains: [+table] sql | sql.json.are | the {ordinal} row {word} property for the selection json properties are: [+table] sql | sql.json.are | the {ordinal} row {word} property for the {ordinal} selection json properties are: [+table] sql | sql.json.are | the {ordinal} row {word} property for the selection on {dbService} json properties are: [+table] sql | sql.json.are | the {ordinal} row {word} property for the {ordinal} selection on {dbService} json properties are: [+table] sql | sql.json.match | the {ordinal} row {word} property for the selection json properties match: [+table] sql | sql.json.match | the {ordinal} row {word} property for the {ordinal} selection json properties match: [+table] sql | sql.json.match | the {ordinal} row {word} property for the selection on {dbService} json properties match: [+table] sql | sql.json.match | the {ordinal} row {word} property for the {ordinal} selection on {dbService} json properties match: [+table] sql | sql.rows.eq | the selection has {int} row(s) sql | sql.rows.eq | the {ordinal} selection has {int} row(s) sql | sql.rows.eq | the selection on {dbService} has {int} row(s) sql | sql.rows.eq | the {ordinal} selection on {dbService} has {int} row(s) sql | sql.rows.gt | the selection has more than {int} row(s) sql | sql.rows.gt | the {ordinal} selection has more than {int} row(s) sql | sql.rows.gt | the selection on {dbService} has more than {int} row(s) sql | sql.rows.gt | the {ordinal} selection on {dbService} has more than {int} row(s) sql | sql.rows.lt | the selection has fewer than {int} row(s) sql | sql.rows.lt | the {ordinal} selection has fewer than {int} row(s) sql | sql.rows.lt | the selection on {dbService} has fewer than {int} row(s) sql | sql.rows.lt | the {ordinal} selection on {dbService} has fewer than {int} row(s) sql | sql.trigger.raise | a(n) before insert trigger on the {word} table will raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.raise | a(n) {ordinal} ordered before insert trigger on the {word} table will raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.raise | a(n) before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.raise | a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.raise.times | a(n) before insert trigger on the {word} table will raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.raise.times | a(n) {ordinal} ordered before insert trigger on the {word} table will raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.raise.times | a(n) before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.raise.times | a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.insertRaise | a(n) before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.insertRaise | a(n) {ordinal} ordered before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.insertRaise | a(n) before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.insertRaise | a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception where: [+table] sql | sql.trigger.insertRaise.times | a(n) before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.insertRaise.times | a(n) {ordinal} ordered before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.insertRaise.times | a(n) before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.insertRaise.times | a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception {int} time(s) where: [+table] sql | sql.trigger.raised | the before insert trigger on the {word} table was raised {int} time(s) sql | sql.trigger.raised | the {ordinal} ordered before insert trigger on the {word} table was raised {int} time(s) sql | sql.trigger.raised | the before insert trigger on the {word} table on {dbService} was raised {int} time(s) sql | sql.trigger.raised | the {ordinal} ordered before insert trigger on the {word} table on {dbService} was raised {int} time(s) mongo | mongo.service | a(n) {word} mongo database with the following properties: [+table] mongo | mongo.seed | a {filepath} mongo db seed mongo | mongo.seed.named | a {filepath} MongoDB seed for {word} mongo | mongo.seed.named.alt | a {filepath} mongo db seed for {word} mongo | mongo.find | a selection of documents is retrieved from the {word} collection where: [+table] mongo | mongo.find | a {ordinal} selection of documents is retrieved from the {word} collection where: [+table] mongo | mongo.find | a selection of documents is retrieved from the {word} collection on {mongoService} where: [+table] mongo | mongo.find | a {ordinal} selection of documents is retrieved from the {word} collection on {mongoService} where: [+table] mongo | mongo.find.poll | within {duration} a selection of at least {int} document(s) is retrieved from the {word} collection where: [+table] mongo | mongo.find.poll | within {duration} a {ordinal} selection of at least {int} document(s) is retrieved from the {word} collection where: [+table] mongo | mongo.find.poll | within {duration} a selection of at least {int} document(s) is retrieved from the {word} collection on {mongoService} where: [+table] mongo | mongo.find.poll | within {duration} a {ordinal} selection of at least {int} document(s) is retrieved from the {word} collection on {mongoService} where: [+table] mongo | mongo.docs.eq | the selection has {int} document(s) mongo | mongo.docs.eq | the {ordinal} selection has {int} document(s) mongo | mongo.docs.eq | the selection on {mongoService} has {int} document(s) mongo | mongo.docs.eq | the {ordinal} selection on {mongoService} has {int} document(s) mongo | mongo.docs.gt | the selection has more than {int} document(s) mongo | mongo.docs.gt | the {ordinal} selection has more than {int} document(s) mongo | mongo.docs.gt | the selection on {mongoService} has more than {int} document(s) mongo | mongo.docs.gt | the {ordinal} selection on {mongoService} has more than {int} document(s) mongo | mongo.docs.lt | the selection has fewer than {int} document(s) mongo | mongo.docs.lt | the {ordinal} selection has fewer than {int} document(s) mongo | mongo.docs.lt | the selection on {mongoService} has fewer than {int} document(s) mongo | mongo.docs.lt | the {ordinal} selection on {mongoService} has fewer than {int} document(s) mongo | mongo.doc.are | the {ordinal} document for the selection properties are: [+table] mongo | mongo.doc.are | the {ordinal} document for the {ordinal} selection properties are: [+table] mongo | mongo.doc.are | the {ordinal} document for the selection on {mongoService} properties are: [+table] mongo | mongo.doc.are | the {ordinal} document for the {ordinal} selection on {mongoService} properties are: [+table] mongo | mongo.doc.match | the {ordinal} document for the selection properties match: [+table] mongo | mongo.doc.match | the {ordinal} document for the {ordinal} selection properties match: [+table] mongo | mongo.doc.match | the {ordinal} document for the selection on {mongoService} properties match: [+table] mongo | mongo.doc.match | the {ordinal} document for the {ordinal} selection on {mongoService} properties match: [+table] kafka | kafka.service | the {word} kafka service with the following properties: [+table] kafka | kafka.client | the {word} kafka topic client kafka | kafka.client.props | a(n) {word} kafka topic client with the following properties: [+table] kafka | kafka.client.props | a(n) {word} kafka topic client on the {word} kafka service with the following properties: [+table] kafka | kafka.event | a(n) {word} kafka event kafka | kafka.event | a(n) {ordinal} ordered {word} kafka event kafka | kafka.event | a(n) {word} kafka event on {word} kafka service kafka | kafka.event | a(n) {ordinal} ordered {word} kafka event on {word} kafka service kafka | kafka.event.key | the {word} kafka event key is {word} kafka | kafka.event.key | the {ordinal} ordered {word} kafka event key is {word} kafka | kafka.event.key | the {word} kafka event key is {word} on the {word} kafka service kafka | kafka.event.key | the {ordinal} ordered {word} kafka event key is {word} on the {word} kafka service kafka | kafka.event.headers | the {word} kafka event headers are: [+table] kafka | kafka.event.headers | the {ordinal} ordered {word} kafka event headers are: [+table] kafka | kafka.event.headers | the {word} kafka event headers on the {word} kafka service are: [+table] kafka | kafka.event.headers | the {ordinal} ordered {word} kafka event headers on the {word} kafka service are: [+table] kafka | kafka.event.payload.resource | the {word} kafka event payload is a(n) {filepath} resource kafka | kafka.event.payload.resource | the {ordinal} ordered {word} kafka event payload is a(n) {filepath} resource kafka | kafka.event.payload.resource | the {word} kafka event payload is a(n) {filepath} resource on the {word} kafka service kafka | kafka.event.payload.resource | the {ordinal} ordered {word} kafka event payload is a(n) {filepath} resource on the {word} kafka service kafka | kafka.event.properties.first | the kafka event payload properties are: [+table] kafka | kafka.event.properties.first | the {ordinal} ordered kafka event payload properties are: [+table] kafka | kafka.event.properties.first | the kafka event payload properties on the {word} kafka service are: [+table] kafka | kafka.event.properties.first | the {ordinal} ordered kafka event payload properties on the {word} kafka service are: [+table] kafka | kafka.event.properties | the {word} kafka event payload properties are: [+table] kafka | kafka.event.properties | the {word} kafka event payload properties on the {word} kafka service are: [+table] kafka | kafka.event.properties.ordinal | the {ordinal} ordered {word} kafka event payload properties are: [+table] kafka | kafka.event.properties.ordinal | the {ordinal} ordered {word} kafka event payload properties on the {word} kafka service are: [+table] kafka | kafka.event.property.null | the {word} kafka event payload property {word} is null kafka | kafka.event.property.null | the {ordinal} ordered {word} kafka event payload property {word} is null kafka | kafka.event.property.null | the {word} kafka event payload property {word} is null on the {word} kafka service kafka | kafka.event.property.null | the {ordinal} ordered {word} kafka event payload property {word} is null on the {word} kafka service kafka | kafka.event.publish.schema | the {word} kafka event is published using schema {filepath} kafka | kafka.event.publish.schema | the {ordinal} ordered {word} kafka event is published using schema {filepath} kafka | kafka.event.publish.schema | the {word} kafka event is published using schema {filepath} on the {word} kafka service kafka | kafka.event.publish.schema | the {ordinal} ordered {word} kafka event is published using schema {filepath} on the {word} kafka service kafka | kafka.event.publish | the {word} kafka event is published kafka | kafka.event.publish | the {ordinal} ordered {word} kafka event is published kafka | kafka.event.publish | the {word} kafka event is published on the {word} kafka service kafka | kafka.event.publish | the {ordinal} ordered {word} kafka event is published on the {word} kafka service kafka | kafka.consumed.key | the {word} kafka event named {word} key is {word} kafka | kafka.consumed.key | the {word} kafka event named {word} key is {word} on the {word} kafka service kafka | kafka.consumed.properties | the {word} kafka event named {word} payload properties are: [+table] kafka | kafka.consumed.properties | the {word} kafka event named {word} payload properties on the {word} kafka service are: [+table] kafka | kafka.consumed.headers | the {word} kafka event named {word} headers are: [+table] kafka | kafka.consumed.headers | the {word} kafka event named {word} headers on the {word} kafka service are: [+table] kafka | kafka.consumed.headers.match | the {word} kafka event named {word} headers match: [+table] kafka | kafka.consumed.headers.match.service | the {word} kafka event named {word} headers on the {word} kafka service match: [+table] logs | logs.log | the {word} log with the following properties: [+table] logs | logs.entry | the {word} log has an entry matching {string} logs | logs.entry | within {duration} the {word} log has an entry matching {string} logs | logs.entries | the {word} log has entries matching: [+table] logs | logs.entries | within {duration} the {word} log has entries matching: [+table] logs | logs.count | the {word} log has {int} entry/entries matching {string} logs | logs.count | within {duration} the {word} log has {int} entry/entries matching {string} logs | logs.across | the logs have entries matching: [+table] logs | logs.across | within {duration} the logs have entries matching: [+table] aws-core | aws-core.account | the {word} aws account with the following properties: [+table] aws-s3 | aws-s3.upload | the {filepath} file is uploaded to the {word} s3 bucket aws-s3 | aws-s3.upload | the {filepath} file is uploaded to the {word} s3 bucket as {word} aws-s3 | aws-s3.has | the {word} s3 bucket has a(n) object named {word} aws-s3 | aws-s3.has | within {duration} the {word} s3 bucket has a(n) object named {word} aws-s3 | aws-s3.identical | the {word} object in the {word} s3 bucket is identical to the {filepath} file aws-s3 | aws-s3.identical | within {duration} the {word} object in the {word} s3 bucket is identical to the {filepath} file aws-s3 | aws-s3.properties | the {word} object in the {word} s3 bucket has the following properties: [+table] aws-s3 | aws-s3.properties | within {duration} the {word} object in the {word} s3 bucket has the following properties: [+table] aws-sqs | aws-sqs.send | a message is sent to the {word} sqs queue: [+docstring] aws-sqs | aws-sqs.send.file | the {filepath} message is sent to the {word} sqs queue aws-sqs | aws-sqs.send.file | the {filepath} message is sent to the {word} sqs queue with the following attributes: aws-sqs | aws-sqs.received | the {word} sqs queue has a message where: [+table] aws-sqs | aws-sqs.received | within {duration} the {word} sqs queue has a message where: [+table] aws-sns | aws-sns.send | a message is published to the {word} sns topic: [+docstring] aws-sns | aws-sns.send.file | the {filepath} message is published to the {word} sns topic aws-sns | aws-sns.send.file | the {filepath} message is published to the {word} sns topic with the following attributes: aws-sns | aws-sns.received | the {word} sns topic has a message where: [+table] aws-sns | aws-sns.received | within {duration} the {word} sns topic has a message where: [+table] aws-eventbridge | aws-eventbridge.put | a(n) {string} event from {word} is put on the {word} eventbridge bus: [+docstring] aws-eventbridge | aws-eventbridge.received | the {word} eventbridge bus has an event where: [+table] aws-eventbridge | aws-eventbridge.received | within {duration} the {word} eventbridge bus has an event where: [+table] aws-dynamodb | aws-dynamodb.seed | a {filepath} dynamodb seed aws-dynamodb | aws-dynamodb.item | the {word} dynamodb table has an item where: [+table] aws-dynamodb | aws-dynamodb.item | within {duration} the {word} dynamodb table has an item where: [+table] aws-dynamodb | aws-dynamodb.items | the {word} dynamodb table has {int} item(s) where: [+table] aws-dynamodb | aws-dynamodb.items | within {duration} the {word} dynamodb table has {int} item(s) where: [+table] gcp-core | gcp-core.project | the {word} gcp project with the following properties: [+table] gcp-storage | gcp-storage.upload | the {filepath} file is uploaded to the {word} gcs bucket gcp-storage | gcp-storage.upload | the {filepath} file is uploaded to the {word} gcs bucket as {word} gcp-storage | gcp-storage.has | the {word} gcs bucket has a(n) object named {word} gcp-storage | gcp-storage.has | within {duration} the {word} gcs bucket has a(n) object named {word} gcp-storage | gcp-storage.identical | the {word} object in the {word} gcs bucket is identical to the {filepath} file gcp-storage | gcp-storage.identical | within {duration} the {word} object in the {word} gcs bucket is identical to the {filepath} file gcp-storage | gcp-storage.properties | the {word} object in the {word} gcs bucket has the following properties: [+table] gcp-storage | gcp-storage.properties | within {duration} the {word} object in the {word} gcs bucket has the following properties: [+table] gcp-pubsub | gcp-pubsub.send | a message is published to the {word} pubsub topic: [+docstring] gcp-pubsub | gcp-pubsub.send.file | the {filepath} message is published to the {word} pubsub topic gcp-pubsub | gcp-pubsub.send.file | the {filepath} message is published to the {word} pubsub topic with the following attributes: gcp-pubsub | gcp-pubsub.received | the {word} pubsub topic has a message where: [+table] gcp-pubsub | gcp-pubsub.received | within {duration} the {word} pubsub topic has a message where: [+table] gcp-bigquery | gcp-bigquery.seed | a {filepath} bigquery seed gcp-bigquery | gcp-bigquery.row | the {word} bigquery table has a row where: [+table] gcp-bigquery | gcp-bigquery.row | within {duration} the {word} bigquery table has a row where: [+table] gcp-bigquery | gcp-bigquery.rows | the {word} bigquery table has {int} row(s) where: [+table] gcp-bigquery | gcp-bigquery.rows | within {duration} the {word} bigquery table has {int} row(s) where: [+table] gcp-firestore | gcp-firestore.seed | a {filepath} firestore seed gcp-firestore | gcp-firestore.document | the {word} firestore document has the following properties: [+table] gcp-firestore | gcp-firestore.document | within {duration} the {word} firestore document has the following properties: [+table] gcp-firestore | gcp-firestore.collection | the {word} firestore collection has a document where: [+table] gcp-firestore | gcp-firestore.collection | within {duration} the {word} firestore collection has a document where: [+table] azure-blob | azure-blob.account | the {word} azure storage account with the following properties: [+table] azure-blob | azure-blob.upload | the {filepath} file is uploaded to the {word} blob container azure-blob | azure-blob.upload | the {filepath} file is uploaded to the {word} blob container as {word} azure-blob | azure-blob.has | the {word} blob container has a(n) blob named {word} azure-blob | azure-blob.has | within {duration} the {word} blob container has a(n) blob named {word} azure-blob | azure-blob.identical | the {word} blob in the {word} blob container is identical to the {filepath} file azure-blob | azure-blob.identical | within {duration} the {word} blob in the {word} blob container is identical to the {filepath} file azure-blob | azure-blob.properties | the {word} blob in the {word} blob container has the following properties: [+table] azure-blob | azure-blob.properties | within {duration} the {word} blob in the {word} blob container has the following properties: [+table] azure-servicebus | azure-servicebus.namespace | the {word} service bus namespace with the following properties: [+table] azure-servicebus | azure-servicebus.send | a message is sent to the {word} service bus queue:/topic: [+docstring] azure-servicebus | azure-servicebus.send.file | the {filepath} message is sent to the {word} service bus queue/topic azure-servicebus | azure-servicebus.send.file | the {filepath} message is sent to the {word} service bus queue/topic with the following properties: azure-servicebus | azure-servicebus.received | the {word} service bus queue/topic has a message where: [+table] azure-servicebus | azure-servicebus.received | within {duration} the {word} service bus queue/topic has a message where: [+table] ``` # How steps read > The grammar every step of Axx's packs follows (optional parts, services, ordinals, arguments) and the parameter types. Every step of Axx’s packs follows the same grammar. The pack pages list each step with its variants, parameters and an example. ## Optional parts [Section titled “Optional parts”](#optional-parts) An expression writes optional parts as `[[ ... ]]`. A step matches with or without each of them. `the[[ {ordinal} ordered]] response status code is {int}[[ on {service}]]` matches: * `the response status code is 200` * `the 2nd ordered response status code is 201` * `the response status code is 200 on parcels` * `the 2nd ordered response status code is 201 on parcels` `axx steps show ` prints every variant of a step. ## Services [Section titled “Services”](#services) A step without a service name uses the default service: the first one of its kind registered in the scenario. The named form (`on parcels`, `on parcels-db`, `on the events kafka service`) picks another. ## Ordinals [Section titled “Ordinals”](#ordinals) `1st`, `2nd`, `3rd`, `4th`, … count requests, selections and events within a scenario, starting at 1. A step without an ordinal uses the first. ## Arguments [Section titled “Arguments”](#arguments) * `{string}` takes single or double quotes; the quotes are removed. * `a(n)`, `time(s)`, `row(s)` and `document(s)` are optional text: `1 time` and `2 times` both match. * A step that ends with `:` takes a two-column data table on the following lines. * How a value is typed (string, number, `null`, `undefined`) depends on the step; each step’s entry says how it reads its values. ## Parameter types [Section titled “Parameter types”](#parameter-types) | Parameter | Matches | Description | Provided by | | ----------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | | `{(anonymous)}` | `.*` | anonymous: any text | cucumber | | `{bigdecimal}` | `[-+]?(?:\d+(?:\.\d+)?\|\.\d+)(?:[E][+-]?\d+)?` | an arbitrary-precision decimal | cucumber | | `{biginteger}` | `-?\d+` or `\d+` | an arbitrary-precision integer | cucumber | | `{byte}` | `-?\d+` or `\d+` | an 8-bit integer | cucumber | | `{dbService}` | `([^\s]+)` | The name of a database registered in the scenario. | sql | | `{double}` | `[-+]?(?:\d+(?:\.\d+)?\|\.\d+)(?:[E][+-]?\d+)?` | a 64-bit float | cucumber | | `{duration}` | `(\d+)(s\|m)` | A duration in seconds or minutes, e.g. `5s` or `2m`. | core | | `{filepath}` | `([^\s]+)` | A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file. | core | | `{float}` | `[-+]?(?:\d+(?:\.\d+)?\|\.\d+)(?:[E][+-]?\d+)?` | a 32-bit float | cucumber | | `{int}` | `-?\d+` or `\d+` | a 32-bit integer | cucumber | | `{long}` | `-?\d+` or `\d+` | a 64-bit integer | cucumber | | `{mimeType}` | `([^\s]+)` | One of `application/json`, `text/json`, `application/problem+json`, `application/x-www-form-urlencoded`. | core | | `{mockedService}` | `([^\s]+)` | The name of a mocked service registered in the scenario. | mock | | `{mongoService}` | `([^\s]+)` | The name of a MongoDB database registered in the scenario. | mongo | | `{ordinal}` | `(\d+)(?:st\|nd\|rd\|th)` | A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first. | core | | `{pattern}` | `([^\s]+)` | A regular expression (Java syntax) without whitespace. It must match the whole value. | core | | `{service}` | `([^\s]+)` | The name of a REST service registered in the scenario. | rest | | `{short}` | `-?\d+` or `\d+` | a 16-bit integer | cucumber | | `{sqlState}` | `[0-9A-Za-z]{5}` | A five-character SQLSTATE code, e.g. `23505` (unique violation). | sql | | `{string}` | `"([^"\\]*(\\.[^"\\]*)*)"\|'([^'\\]*(\\.[^'\\]*)*)'` | text in single or double quotes; the quotes are removed | cucumber | | `{word}` | `[^\s]+` | one word, no spaces | cucumber | # AWS account > The AWS account the aws-* packs talk to, set up the way the AWS SDK is set up for the real services. The AWS account the aws-\* packs talk to, set up the way the AWS SDK is set up for the real services. Register the account once with `the {word} aws account with the following properties:`; every aws-\* step of the scenario uses it (the first account registered is the default). | Property | | | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `region` | required, e.g. `eu-west-1` | | `endpoint` | where every service of the account is, instead of AWS: a local emulator such as `http://localhost:4566` (S3 is then addressed by path) | | `profile` | a profile of the shared AWS config and credentials files | | `access key id`, `secret access key`, `session token` | static credentials | Without credentials in the table, the SDK finds them as it always does: the `AWS_*` environment variables, the shared files, then the container or instance role. `AWS_ENDPOINT_URL` is honored too, so the same features run against AWS and against an emulator. Values expand `${env:..}` and `${sys:..}`. ## `aws-core.account` [Section titled “aws-core.account”](#aws-coreaccount) ```gherkin Given the {word} aws account with the following properties: | ... | ... | ``` Register the AWS account the aws-\* steps talk to: `region` (required), `endpoint` (an emulator), `profile`, or `access key id` and `secret access key` (and `session token`). Without credentials the SDK’s default chain is used. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the parcels aws account with the following properties: ``` *Since 0.1.0.* # DynamoDB > Seed DynamoDB tables and check the items your services write. Seed DynamoDB tables and check the items your services write. The steps use the scenario’s AWS account (`the {word} aws account with the following properties:`, from aws-core). A **seed** is a YAML or JSON file that maps table names to the items to put, written as plain JSON (numbers become `N`, objects `M`, arrays `L`): ```yaml insured-parcels: - reference: PX-CLM-1001 declaredValue: 120 carrier: KESTREL ``` **Checks** wait (10 seconds unless `within {duration}` says otherwise) until the table has an item, or a number of items, meeting every condition: `attribute | value` rows with a dotted path into maps (`address.city`), compared as text, `null` for null and `undefined` for absent. Items are read with a scan, which suits the small tables of a test environment. ## `aws-dynamodb.seed` [Section titled “aws-dynamodb.seed”](#aws-dynamodbseed) ```gherkin Given a {filepath} dynamodb seed ``` Put the items of a seed file (resolved against `resources`): YAML or JSON mapping table names to lists of items. **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Given a seeds/insured-parcels.yaml dynamodb seed ``` *Since 0.1.0.* ## `aws-dynamodb.item` [Section titled “aws-dynamodb.item”](#aws-dynamodbitem) ```gherkin Then [[within {duration} ]]the {word} dynamodb table has an item where: | ... | ... | ``` Wait (10s, or the given time) until the table has an item meeting every `attribute | value` row. **Variants** (optional parts in `[[...]]` above): * `the {word} dynamodb table has an item where:` * `within {duration} the {word} dynamodb table has an item where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the claims dynamodb table has an item where: ``` *Since 0.1.0.* ## `aws-dynamodb.items` [Section titled “aws-dynamodb.items”](#aws-dynamodbitems) ```gherkin Then [[within {duration} ]]the {word} dynamodb table has {int} item(s) where: | ... | ... | ``` Wait (10s, or the given time) until exactly that many items of the table meet every `attribute | value` row. **Variants** (optional parts in `[[...]]` above): * `the {word} dynamodb table has {int} item(s) where:` * `within {duration} the {word} dynamodb table has {int} item(s) where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the claims dynamodb table has 1 item where: ``` *Since 0.1.0.* # EventBridge > Put events on EventBridge buses and check the events your services put there. Put events on EventBridge buses and check the events your services put there. The steps use the scenario’s AWS account (`the {word} aws account with the following properties:`, from aws-core). An event has a detail type, a source and a JSON detail, as in `When a "DamageReported" event from carrier.kestrel is put on the carrier-events eventbridge bus:` with the detail as the doc string. **Checking a bus** does not take events from anyone: for the buses a run checks, Axx adds a rule of its own (`axx--`, matching every event of the account) with a queue of its own as its target once the apps are up, and removes both when the run ends. The conditions are paths into the event as EventBridge delivers it: `detail-type`, `source`, and `detail.` for the detail. A check only looks at the events received since its scenario started. ## `aws-eventbridge.put` [Section titled “aws-eventbridge.put”](#aws-eventbridgeput) ```gherkin When a(n) {string} event from {word} is put on the {word} eventbridge bus: """ ... """ ``` Put an event with the detail type and source on a bus; the doc string is its JSON detail. **Parameters:** `{string}` (text in single or double quotes; the quotes are removed), `{word}` (one word, no spaces) **Example:** ```gherkin When a "DamageReported" event from carrier.kestrel is put on the carrier-events eventbridge bus: ``` *Since 0.1.0.* ## `aws-eventbridge.received` [Section titled “aws-eventbridge.received”](#aws-eventbridgereceived) ```gherkin Then [[within {duration} ]]the {word} eventbridge bus has an event where: | ... | ... | ``` Wait (10s, or the given time) until the bus has an event, put since the scenario started, that meets every row: `path | value` on the event (`detail-type`, `source`, `detail.`), compared as text; `null` for null and `undefined` for absent. Axx adds a rule of its own to the bus for the run. **Variants** (optional parts in `[[...]]` above): * `the {word} eventbridge bus has an event where:` * `within {duration} the {word} eventbridge bus has an event where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the parcels eventbridge bus has an event where: ``` *Since 0.1.0.* # S3 > Upload files to S3 buckets and check the objects your services write there. Upload files to S3 buckets and check the objects your services write there. The steps use the scenario’s AWS account (`the {word} aws account with the following properties:`, from aws-core). Checks wait for the object (10 seconds unless `within {duration}` says otherwise), since services write asynchronously: an upload that triggers processing (an S3 event notification to a queue, say) and the object that processing writes. ## `aws-s3.upload` [Section titled “aws-s3.upload”](#aws-s3upload) ```gherkin When the {filepath} file is uploaded to the {word} s3 bucket[[ as {word}]] ``` Upload a file (resolved against `resources`) to a s3 bucket, named after the file or as given. The content type follows the file’s extension. **Variants** (optional parts in `[[...]]` above): * `the {filepath} file is uploaded to the {word} s3 bucket` * `the {filepath} file is uploaded to the {word} s3 bucket as {word}` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the invoices/kestrel-2026-09.csv file is uploaded to the carrier-drops s3 bucket When the invoices/kestrel-2026-09.csv file is uploaded to the carrier-drops s3 bucket as incoming/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `aws-s3.has` [Section titled “aws-s3.has”](#aws-s3has) ```gherkin Then [[within {duration} ]]the {word} s3 bucket has a(n) object named {word} ``` Wait (10s, or the given time) until the s3 bucket has an object with that name. **Variants** (optional parts in `[[...]]` above): * `the {word} s3 bucket has a(n) object named {word}` * `within {duration} the {word} s3 bucket has a(n) object named {word}` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the carrier-drops s3 bucket has a(n) object named disputes/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `aws-s3.identical` [Section titled “aws-s3.identical”](#aws-s3identical) ```gherkin Then [[within {duration} ]]the {word} object in the {word} s3 bucket is identical to the {filepath} file ``` Wait (10s, or the given time) until the object exists with exactly the content of the file (resolved against `resources`). **Variants** (optional parts in `[[...]]` above): * `the {word} object in the {word} s3 bucket is identical to the {filepath} file` * `within {duration} the {word} object in the {word} s3 bucket is identical to the {filepath} file` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Then the disputes/kestrel-2026-09.csv object in the carrier-drops s3 bucket is identical to the expected/kestrel-disputes.csv file ``` *Since 0.1.0.* ## `aws-s3.properties` [Section titled “aws-s3.properties”](#aws-s3properties) ```gherkin Then [[within {duration} ]]the {word} object in the {word} s3 bucket has the following properties: | ... | ... | ``` Wait (10s, or the given time) until the object exists and its JSON content has the properties: `path | value` rows compared as text, `null` for null and `undefined` for absent, as in the other JSON property steps. **Variants** (optional parts in `[[...]]` above): * `the {word} object in the {word} s3 bucket has the following properties:` * `within {duration} the {word} object in the {word} s3 bucket has the following properties:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then the summaries/kestrel-2026-09.json object in the carrier-drops s3 bucket has the following properties: ``` *Since 0.1.0.* # SNS > Publish messages to SNS topics and check the messages your services publish. Publish messages to SNS topics and check the messages your services publish. The steps use the scenario’s AWS account (`the {word} aws account with the following properties:`, from aws-core). **Checking a topic** does not take messages from anyone: for the topics a run checks, Axx subscribes a queue of its own (`axx--`, with raw message delivery) once the apps are up, and removes the subscription and the queue when the run ends. A check only looks at the messages received since its scenario started; match on data unique to the scenario, since scenarios run in parallel. ## `aws-sns.send` [Section titled “aws-sns.send”](#aws-snssend) ```gherkin When a message is published to the {word} sns topic: """ ... """ ``` Send a message whose body is the doc string to a sns topic. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin When a message is published to the claim-decisions sns topic: ``` *Since 0.1.0.* ## `aws-sns.send.file` [Section titled “aws-sns.send.file”](#aws-snssendfile) ```gherkin When the {filepath} message is published to the {word} sns topic[[ with the following attributes:]] ``` Send a message whose body is the file (resolved against `resources`) to a sns topic, with the attributes of the table (`name | value`). **Variants** (optional parts in `[[...]]` above): * `the {filepath} message is published to the {word} sns topic` * `the {filepath} message is published to the {word} sns topic with the following attributes:` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the messages/shipment-delivered.json message is published to the claim-decisions sns topic When the messages/shipment-delivered.json message is published to the claim-decisions sns topic with the following attributes: ``` *Since 0.1.0.* ## `aws-sns.received` [Section titled “aws-sns.received”](#aws-snsreceived) ```gherkin Then [[within {duration} ]]the {word} sns topic has a message where: | ... | ... | ``` Wait (10s, or the given time) until the sns topic has a message, received since the scenario started, that meets every row: `path | value` on the JSON body (a field name, a dotted path or a JSONPath, compared as text; `null` for null and `undefined` for absent), or `attribute | value` on a attribute sent with it. Axx subscribes a queue of its own to the topic for the run. **Variants** (optional parts in `[[...]]` above): * `the {word} sns topic has a message where:` * `within {duration} the {word} sns topic has a message where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the claim-decisions sns topic has a message where: ``` *Since 0.1.0.* # SQS > Send messages to SQS queues and check the messages your services send to them. Send messages to SQS queues and check the messages your services send to them. The steps use the scenario’s AWS account (`the {word} aws account with the following properties:`, from aws-core). **Checking a queue receives from it**: Axx takes each message off the queue, as any consumer would. Check the queues your services **write** to (an outbound queue another system reads); a queue your service consumes is checked by what the service does with the messages. Axx starts receiving from the queues a run checks once the apps are up, and a check only looks at the messages received since its scenario started. Match on data unique to the scenario: scenarios run in parallel and share the queue. ## `aws-sqs.send` [Section titled “aws-sqs.send”](#aws-sqssend) ```gherkin When a message is sent to the {word} sqs queue: """ ... """ ``` Send a message whose body is the doc string to a sqs queue. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin When a message is sent to the refund-requests sqs queue: ``` *Since 0.1.0.* ## `aws-sqs.send.file` [Section titled “aws-sqs.send.file”](#aws-sqssendfile) ```gherkin When the {filepath} message is sent to the {word} sqs queue[[ with the following attributes:]] ``` Send a message whose body is the file (resolved against `resources`) to a sqs queue, with the attributes of the table (`name | value`). **Variants** (optional parts in `[[...]]` above): * `the {filepath} message is sent to the {word} sqs queue` * `the {filepath} message is sent to the {word} sqs queue with the following attributes:` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the messages/shipment-delivered.json message is sent to the refund-requests sqs queue When the messages/shipment-delivered.json message is sent to the refund-requests sqs queue with the following attributes: ``` *Since 0.1.0.* ## `aws-sqs.received` [Section titled “aws-sqs.received”](#aws-sqsreceived) ```gherkin Then [[within {duration} ]]the {word} sqs queue has a message where: | ... | ... | ``` Wait (10s, or the given time) until the sqs queue has a message, received since the scenario started, that meets every row: `path | value` on the JSON body (a field name, a dotted path or a JSONPath, compared as text; `null` for null and `undefined` for absent), or `attribute | value` on a attribute sent with it. Axx takes the queue’s messages off it: check queues your services write to. **Variants** (optional parts in `[[...]]` above): * `the {word} sqs queue has a message where:` * `within {duration} the {word} sqs queue has a message where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the refund-requests sqs queue has a message where: ``` *Since 0.1.0.* # Blob Storage > Upload files to Blob Storage containers and check the blobs your services write there. Upload files to Blob Storage containers and check the blobs your services write there. Register the storage account with `the {word} azure storage account with the following properties:` (the first account registered is the default), set up as the Azure SDK is set up for the real service: | Property | | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `connection string` | the account’s connection string, e.g. `${env:AZURE_STORAGE_CONNECTION_STRING}`, or a local emulator’s | | `url` | the account’s blob endpoint (`https://.blob.core.windows.net`), signed in with the Azure default credential chain (environment, workload identity, managed identity, Azure CLI) | Checks wait for the blob (10 seconds unless `within {duration}` says otherwise), since services write asynchronously. Values expand `${env:..}` and `${sys:..}`. ## `azure-blob.account` [Section titled “azure-blob.account”](#azure-blobaccount) ```gherkin Given the {word} azure storage account with the following properties: | ... | ... | ``` Register the storage account the blob steps talk to: `connection string`, or `url` with the Azure default credential chain. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the customs azure storage account with the following properties: ``` *Since 0.1.0.* ## `azure-blob.upload` [Section titled “azure-blob.upload”](#azure-blobupload) ```gherkin When the {filepath} file is uploaded to the {word} blob container[[ as {word}]] ``` Upload a file (resolved against `resources`) to a blob container, named after the file or as given. The content type follows the file’s extension. **Variants** (optional parts in `[[...]]` above): * `the {filepath} file is uploaded to the {word} blob container` * `the {filepath} file is uploaded to the {word} blob container as {word}` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the invoices/kestrel-2026-09.csv file is uploaded to the declarations blob container When the invoices/kestrel-2026-09.csv file is uploaded to the declarations blob container as incoming/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `azure-blob.has` [Section titled “azure-blob.has”](#azure-blobhas) ```gherkin Then [[within {duration} ]]the {word} blob container has a(n) blob named {word} ``` Wait (10s, or the given time) until the blob container has an blob with that name. **Variants** (optional parts in `[[...]]` above): * `the {word} blob container has a(n) blob named {word}` * `within {duration} the {word} blob container has a(n) blob named {word}` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the declarations blob container has a(n) blob named disputes/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `azure-blob.identical` [Section titled “azure-blob.identical”](#azure-blobidentical) ```gherkin Then [[within {duration} ]]the {word} blob in the {word} blob container is identical to the {filepath} file ``` Wait (10s, or the given time) until the blob exists with exactly the content of the file (resolved against `resources`). **Variants** (optional parts in `[[...]]` above): * `the {word} blob in the {word} blob container is identical to the {filepath} file` * `within {duration} the {word} blob in the {word} blob container is identical to the {filepath} file` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Then the disputes/kestrel-2026-09.csv blob in the declarations blob container is identical to the expected/kestrel-disputes.csv file ``` *Since 0.1.0.* ## `azure-blob.properties` [Section titled “azure-blob.properties”](#azure-blobproperties) ```gherkin Then [[within {duration} ]]the {word} blob in the {word} blob container has the following properties: | ... | ... | ``` Wait (10s, or the given time) until the blob exists and its JSON content has the properties: `path | value` rows compared as text, `null` for null and `undefined` for absent, as in the other JSON property steps. **Variants** (optional parts in `[[...]]` above): * `the {word} blob in the {word} blob container has the following properties:` * `within {duration} the {word} blob in the {word} blob container has the following properties:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then the summaries/kestrel-2026-09.json blob in the declarations blob container has the following properties: ``` *Since 0.1.0.* # Service Bus > Send messages to Service Bus queues and topics, and check the messages your services send there. Send messages to Service Bus queues and topics, and check the messages your services send there. Register the namespace with `the {word} service bus namespace with the following properties:` (the first namespace registered is the default), set up as the Azure SDK is set up for the real service: | Property | | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `connection string` | the namespace’s connection string, e.g. `${env:SERVICEBUS_CONNECTION_STRING}`, or a local emulator’s (`UseDevelopmentEmulator=true`) | | `namespace` | the fully qualified namespace (`.servicebus.windows.net`), signed in with the Azure default credential chain | | `management endpoint` | where the namespace’s management API is when it is not the namespace’s own host, as with local emulators | Steps name a **queue** or a **topic**: `the customs-filings service bus queue`, `the customs-events service bus topic`. Messages carry application properties (`with the following properties:`, and `property ` rows in checks). **Checking a queue receives from it**: Axx completes each message it receives, as any consumer would, so check the queues your services **write** to. **Checking a topic** takes nothing from anyone: for the topics a run checks, Axx creates a subscription of its own (`axx-`, through the management API, so it needs the Manage right) once the apps are up, and deletes it when the run ends. A check only looks at the messages received since its scenario started. ## `azure-servicebus.namespace` [Section titled “azure-servicebus.namespace”](#azure-servicebusnamespace) ```gherkin Given the {word} service bus namespace with the following properties: | ... | ... | ``` Register the Service Bus namespace the steps talk to: `connection string`, or `namespace` with the Azure default credential chain; `management endpoint` for an emulator whose management API is elsewhere. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the customs service bus namespace with the following properties: ``` *Since 0.1.0.* ## `azure-servicebus.send` [Section titled “azure-servicebus.send”](#azure-servicebussend) ```gherkin When a message is sent to the {word} service bus queue:/topic: """ ... """ ``` Send a message whose body is the doc string to a service bus queue/topic. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin When a message is sent to the customs-filings service bus queue/topic: ``` *Since 0.1.0.* ## `azure-servicebus.send.file` [Section titled “azure-servicebus.send.file”](#azure-servicebussendfile) ```gherkin When the {filepath} message is sent to the {word} service bus queue/topic[[ with the following properties:]] ``` Send a message whose body is the file (resolved against `resources`) to a service bus queue/topic, with the properties of the table (`name | value`). **Variants** (optional parts in `[[...]]` above): * `the {filepath} message is sent to the {word} service bus queue/topic` * `the {filepath} message is sent to the {word} service bus queue/topic with the following properties:` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the messages/shipment-delivered.json message is sent to the customs-filings service bus queue/topic When the messages/shipment-delivered.json message is sent to the customs-filings service bus queue/topic with the following properties: ``` *Since 0.1.0.* ## `azure-servicebus.received` [Section titled “azure-servicebus.received”](#azure-servicebusreceived) ```gherkin Then [[within {duration} ]]the {word} service bus queue/topic has a message where: | ... | ... | ``` Wait (10s, or the given time) until the service bus queue/topic has a message, received since the scenario started, that meets every row: `path | value` on the JSON body (a field name, a dotted path or a JSONPath, compared as text; `null` for null and `undefined` for absent), or `property | value` on a property sent with it. Axx completes the messages of a queue it checks, and subscribes to a topic for the run. **Variants** (optional parts in `[[...]]` above): * `the {word} service bus queue/topic has a message where:` * `within {duration} the {word} service bus queue/topic has a message where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the customs-filings service bus queue/topic has a message where: ``` *Since 0.1.0.* # BigQuery > Seed BigQuery tables and check the rows your services write. Seed BigQuery tables and check the rows your services write. The steps use the scenario’s project (`the {word} gcp project with the following properties:`, from gcp-core). Tables are named `dataset.table` (in the project) or `project.dataset.table`. A **seed** is a YAML or JSON file that maps tables to the rows to insert (a streaming insert, `tabledata.insertAll`): ```yaml billing.carrier_rates: - carrier: KESTREL service: express price_per_kg: 1.35 ``` **Checks** wait (10 seconds unless `within {duration}` says otherwise) until the table has a row, or a number of rows, meeting every condition: `column | value` rows, with a dotted path into `RECORD` columns (`address.city`), compared as text: numbers as written, `NUMERIC` as its decimal, `TIMESTAMP` in RFC 3339 (`2026-09-24T09:30:00Z`), `DATE` as `2026-09-24`; `null` for NULL. A check reads the columns its conditions name, of up to 5,000 rows of the table: check the tables your scenarios write, not warehouse-size ones. ## `gcp-bigquery.seed` [Section titled “gcp-bigquery.seed”](#gcp-bigqueryseed) ```gherkin Given a {filepath} bigquery seed ``` Insert the rows of a seed file (resolved against `resources`): YAML or JSON mapping tables to lists of rows. **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Given a seeds/carrier-rates.yaml bigquery seed ``` *Since 0.1.0.* ## `gcp-bigquery.row` [Section titled “gcp-bigquery.row”](#gcp-bigqueryrow) ```gherkin Then [[within {duration} ]]the {word} bigquery table has a row where: | ... | ... | ``` Wait (10s, or the given time) until the table has a row meeting every `column | value` row. **Variants** (optional parts in `[[...]]` above): * `the {word} bigquery table has a row where:` * `within {duration} the {word} bigquery table has a row where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the billing.invoice_lines bigquery table has a row where: ``` *Since 0.1.0.* ## `gcp-bigquery.rows` [Section titled “gcp-bigquery.rows”](#gcp-bigqueryrows) ```gherkin Then [[within {duration} ]]the {word} bigquery table has {int} row(s) where: | ... | ... | ``` Wait (10s, or the given time) until exactly that many rows of the table meet every `column | value` row. **Variants** (optional parts in `[[...]]` above): * `the {word} bigquery table has {int} row(s) where:` * `within {duration} the {word} bigquery table has {int} row(s) where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the billing.invoice_lines bigquery table has 3 rows where: ``` *Since 0.1.0.* # Google Cloud project > The Google Cloud project the gcp-* packs talk to, set up the way the Google Cloud client libraries are set up for the real services. The Google Cloud project the gcp-\* packs talk to, set up the way the Google Cloud client libraries are set up for the real services. Register the project once with `the {word} gcp project with the following properties:`; every gcp-\* step of the scenario uses it (the first project registered is the default). | Property | | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `project` | required: the project ID | | `endpoint` | where every service of the project is, instead of Google Cloud: a local emulator such as `http://localhost:4588`. The clients then connect without credentials (and without TLS for an `http://` endpoint) | | `credentials` | a service account key file (resolved against `resources`) | Without `credentials`, the clients use Application Default Credentials, as they always do: `GOOGLE_APPLICATION_CREDENTIALS`, the gcloud login, or the workload’s service account. Values expand `${env:..}` and `${sys:..}`, so the same features run against Google Cloud and against an emulator. ## `gcp-core.project` [Section titled “gcp-core.project”](#gcp-coreproject) ```gherkin Given the {word} gcp project with the following properties: | ... | ... | ``` Register the Google Cloud project the gcp-\* steps talk to: `project` (required), `endpoint` (an emulator), `credentials` (a service account key file). Without credentials the clients use Application Default Credentials. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the billing gcp project with the following properties: ``` *Since 0.1.0.* # Firestore > Seed Firestore and check the documents your services write. Seed Firestore and check the documents your services write. The steps use the scenario’s project (`the {word} gcp project with the following properties:`, from gcp-core) and its default database. A **seed** is a YAML or JSON file that maps collections to their documents, by ID (a collection can be nested: `shipments/SHP-1/scans`): ```yaml shipments: SHP-1001: carrier: KESTREL weightKg: 2.5 agreedPrice: 3.38 ``` **Checks** wait (10 seconds unless `within {duration}` says otherwise): for a document at a path (`invoices/INV-2026-09-KESTREL`) to have properties, or for a collection to have a document meeting every condition. Conditions and properties are `field | value` rows, with a dotted path into maps (`totals.billed`), compared as text: timestamps in RFC 3339, `null` for null and `undefined` for absent. A collection check reads up to 5,000 of its documents. ## `gcp-firestore.seed` [Section titled “gcp-firestore.seed”](#gcp-firestoreseed) ```gherkin Given a {filepath} firestore seed ``` Write the documents of a seed file (resolved against `resources`): YAML or JSON mapping collections to documents by ID. **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Given a seeds/shipments.yaml firestore seed ``` *Since 0.1.0.* ## `gcp-firestore.document` [Section titled “gcp-firestore.document”](#gcp-firestoredocument) ```gherkin Then [[within {duration} ]]the {word} firestore document has the following properties: | ... | ... | ``` Wait (10s, or the given time) until the document at the path exists with every `field | value` property. **Variants** (optional parts in `[[...]]` above): * `the {word} firestore document has the following properties:` * `within {duration} the {word} firestore document has the following properties:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the invoices/INV-2026-09-KESTREL firestore document has the following properties: ``` *Since 0.1.0.* ## `gcp-firestore.collection` [Section titled “gcp-firestore.collection”](#gcp-firestorecollection) ```gherkin Then [[within {duration} ]]the {word} firestore collection has a document where: | ... | ... | ``` Wait (10s, or the given time) until the collection has a document meeting every `field | value` row. **Variants** (optional parts in `[[...]]` above): * `the {word} firestore collection has a document where:` * `within {duration} the {word} firestore collection has a document where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then the disputes firestore collection has a document where: ``` *Since 0.1.0.* # Pub/Sub > Publish messages to Pub/Sub topics and check the messages your services publish. Publish messages to Pub/Sub topics and check the messages your services publish. The steps use the scenario’s project (`the {word} gcp project with the following properties:`, from gcp-core). **Checking a topic** does not take messages from anyone: for the topics a run checks, Axx creates a subscription of its own (`axx--`) once the apps are up, and deletes it when the run ends. A check only looks at the messages received since its scenario started; match on data unique to the scenario, since scenarios run in parallel. The conditions are paths into the message data (JSON) and `attribute ` rows for its attributes. ## `gcp-pubsub.send` [Section titled “gcp-pubsub.send”](#gcp-pubsubsend) ```gherkin When a message is published to the {word} pubsub topic: """ ... """ ``` Send a message whose body is the doc string to a pubsub topic. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin When a message is published to the invoice-events pubsub topic: ``` *Since 0.1.0.* ## `gcp-pubsub.send.file` [Section titled “gcp-pubsub.send.file”](#gcp-pubsubsendfile) ```gherkin When the {filepath} message is published to the {word} pubsub topic[[ with the following attributes:]] ``` Send a message whose body is the file (resolved against `resources`) to a pubsub topic, with the attributes of the table (`name | value`). **Variants** (optional parts in `[[...]]` above): * `the {filepath} message is published to the {word} pubsub topic` * `the {filepath} message is published to the {word} pubsub topic with the following attributes:` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the messages/shipment-delivered.json message is published to the invoice-events pubsub topic When the messages/shipment-delivered.json message is published to the invoice-events pubsub topic with the following attributes: ``` *Since 0.1.0.* ## `gcp-pubsub.received` [Section titled “gcp-pubsub.received”](#gcp-pubsubreceived) ```gherkin Then [[within {duration} ]]the {word} pubsub topic has a message where: | ... | ... | ``` Wait (10s, or the given time) until the pubsub topic has a message, received since the scenario started, that meets every row: `path | value` on the JSON body (a field name, a dotted path or a JSONPath, compared as text; `null` for null and `undefined` for absent), or `attribute | value` on a attribute sent with it. Axx creates a subscription of its own to the topic for the run. **Variants** (optional parts in `[[...]]` above): * `the {word} pubsub topic has a message where:` * `within {duration} the {word} pubsub topic has a message where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the invoice-events pubsub topic has a message where: ``` *Since 0.1.0.* # Cloud Storage > Upload files to Cloud Storage buckets and check the objects your services write there. Upload files to Cloud Storage buckets and check the objects your services write there. The steps use the scenario’s project (`the {word} gcp project with the following properties:`, from gcp-core). Checks wait for the object (10 seconds unless `within {duration}` says otherwise), since services write asynchronously: an upload that triggers processing (a Pub/Sub notification or an Eventarc trigger, say) and the object that processing writes. ## `gcp-storage.upload` [Section titled “gcp-storage.upload”](#gcp-storageupload) ```gherkin When the {filepath} file is uploaded to the {word} gcs bucket[[ as {word}]] ``` Upload a file (resolved against `resources`) to a gcs bucket, named after the file or as given. The content type follows the file’s extension. **Variants** (optional parts in `[[...]]` above): * `the {filepath} file is uploaded to the {word} gcs bucket` * `the {filepath} file is uploaded to the {word} gcs bucket as {word}` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin When the invoices/kestrel-2026-09.csv file is uploaded to the carrier-invoices gcs bucket When the invoices/kestrel-2026-09.csv file is uploaded to the carrier-invoices gcs bucket as incoming/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `gcp-storage.has` [Section titled “gcp-storage.has”](#gcp-storagehas) ```gherkin Then [[within {duration} ]]the {word} gcs bucket has a(n) object named {word} ``` Wait (10s, or the given time) until the gcs bucket has an object with that name. **Variants** (optional parts in `[[...]]` above): * `the {word} gcs bucket has a(n) object named {word}` * `within {duration} the {word} gcs bucket has a(n) object named {word}` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then within 30s the carrier-invoices gcs bucket has a(n) object named disputes/kestrel-2026-09.csv ``` *Since 0.1.0.* ## `gcp-storage.identical` [Section titled “gcp-storage.identical”](#gcp-storageidentical) ```gherkin Then [[within {duration} ]]the {word} object in the {word} gcs bucket is identical to the {filepath} file ``` Wait (10s, or the given time) until the object exists with exactly the content of the file (resolved against `resources`). **Variants** (optional parts in `[[...]]` above): * `the {word} object in the {word} gcs bucket is identical to the {filepath} file` * `within {duration} the {word} object in the {word} gcs bucket is identical to the {filepath} file` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Then the disputes/kestrel-2026-09.csv object in the carrier-invoices gcs bucket is identical to the expected/kestrel-disputes.csv file ``` *Since 0.1.0.* ## `gcp-storage.properties` [Section titled “gcp-storage.properties”](#gcp-storageproperties) ```gherkin Then [[within {duration} ]]the {word} object in the {word} gcs bucket has the following properties: | ... | ... | ``` Wait (10s, or the given time) until the object exists and its JSON content has the properties: `path | value` rows compared as text, `null` for null and `undefined` for absent, as in the other JSON property steps. **Variants** (optional parts in `[[...]]` above): * `the {word} object in the {word} gcs bucket has the following properties:` * `within {duration} the {word} object in the {word} gcs bucket has the following properties:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then the summaries/kestrel-2026-09.json object in the carrier-invoices gcs bucket has the following properties: ``` *Since 0.1.0.* # Kafka > Publish events to Kafka topics (plain text or Confluent Avro through a Schema Registry) and assert on the records of a topic. Publish events to Kafka topics (plain text or Confluent Avro through a Schema Registry) and assert on the records of a topic. A **kafka service** names a cluster; a **topic client** gives one topic its producer and consumer configuration, written as Java client properties (`producer.*`, `consumer.*`); **kafka events** are drafts (key, headers, payload) you build and then publish; a **named kafka event** (`the parcel-events kafka event named registered ...`) is an expectation that one record of the topic must meet. Consumer assertions scan every record of the topic from its first offset, re-checking for up to 30 seconds as records arrive. Avro values are compared through Apache Avro’s text form of them (GenericData.toString), so JSONPath expectations such as `$.reference` work on Avro records as on JSON; union values appear without their wrapper. **Configuration** (`packs.kafka` in axx.yaml): `timeout` (default `30s`), `maxRecords` kept per topic (default 100000), `lenientUnions` (accept Avro union values without their `{"": value}` wrapper when exactly one branch fits). **Client properties.** Rows prefixed `producer.` or `consumer.` configure that client (without the prefix); values are expanded (`${env:..}`, `${sys:..}`). Defaults: `StringSerializer`/`StringDeserializer`, `auto.offset.reset=earliest`, no consumer group. How each Java property is applied: | Property | Client | In Axx | | --------------------------------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `acks` | producer | `all`/`-1` (default), `1` or `0` (`kgo.RequiredAcks`); `0` and `1` disable idempotence unless it is set explicitly, as in Java. | | `allow.auto.create.topics` | consumer | `true` (default) lets reading a missing topic create it (`kgo.AllowAutoTopicCreation`). Producers always may, as in Java. | | `auto.offset.reset` | consumer | `earliest` (default): assertions consider every record of the topic; `latest`: only records produced after the assertion starts. `none` is an error. | | `bootstrap.servers` | producer, consumer | Seed brokers (`kgo.SeedBrokers`); defaults to the service’s `brokers`. | | `buffer.memory` | producer | `kgo.MaxBufferedBytes`. | | `client.id` | producer, consumer | `kgo.ClientID`. | | `compression.type` | producer | `none` (default), `gzip`, `snappy`, `lz4` or `zstd` (`kgo.ProducerBatchCompression`). | | `connections.max.idle.ms` | producer, consumer | `kgo.ConnIdleTimeout`. | | `delivery.timeout.ms` | producer | `kgo.RecordDeliveryTimeout`. | | `enable.idempotence` | producer | `false` sets `kgo.DisableIdempotentWrite`. | | `fetch.max.bytes` | consumer | `kgo.FetchMaxBytes`. | | `fetch.max.wait.ms` | consumer | `kgo.FetchMaxWait`. | | `fetch.min.bytes` | consumer | `kgo.FetchMinBytes`. | | `isolation.level` | consumer | `read_uncommitted` (default) or `read_committed` (`kgo.FetchIsolationLevel`). | | `key.deserializer` | consumer | `StringDeserializer` (default), `ByteArrayDeserializer` or `KafkaAvroDeserializer`. | | `key.serializer` | producer | `StringSerializer` (default), `ByteArraySerializer` (the key text’s bytes) or `KafkaAvroSerializer` (the key as an Avro string). | | `linger.ms` | producer | `kgo.ProducerLinger` (default 5 ms, as in Java). | | `max.in.flight.requests.per.connection` | producer | `kgo.MaxProduceRequestsInflightPerBroker`. | | `max.partition.fetch.bytes` | consumer | `kgo.FetchMaxPartitionBytes`. | | `max.request.size` | producer | `kgo.ProducerBatchMaxBytes`. | | `metadata.max.age.ms` | producer, consumer | `kgo.MetadataMaxAge`. | | `partitioner.class` | producer | `DefaultPartitioner` (murmur2 of the key, as by default), `RoundRobinPartitioner` or `UniformStickyPartitioner`; other classes are errors. | | `request.timeout.ms` | producer, consumer | Producer: `kgo.ProduceRequestTimeout`; consumer: `kgo.RequestTimeoutOverhead`. | | `retries` | producer | `kgo.RecordRetries`. | | `retry.backoff.ms` | producer, consumer | Constant `kgo.RetryBackoffFn`. | | `sasl.jaas.config` | producer, consumer | The `username` and `password` of a `PlainLoginModule` or `ScramLoginModule` entry. | | `sasl.mechanism` | producer, consumer | `PLAIN`, `SCRAM-SHA-256` or `SCRAM-SHA-512` (GSSAPI and OAUTHBEARER are not supported). | | `security.protocol` | producer, consumer | `PLAINTEXT`, `SSL` (TLS dialer), `SASL_PLAINTEXT` or `SASL_SSL` (`kgo.SASL`). | | `socket.connection.setup.timeout.ms` | producer, consumer | `kgo.DialTimeout`. | | `ssl.enabled.protocols` | producer, consumer | Limits TLS versions to the listed `TLSv1.2`/`TLSv1.3`. | | `ssl.endpoint.identification.algorithm` | producer, consumer | `https` (default) verifies the broker host name; empty skips that check (the chain is still verified). | | `ssl.key.password` | producer, consumer | Private key password (JKS key entries, encrypted PEM keys); defaults to the keystore password. | | `ssl.keystore.certificate.chain` | producer, consumer | Inline PEM certificate chain. | | `ssl.keystore.key` | producer, consumer | Inline PEM private key (with `ssl.keystore.certificate.chain`). | | `ssl.keystore.location` | producer, consumer | Client certificate and key (JKS, PKCS12 or PEM file) for mutual TLS. | | `ssl.keystore.password` | producer, consumer | Keystore password. | | `ssl.keystore.type` | producer, consumer | `JKS` (default), `PKCS12` or `PEM`. | | `ssl.protocol` | producer, consumer | `TLSv1.2` or `TLSv1.3` sets the minimum TLS version (`TLS` allows both). | | `ssl.truststore.certificates` | producer, consumer | Inline PEM CA certificates. | | `ssl.truststore.location` | producer, consumer | CA certificates (JKS, PKCS12 or PEM file, resolved against `resources`). | | `ssl.truststore.password` | producer, consumer | Truststore password (optional for JKS, as in Java). | | `ssl.truststore.type` | producer, consumer | `JKS` (default), `PKCS12` or `PEM`; JKS and PKCS12 files are recognized by content. | | `transactional.id` | producer | Error: the steps never begin a transaction, so a transactional producer cannot send. | | `value.deserializer` | consumer | `StringDeserializer` (default), `ByteArrayDeserializer` or `KafkaAvroDeserializer` (payloads are checked against the record’s Java `toString()`). | | `value.serializer` | producer | `StringSerializer` (default), `ByteArraySerializer` or `KafkaAvroSerializer` (needed to publish with a schema). | | `auto.register.schemas` | producer | `true` (default) registers the schema under the subject; `false` looks its ID up and fails if it is not registered. | | `basic.auth.credentials.source` | producer, consumer | `URL` (default), `USER_INFO` or `SASL_INHERIT` (the SASL username and password). | | `basic.auth.user.info` | producer, consumer | `user:password` for `USER_INFO`. | | `bearer.auth.credentials.source` | producer, consumer | Only `STATIC_TOKEN` is supported. | | `bearer.auth.token` | producer, consumer | Static bearer token for the registry. | | `key.subject.name.strategy` | producer | `TopicNameStrategy` (default: `-key`), `RecordNameStrategy` or `TopicRecordNameStrategy`. | | `normalize.schemas` | producer | Passes `normalize=true` when registering or looking up. | | `schema.reflection` | producer, consumer | Only `false`: reflection needs Java classes. | | `schema.registry.basic.auth.user.info` | producer, consumer | Older name of `basic.auth.user.info`. | | `schema.registry.url` | producer, consumer | Schema Registry URLs (comma-separated); required by the Avro (de)serializers. `user:password@` in a URL is used for basic auth. | | `specific.avro.reader` | consumer | Only `false`: Axx has no generated classes and reads generic records. | | `use.latest.version` | producer | With `auto.register.schemas=false`, writes with the subject’s latest schema and ID. | | `use.schema.id` | producer | Writes with this schema ID (with `auto.register.schemas=false`). | | `value.subject.name.strategy` | producer | Same choices; the default subject is `-value`. | | `schema.registry.ssl.*` | producer, consumer | The `ssl.*` settings above, for HTTPS to the Schema Registry. | Accepted without effect: `group.id`, `group.instance.id`, `group.protocol`, `group.remote.assignor`, `enable.auto.commit`, `auto.commit.interval.ms`, `session.timeout.ms`, `heartbeat.interval.ms`, `max.poll.interval.ms`, `max.poll.records`, `partition.assignment.strategy`, `internal.leave.group.on.close`, `exclude.internal.topics`, `default.api.timeout.ms`, `client.rack`, `check.crcs`, `internal.throw.on.fetch.stable.offset.unsupported` (Axx reads each topic from the start without a consumer group and never commits offsets). `batch.size`, `max.block.ms`, `metadata.max.idle.ms`, `partitioner.ignore.keys`, `partitioner.adaptive.partitioning.enable`, `partitioner.availability.timeout.ms`, `transaction.timeout.ms`, `compression.gzip.level`, `compression.lz4.level`, `compression.zstd.level` (franz-go sizes batches by `max.request.size` and publishes each event synchronously). `client.dns.lookup`, `receive.buffer.bytes`, `send.buffer.bytes`, `reconnect.backoff.ms`, `reconnect.backoff.max.ms`, `retry.backoff.max.ms`, `socket.connection.setup.timeout.max.ms`, `metadata.recovery.strategy`, `metrics.num.samples`, `metrics.recording.level`, `metrics.sample.window.ms`, `auto.include.jmx.reporter`, `enable.metrics.push`, `ssl.provider`, `ssl.cipher.suites`, `ssl.keymanager.algorithm`, `ssl.trustmanager.algorithm`, `ssl.secure.random.implementation`, `sasl.kerberos.service.name`, `sasl.login.connect.timeout.ms`, `sasl.login.read.timeout.ms`, `sasl.login.retry.backoff.ms`, `sasl.login.retry.backoff.max.ms`, `sasl.login.refresh.window.factor`, `sasl.login.refresh.window.jitter`, `sasl.login.refresh.min.period.seconds`, `sasl.login.refresh.buffer.seconds`, `key.serializer.encoding`, `value.serializer.encoding`, `serializer.encoding`, `key.deserializer.encoding`, `value.deserializer.encoding`, `deserializer.encoding` (Java tuning without a franz-go counterpart (strings are always UTF-8)). `latest.compatibility.strict`, `id.compatibility.strict`, `avro.remove.java.properties`, `avro.use.logical.type.converters`, `avro.reflection.allow.null`, `max.schemas.per.subject`, `use.latest.with.metadata`, `auto.register.schemas.retry` (Axx writes and reads generic Avro records as described above). Rejected (they name Java classes): `context.name.strategy`, `interceptor.classes`, `sasl.client.callback.handler.class`, `sasl.login.callback.handler.class`, `sasl.login.class`, `security.providers`, `specific.avro.key.type`, `specific.avro.value.type`, `ssl.engine.factory.class`, `metric.reporters` other than JmxReporter, and any unknown `*.class`/`*.classes` property. Other unknown properties are logged as warnings and ignored. ## `kafka.service` [Section titled “kafka.service”](#kafkaservice) ```gherkin Given the {word} kafka service with the following properties: | ... | ... | ``` Register a Kafka cluster. The first one registered in a scenario is the default for steps without `on the {word} kafka service`. Properties: `brokers` (required; `host:port` list, `${env:..}`/`${sys:..}` expanded). **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the events kafka service with the following properties: ``` ## `kafka.client` [Section titled “kafka.client”](#kafkaclient) ```gherkin Given the {word} kafka topic client ``` Create a topic client on the default Kafka service with the default configuration: string keys and values, records read from the start of the topic. A topic can have one client per service in a scenario. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the parcel-events kafka topic client ``` ## `kafka.client.props` [Section titled “kafka.client.props”](#kafkaclientprops) ```gherkin Given a(n) {word} kafka topic client[[ on the {word} kafka service]] with the following properties: | ... | ... | ``` Create a topic client configured with Java Kafka client properties. Rows prefixed `producer.` configure publishing, rows prefixed `consumer.` configure assertions (the prefix is removed); values are expanded. For Avro use `producer.value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer`, `consumer.value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer` and `*.schema.registry.url`. See the pack documentation for every supported property; unknown properties are logged as warnings. **Variants** (optional parts in `[[...]]` above): * `a(n) {word} kafka topic client with the following properties:` * `a(n) {word} kafka topic client on the {word} kafka service with the following properties:` **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given a depot-scans kafka topic client with the following properties: Given a parcel-events kafka topic client on the events kafka service with the following properties: ``` ## `kafka.event` [Section titled “kafka.event”](#kafkaevent) ```gherkin Given a(n)[[ {ordinal} ordered]] {word} kafka event[[ on {word} kafka service]] ``` Draft a new event (key, headers and payload are set by the following steps; the payload starts as `{}`). Without an ordinal the event is appended. With one, it must be the next position (`a 2nd ordered` after one event); an ordinal equal to the number of existing events still appends, with a warning (it will be an error in Axx 1.0). **Variants** (optional parts in `[[...]]` above): * `a(n) {word} kafka event` * `a(n) {ordinal} ordered {word} kafka event` * `a(n) {word} kafka event on {word} kafka service` * `a(n) {ordinal} ordered {word} kafka event on {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given a depot-scans kafka event Given a 2nd ordered depot-scans kafka event Given a depot-scans kafka event on events kafka service ``` ## `kafka.event.key` [Section titled “kafka.event.key”](#kafkaeventkey) ```gherkin Given the[[ {ordinal} ordered]] {word} kafka event key is {word}[[ on the {word} kafka service]] ``` Set the key of a drafted event. With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event key is {word}` * `the {ordinal} ordered {word} kafka event key is {word}` * `the {word} kafka event key is {word} on the {word} kafka service` * `the {ordinal} ordered {word} kafka event key is {word} on the {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given the depot-scans kafka event key is PX-1001 Given the 2nd ordered depot-scans kafka event key is PX-1002 on the events kafka service ``` ## `kafka.event.headers` [Section titled “kafka.event.headers”](#kafkaeventheaders) ```gherkin Given the[[ {ordinal} ordered]] {word} kafka event headers[[ on the {word} kafka service]] are: | ... | ... | ``` Set headers of a drafted event (`name | value` rows; setting a header again replaces its value; an empty cell sends the text `null`). With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event headers are:` * `the {ordinal} ordered {word} kafka event headers are:` * `the {word} kafka event headers on the {word} kafka service are:` * `the {ordinal} ordered {word} kafka event headers on the {word} kafka service are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given the depot-scans kafka event headers are: Given the 1st ordered depot-scans kafka event headers on the events kafka service are: ``` ## `kafka.event.payload.resource` [Section titled “kafka.event.payload.resource”](#kafkaeventpayloadresource) ```gherkin Given the[[ {ordinal} ordered]] {word} kafka event payload is a(n) {filepath} resource[[ on the {word} kafka service]] ``` Set the payload of a drafted event to the contents of a file (resolved against `resources`). For Avro events the file is Avro’s JSON encoding of the record (unions as `{"": value}`). With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event payload is a(n) {filepath} resource` * `the {ordinal} ordered {word} kafka event payload is a(n) {filepath} resource` * `the {word} kafka event payload is a(n) {filepath} resource on the {word} kafka service` * `the {ordinal} ordered {word} kafka event payload is a(n) {filepath} resource on the {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Given the depot-scans kafka event payload is a kafka/scan-delivered.json resource Given the 3rd ordered depot-scans kafka event payload is a kafka/scan-out-for-delivery.json resource on the events kafka service ``` ## `kafka.event.properties.first` [Section titled “kafka.event.properties.first”](#kafkaeventpropertiesfirst) ```gherkin Given the[[ {ordinal} ordered]] kafka event payload properties[[ on the {word} kafka service]] are: | ... | ... | ``` Set JSONPath properties (`path | value` rows) of an event of the service’s **first** topic client (the first one created in the scenario). Values are always set as strings, an empty cell sets JSON null, and every property must already exist in the payload. With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the kafka event payload properties are:` * `the {ordinal} ordered kafka event payload properties are:` * `the kafka event payload properties on the {word} kafka service are:` * `the {ordinal} ordered kafka event payload properties on the {word} kafka service are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given the kafka event payload properties are: Given the 2nd ordered kafka event payload properties on the events kafka service are: ``` ## `kafka.event.properties` [Section titled “kafka.event.properties”](#kafkaeventproperties) ```gherkin Given the {word} kafka event payload properties[[ on the {word} kafka service]] are: | ... | ... | ``` Set JSONPath properties (`path | value` rows) of the topic’s first event. Values are always set as strings, an empty cell sets JSON null, and every property must already exist in the payload (set it in the payload file first). **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event payload properties are:` * `the {word} kafka event payload properties on the {word} kafka service are:` **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the depot-scans kafka event payload properties are: Given the depot-scans kafka event payload properties on the events kafka service are: ``` ## `kafka.event.properties.ordinal` [Section titled “kafka.event.properties.ordinal”](#kafkaeventpropertiesordinal) ```gherkin Given the {ordinal} ordered {word} kafka event payload properties[[ on the {word} kafka service]] are: | ... | ... | ``` Like the topic form, for the given event of the topic (`1st` is the first event created). **Variants** (optional parts in `[[...]]` above): * `the {ordinal} ordered {word} kafka event payload properties are:` * `the {ordinal} ordered {word} kafka event payload properties on the {word} kafka service are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given the 2nd ordered depot-scans kafka event payload properties are: ``` *Since 0.1.0.* ## `kafka.event.property.null` [Section titled “kafka.event.property.null”](#kafkaeventpropertynull) ```gherkin Given the[[ {ordinal} ordered]] {word} kafka event payload property {word} is null[[ on the {word} kafka service]] ``` Set a JSONPath property of a drafted event’s payload to JSON null; the property must exist. With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event payload property {word} is null` * `the {ordinal} ordered {word} kafka event payload property {word} is null` * `the {word} kafka event payload property {word} is null on the {word} kafka service` * `the {ordinal} ordered {word} kafka event payload property {word} is null on the {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin Given the depot-scans kafka event payload property $.location is null Given the 2nd ordered depot-scans kafka event payload property $.location is null on the events kafka service ``` ## `kafka.event.publish.schema` [Section titled “kafka.event.publish.schema”](#kafkaeventpublishschema) ```gherkin When the[[ {ordinal} ordered]] {word} kafka event is published using schema {filepath}[[ on the {word} kafka service]] ``` Publish a drafted event as Confluent Avro: the payload (Avro’s JSON encoding) is read with the `.avsc` schema file, the schema is registered (or looked up) in the Schema Registry under the subject of `value.subject.name.strategy` (`-value` by default), and the record is written as magic byte 0, the schema ID and the Avro binary. Needs `producer.value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer` and `producer.schema.registry.url`. A payload that does not fit the schema fails with the JSONPath of the mismatch. With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event is published using schema {filepath}` * `the {ordinal} ordered {word} kafka event is published using schema {filepath}` * `the {word} kafka event is published using schema {filepath} on the {word} kafka service` * `the {ordinal} ordered {word} kafka event is published using schema {filepath} on the {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin When the depot-scans kafka event is published using schema schemas/depot-scan.avsc When the 2nd ordered depot-scans kafka event is published using schema schemas/depot-scan.avsc on the events kafka service ``` ## `kafka.event.publish` [Section titled “kafka.event.publish”](#kafkaeventpublish) ```gherkin When the[[ {ordinal} ordered]] {word} kafka event is published[[ on the {word} kafka service]] ``` Publish a drafted event as it is: the payload text with the producer’s value serializer (`StringSerializer` by default, or `ByteArraySerializer`), with its key and headers. Use `published using schema` for Avro. With `the {ordinal} ordered` the step works on that event of the topic (`1st` is the first event created); without it, on the first event. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event is published` * `the {ordinal} ordered {word} kafka event is published` * `the {word} kafka event is published on the {word} kafka service` * `the {ordinal} ordered {word} kafka event is published on the {word} kafka service` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces) **Example:** ```gherkin When the depot-scans kafka event is published When the 2nd ordered depot-scans kafka event is published on the events kafka service ``` *Since 0.1.0.* ## `kafka.consumed.key` [Section titled “kafka.consumed.key”](#kafkaconsumedkey) ```gherkin Then the {word} kafka event named {word} key is {word}[[ on the {word} kafka service]] ``` Expect the label’s record to have this key (the consumer’s key deserializer decides how keys read). Adds the expectation to the label (`named {word}`) and then waits until **one record** of the topic satisfies **every** expectation added to that label in the scenario (key, payload properties and headers together). Records are read from the start of the topic (or, with `consumer.auto.offset.reset=latest`, only those produced after the step starts); the step fails after 30 seconds (`packs.kafka.timeout`) without a match, and the failure report lists the label’s expectations and the latest records with the reason each one did not match. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event named {word} key is {word}` * `the {word} kafka event named {word} key is {word} on the {word} kafka service` **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcel-events kafka event named registered key is PX-1001 Then the parcel-events kafka event named registered key is PX-1001 on the events kafka service ``` ## `kafka.consumed.properties` [Section titled “kafka.consumed.properties”](#kafkaconsumedproperties) ```gherkin Then the {word} kafka event named {word} payload properties[[ on the {word} kafka service]] are: | ... | ... | ``` Expect JSONPath properties of the label’s record payload (`path | value` rows). Values are typed: `"text"` is a string, `null` is JSON null, `12` an integer, `1.5` a decimal, `true`/`false` booleans, `{...}`/`[...]` JSON; anything else is a string. Numbers must match in type (`2` does not equal `2.0`). Adds the expectation to the label (`named {word}`) and then waits until **one record** of the topic satisfies **every** expectation added to that label in the scenario (key, payload properties and headers together). Records are read from the start of the topic (or, with `consumer.auto.offset.reset=latest`, only those produced after the step starts); the step fails after 30 seconds (`packs.kafka.timeout`) without a match, and the failure report lists the label’s expectations and the latest records with the reason each one did not match. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event named {word} payload properties are:` * `the {word} kafka event named {word} payload properties on the {word} kafka service are:` **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcel-events kafka event named registered payload properties are: Then the parcel-events kafka event named registered payload properties on the events kafka service are: ``` ## `kafka.consumed.headers` [Section titled “kafka.consumed.headers”](#kafkaconsumedheaders) ```gherkin Then the {word} kafka event named {word} headers[[ on the {word} kafka service]] are: | ... | ... | ``` Expect headers of the label’s record (`name | value` rows): each header must occur exactly once with exactly this value. Adds the expectation to the label (`named {word}`) and then waits until **one record** of the topic satisfies **every** expectation added to that label in the scenario (key, payload properties and headers together). Records are read from the start of the topic (or, with `consumer.auto.offset.reset=latest`, only those produced after the step starts); the step fails after 30 seconds (`packs.kafka.timeout`) without a match, and the failure report lists the label’s expectations and the latest records with the reason each one did not match. **Variants** (optional parts in `[[...]]` above): * `the {word} kafka event named {word} headers are:` * `the {word} kafka event named {word} headers on the {word} kafka service are:` **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcel-events kafka event named registered headers are: Then the parcel-events kafka event named registered headers on the events kafka service are: ``` ## `kafka.consumed.headers.match` [Section titled “kafka.consumed.headers.match”](#kafkaconsumedheadersmatch) ```gherkin Then the {word} kafka event named {word} headers match: | ... | ... | ``` Expect headers of the label’s record to match regular expressions (`name | pattern` rows, Java syntax, whole value). A header must have one distinct value; this step (unlike the others) ignores repeated identical values of a header. Adds the expectation to the label (`named {word}`) and then waits until **one record** of the topic satisfies **every** expectation added to that label in the scenario (key, payload properties and headers together). Records are read from the start of the topic (or, with `consumer.auto.offset.reset=latest`, only those produced after the step starts); the step fails after 30 seconds (`packs.kafka.timeout`) without a match, and the failure report lists the label’s expectations and the latest records with the reason each one did not match. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcel-events kafka event named registered headers match: ``` ## `kafka.consumed.headers.match.service` [Section titled “kafka.consumed.headers.match.service”](#kafkaconsumedheadersmatchservice) ```gherkin Then the {word} kafka event named {word} headers on the {word} kafka service match: | ... | ... | ``` The `headers match` expectation for a topic client of a named Kafka service. Adds the expectation to the label (`named {word}`) and then waits until **one record** of the topic satisfies **every** expectation added to that label in the scenario (key, payload properties and headers together). Records are read from the start of the topic (or, with `consumer.auto.offset.reset=latest`, only those produced after the step starts); the step fails after 30 seconds (`packs.kafka.timeout`) without a match, and the failure report lists the label’s expectations and the latest records with the reason each one did not match. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcel-events kafka event named registered headers on the events kafka service match: ``` *Since 0.1.0.* # Logs > Assert the entries your services log, read from files or sent to Axx over UDP, TCP or HTTP. Assert the entries your services log, read from files or sent to Axx over UDP, TCP or HTTP. A **log** is where a service’s log lines are: a file Axx reads, or an address Axx listens on while services send their lines to it. Register it with `the {word} log with the following properties:`, then assert the entries a scenario must produce. Only what the log received after the scenario registered it counts, and every step waits for its entries (10 seconds unless `within {duration}` says otherwise). Use logs to prove that something did **not** happen: have the service log its decision (“line ML-KES-0413-1 rejected: duplicate reference”) and assert that entry, rather than waiting and hoping nothing arrives. Match on data unique to the scenario (a reference, an id): scenarios run in parallel and share the log. **The url** says where the lines are: | url | Axx | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | | `file:///var/log/app.log`, `file://logs/app.log` | reads what is appended to the file (relative to axx.yaml); `file://.axx/logs/apps.log` is the console of the apps Axx starts | | `udp://0.0.0.0:5140` | listens; each datagram is one or more lines (e.g. Docker’s syslog log driver, an app’s syslog handler) | | `tcp://0.0.0.0:5150` | listens; newline-delimited or octet-counted (RFC 6587) messages | | `http://0.0.0.0:5160/logs`, `https://...` | listens; the body of each POST or PUT to that path (e.g. a Fluent Bit or Vector http output). https uses a self-signed certificate | Axx opens listeners before it starts the apps, for the log steps of the scenarios in the run, so services can send from the start; `axx up` keeps them open between runs. Services in containers reach them at `host.docker.internal`. **Patterns** are regular expressions (Java syntax), searched in the log’s text, not matched against whole lines: `^` and `$` match at line boundaries, every match counts, and a pattern can span lines (`\n`, or `(?s)` to let `.` match newlines), which covers multi-line entries such as stack traces. ## `logs.log` [Section titled “logs.log”](#logslog) ```gherkin Given the {word} log with the following properties: | ... | ... | ``` Register a log under a name. Properties: `url` (required; `file://`, `udp://`, `tcp://`, `http://` or `https://`, `${env:..}`/`${sys:..}` are expanded). Assertions only look at what the log receives from now on. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the parcels log with the following properties: ``` ## `logs.entry` [Section titled “logs.entry”](#logsentry) ```gherkin Then [[within {duration} ]]the {word} log has an entry matching {string} ``` Wait (10s, or the given time) until the log has a match for the regular expression. The pattern is searched in the log’s text: `^` and `$` match at line boundaries and a pattern can span lines. **Variants** (optional parts in `[[...]]` above): * `the {word} log has an entry matching {string}` * `within {duration} the {word} log has an entry matching {string}` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed) **Example:** ```gherkin Then the parcels log has an entry matching 'registration refused reference=PX-EVT-4003' Then within 30s the parcels log has an entry matching 'manifest line processed line=ML-MAP-0019-3 status=IMPORTED' ``` ## `logs.entries` [Section titled “logs.entries”](#logsentries) ```gherkin Then [[within {duration} ]]the {word} log has entries matching: | ... | ... | ``` Wait until the log has a match for every regular expression in the table (one per row). Each row needs a match of its own: the same pattern in two rows needs two matches. **Variants** (optional parts in `[[...]]` above): * `the {word} log has entries matching:` * `within {duration} the {word} log has entries matching:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces) **Example:** ```gherkin Then the parcels log has entries matching: ``` ## `logs.count` [Section titled “logs.count”](#logscount) ```gherkin Then [[within {duration} ]]the {word} log has {int} entry/entries matching {string} ``` Wait until the log has the given number of matches for the regular expression, for example one per retry. More matches than that fail the step. **Variants** (optional parts in `[[...]]` above): * `the {word} log has {int} entry/entries matching {string}` * `within {duration} the {word} log has {int} entry/entries matching {string}` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{word}` (one word, no spaces), `{int}` (a 32-bit integer), `{string}` (text in single or double quotes; the quotes are removed) **Example:** ```gherkin Then the parcels log has 2 entries matching 'storing parcel PX-DBF-3002 failed, retrying' ``` ## `logs.across` [Section titled “logs.across”](#logsacross) ```gherkin Then [[within {duration} ]]the logs have entries matching: | ... | ... | ``` Wait until every log in the table has a match for its regular expression (`log | pattern` rows), for example the service’s own entry and its dependency’s. Each row needs a match of its own. **Variants** (optional parts in `[[...]]` above): * `the logs have entries matching:` * `within {duration} the logs have entries matching:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`) **Example:** ```gherkin Then the logs have entries matching: ``` # Mocks > Verify requests received by WireMock mocks (stubs are defined in WireMock mapping files). Verify requests received by WireMock mocks (stubs are defined in WireMock mapping files). ## `mock.service` [Section titled “mock.service”](#mockservice) ```gherkin Given the mocked {word} service with the following properties: | ... | ... | ``` Register a WireMock server. The first mocked service registered in a scenario is the default one. Properties: `url` (required; `${env:..}`/`${sys:..}` are expanded). **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the mocked addresses service with the following properties: ``` ## `mock.received` [Section titled “mock.received”](#mockreceived) ```gherkin Then the mocked {word} request to {word} named {word} was received by {mockedService} ``` Register a request pattern under a name (method + exact URL, including the query string) and verify WireMock received it at least once. Later steps refer to the pattern by name. **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the mocked GET request to /v1/postcodes/DE/10115 named postcode-check was received by addresses ``` ## `mock.openapi.levels` [Section titled “mock.openapi.levels”](#mockopenapilevels) ```gherkin Given the OpenAPI validation levels for the mocked {mockedService} service are: | ... | ... | ``` Relax, for this scenario, the mocked service’s OpenAPI contract: findings that would fail the checking mock step are reported at the level you set instead (`key | level` rows; WARN logs them, INFO and IGNORE drop them). A key also covers the keys below it: `validation.response.body` covers `validation.response.body.schema.required`. It applies to the calls this scenario’s mock steps check; a stub that is off-contract on purpose is better relaxed in its own metadata (`openApiValidationLevels`), which applies wherever it answers. This is the dependency’s contract: your own service’s is relaxed with `the OpenAPI validation levels are:`. **Parameters:** `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Given the OpenAPI validation levels for the mocked addresses service are: ``` ## `mock.count.exactly` [Section titled “mock.count.exactly”](#mockcountexactly) ```gherkin Then the mocked request named {word}[[ on {mockedService}]] was received exactly {int} time(s) ``` Verify the named request pattern was received exactly the given number of times. **Variants** (optional parts in `[[...]]` above): * `the mocked request named {word} was received exactly {int} time(s)` * `the mocked request named {word} on {mockedService} was received exactly {int} time(s)` **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the mocked request named postcode-check was received exactly 1 time ``` ## `mock.count.atLeast` [Section titled “mock.count.atLeast”](#mockcountatleast) ```gherkin Then the mocked request named {word}[[ on {mockedService}]] was received at least {int} time(s) ``` Verify the named request pattern was received at least the given number of times. **Variants** (optional parts in `[[...]]` above): * `the mocked request named {word} was received at least {int} time(s)` * `the mocked request named {word} on {mockedService} was received at least {int} time(s)` **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the mocked request named postcode-check was received at least 1 time ``` ## `mock.count.atMost` [Section titled “mock.count.atMost”](#mockcountatmost) ```gherkin Then the mocked request named {word}[[ on {mockedService}]] was received at most {int} time(s) ``` Verify the named request pattern was received at most the given number of times. **Variants** (optional parts in `[[...]]` above): * `the mocked request named {word} was received at most {int} time(s)` * `the mocked request named {word} on {mockedService} was received at most {int} time(s)` **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the mocked request named postcode-check was received at most 1 time ``` ## `mock.notReceived` [Section titled “mock.notReceived”](#mocknotreceived) ```gherkin Then the mocked {word} request to {word} named {word}[[ on {mockedService}]] was not received ``` Register a request pattern under a name and verify WireMock received no matching request. **Variants** (optional parts in `[[...]]` above): * `the mocked {word} request to {word} named {word} was not received` * `the mocked {word} request to {word} named {word} on {mockedService} was not received` **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the mocked GET request to /v1/postcodes/DE/12489 named skipped-check was not received ``` ## `mock.header.is` [Section titled “mock.header.is”](#mockheaderis) ```gherkin Then the header {word} for mocked request named {word}[[ on {mockedService}]] is {string} ``` Verify the named request was received with a header equal to the value. The constraint is added to the named pattern. **Variants** (optional parts in `[[...]]` above): * `the header {word} for mocked request named {word} is {string}` * `the header {word} for mocked request named {word} on {mockedService} is {string}` **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario), `{string}` (text in single or double quotes; the quotes are removed) **Example:** ```gherkin Then the header X-Api-Key for mocked request named postcode-check is 'example-address-key' ``` ## `mock.headers.are` [Section titled “mock.headers.are”](#mockheadersare) ```gherkin Then the headers for mocked request named {word} on {mockedService} are: | ... | ... | ``` Verify the named request was received with every header in the table (name | value). **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the headers for mocked request named postcode-check on addresses are: ``` ## `mock.header.matches` [Section titled “mock.header.matches”](#mockheadermatches) ```gherkin Then the header {word} for mocked request named {word} on {mockedService} matches {pattern} ``` Verify the named request was received with a header matching the regular expression (evaluated by WireMock, full match). **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario), `{pattern}` (A regular expression (Java syntax) without whitespace. It must match the whole value) **Example:** ```gherkin Then the header Accept for mocked request named postcode-check on addresses matches ^application/json$ ``` ## `mock.headers.match` [Section titled “mock.headers.match”](#mockheadersmatch) ```gherkin Then the headers for mocked request named {word} on {mockedService} match: | ... | ... | ``` Verify the named request was received with headers matching each regular expression in the table (name | pattern). **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the headers for mocked request named postcode-check on addresses match: ``` ## `mock.header.missing` [Section titled “mock.header.missing”](#mockheadermissing) ```gherkin Then the header {word} for mocked request named {word} on {mockedService} is missing ``` Verify the named request was received without the header. **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the header Authorization for mocked request named postcode-check on addresses is missing ``` ## `mock.headers.missing` [Section titled “mock.headers.missing”](#mockheadersmissing) ```gherkin Then the headers for mocked request named {word} on {mockedService} are missing: | ... | ... | ``` Verify the named request was received without any of the headers listed (one per row). **Parameters:** `{word}` (one word, no spaces), `{mockedService}` (The name of a mocked service registered in the scenario) **Example:** ```gherkin Then the headers for mocked request named postcode-check on addresses are missing: ``` # MongoDB > Register MongoDB databases, seed collections from JSON files, and query and assert on documents. Register MongoDB databases, seed collections from JSON files, and query and assert on documents. ## `mongo.service` [Section titled “mongo.service”](#mongoservice) ```gherkin Given a(n) {word} mongo database with the following properties: | ... | ... | ``` Register a MongoDB database. The first one registered in a scenario is the default. Properties (all required, `${env:..}`/`${sys:..}` expanded): `url` (must include the database name; `authSource` defaults to it), `user`, `password`. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given a tracking-db mongo database with the following properties: ``` ## `mongo.seed` [Section titled “mongo.seed”](#mongoseed) ```gherkin Given a {filepath} mongo db seed ``` Insert documents into the default MongoDB database. The file is a JSON object mapping collection names to arrays of documents (Extended JSON such as `{"$oid": ...}` is supported). **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file) **Example:** ```gherkin Given a seeds/scans-in-transit.json mongo db seed ``` ## `mongo.seed.named` [Section titled “mongo.seed.named”](#mongoseednamed) ```gherkin Given a {filepath} MongoDB seed for {word} ``` Insert documents into the named MongoDB database (same file format as the default-database seed). **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin Given a seeds/scans-in-transit.json MongoDB seed for tracking-db ``` ## `mongo.seed.named.alt` [Section titled “mongo.seed.named.alt”](#mongoseednamedalt) ```gherkin Given a {filepath} mongo db seed for {word} ``` Insert documents into the named MongoDB database (same file format as the default-database seed). **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{word}` (one word, no spaces) **Example:** ```gherkin Given a seeds/scans-in-transit.json mongo db seed for tracking-db ``` *Since 0.1.0.* ## `mongo.find` [Section titled “mongo.find”](#mongofind) ```gherkin Then a[[ {ordinal}]] selection of documents is retrieved from the {word} collection[[ on {mongoService}]] where: | ... | ... | ``` Find documents and keep them as the next selection for later assertions, like the SQL selection steps. Selections are numbered in the order they are retrieved; `the selection` means the first. Each row is a `field | value` condition (dotted field paths reach into nested documents). Values are read as JSON when they parse as JSON (`3`, `true`, `null`, `"3"`, `{"$oid": "..."}`) and as plain strings otherwise. **Variants** (optional parts in `[[...]]` above): * `a selection of documents is retrieved from the {word} collection where:` * `a {ordinal} selection of documents is retrieved from the {word} collection where:` * `a selection of documents is retrieved from the {word} collection on {mongoService} where:` * `a {ordinal} selection of documents is retrieved from the {word} collection on {mongoService} where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{mongoService}` (The name of a MongoDB database registered in the scenario) **Example:** ```gherkin Then a selection of documents is retrieved from the scans collection where: ``` *Since 0.1.0.* ## `mongo.find.poll` [Section titled “mongo.find.poll”](#mongofindpoll) ```gherkin Then within {duration} a[[ {ordinal}]] selection of at least {int} document(s) is retrieved from the {word} collection[[ on {mongoService}]] where: | ... | ... | ``` Poll every 500ms until the find returns at least the given number of documents or the time is up. On timeout the last result (possibly empty) is kept, so assert on it with a document-count step. Each row is a `field | value` condition (dotted field paths reach into nested documents). Values are read as JSON when they parse as JSON (`3`, `true`, `null`, `"3"`, `{"$oid": "..."}`) and as plain strings otherwise. **Variants** (optional parts in `[[...]]` above): * `within {duration} a selection of at least {int} document(s) is retrieved from the {word} collection where:` * `within {duration} a {ordinal} selection of at least {int} document(s) is retrieved from the {word} collection where:` * `within {duration} a selection of at least {int} document(s) is retrieved from the {word} collection on {mongoService} where:` * `within {duration} a {ordinal} selection of at least {int} document(s) is retrieved from the {word} collection on {mongoService} where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{int}` (a 32-bit integer), `{word}` (one word, no spaces), `{mongoService}` (The name of a MongoDB database registered in the scenario) **Example:** ```gherkin Then within 10s a selection of at least 1 document is retrieved from the tracking collection where: ``` *Since 0.1.0.* ## `mongo.docs.eq` [Section titled “mongo.docs.eq”](#mongodocseq) ```gherkin Then the[[ {ordinal}]] selection[[ on {mongoService}]] has {int} document(s) ``` Assert that a selection of documents has exactly the given number of documents. **Variants** (optional parts in `[[...]]` above): * `the selection has {int} document(s)` * `the {ordinal} selection has {int} document(s)` * `the selection on {mongoService} has {int} document(s)` * `the {ordinal} selection on {mongoService} has {int} document(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{mongoService}` (The name of a MongoDB database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has 2 documents ``` *Since 0.1.0.* ## `mongo.docs.gt` [Section titled “mongo.docs.gt”](#mongodocsgt) ```gherkin Then the[[ {ordinal}]] selection[[ on {mongoService}]] has more than {int} document(s) ``` Assert that a selection of documents has more than the given number of documents. **Variants** (optional parts in `[[...]]` above): * `the selection has more than {int} document(s)` * `the {ordinal} selection has more than {int} document(s)` * `the selection on {mongoService} has more than {int} document(s)` * `the {ordinal} selection on {mongoService} has more than {int} document(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{mongoService}` (The name of a MongoDB database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has more than 2 documents ``` *Since 0.1.0.* ## `mongo.docs.lt` [Section titled “mongo.docs.lt”](#mongodocslt) ```gherkin Then the[[ {ordinal}]] selection[[ on {mongoService}]] has fewer than {int} document(s) ``` Assert that a selection of documents has fewer than the given number of documents. **Variants** (optional parts in `[[...]]` above): * `the selection has fewer than {int} document(s)` * `the {ordinal} selection has fewer than {int} document(s)` * `the selection on {mongoService} has fewer than {int} document(s)` * `the {ordinal} selection on {mongoService} has fewer than {int} document(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{mongoService}` (The name of a MongoDB database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has fewer than 2 documents ``` *Since 0.1.0.* ## `mongo.doc.are` [Section titled “mongo.doc.are”](#mongodocare) ```gherkin Then the {ordinal} document for the[[ {ordinal}]] selection[[ on {mongoService}]] properties are: | ... | ... | ``` Assert properties (JSONPath, e.g. `lastLocation` or `scans[0].status`) of one document of a selection. Documents are compared as JSON: ObjectIds become their hex string, dates become ISO-8601 UTC strings, and every scalar is compared as text. `null` means null and `undefined` means the field is absent. **Variants** (optional parts in `[[...]]` above): * `the {ordinal} document for the selection properties are:` * `the {ordinal} document for the {ordinal} selection properties are:` * `the {ordinal} document for the selection on {mongoService} properties are:` * `the {ordinal} document for the {ordinal} selection on {mongoService} properties are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{mongoService}` (The name of a MongoDB database registered in the scenario) **Example:** ```gherkin Then the 1st document for the selection properties are: ``` *Since 0.1.0.* ## `mongo.doc.match` [Section titled “mongo.doc.match”](#mongodocmatch) ```gherkin Then the {ordinal} document for the[[ {ordinal}]] selection[[ on {mongoService}]] properties match: | ... | ... | ``` Like the properties step, but every value is a regular expression (Java syntax) that must match the whole text. **Variants** (optional parts in `[[...]]` above): * `the {ordinal} document for the selection properties match:` * `the {ordinal} document for the {ordinal} selection properties match:` * `the {ordinal} document for the selection on {mongoService} properties match:` * `the {ordinal} document for the {ordinal} selection on {mongoService} properties match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{mongoService}` (The name of a MongoDB database registered in the scenario) **Example:** ```gherkin Then the 1st document for the 2nd selection on tracking-db properties match: ``` *Since 0.1.0.* # REST > Send HTTP requests to REST services, validate them against the services' OpenAPI specifications, and assert on the responses. Send HTTP requests to REST services, validate them against the services’ OpenAPI specifications, and assert on the responses. **Services.** Register services with `the service with the following properties:` (`url`, optional `openapi`). The first service registered in a scenario is the default one; the other steps name a service with `on `. **Requests.** Each service keeps its requests in the order they are added: `a GET request to /path` adds the first (default) request and `a 2nd ordered POST request to /path` the second. Header, payload, execution and response steps address a request with `for 2nd ordered request` / `for 2nd ordered response` (default: the first). A request is executed once; its response stays available to every later step. **Payloads.** A payload starts from an OpenAPI content example (the named one, or the first in document order; `externalValue` examples are read relative to the specification) or from an empty template `{}`, and is edited with the payload property steps (JSONPath, typed values, `null`/`undefined`). `application/x-www-form-urlencoded` payloads are sent form-encoded. **Execution.** Requests go through one HTTP client shared by the run (connections are reused and closed at the end), honor the step timeout, follow redirects for GET and HEAD, and do not verify TLS certificates unless `packs.rest.tls.verify: true` is set in axx.yaml. Without a Content-Type header a payload is sent with its payload step’s media type. **OpenAPI validation.** When a service has an `openapi` specification (OpenAPI 3.0 or 3.1, a URL or a file; parsed once per run), each executed request and its response are validated after sending. Every finding has a key in the style of the swagger request validator (the one the WireMock extension uses), and a level: * `ERROR` (alias `FAIL`, the default for every key) fails the execute step, listing every error with its key; * `WARN` and `INFO` are logged on the step; * `IGNORE` drops the finding. Levels come from `openapi.levels` in axx.yaml and are overridden per scenario with `the OpenAPI validation levels are:`. A key also sets every more specific key (`validation.request.body` covers `validation.request.body.schema.required`); the most specific configured key wins. | Key | Reported when | | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `validation.request.path.missing` | no path of the specification matches the request path | | `validation.request.operation.notAllowed` | the path exists but not for the request method | | `validation.request.body.missing` | the operation requires a body and the request has none | | `validation.request.body.unexpected` | the request has a body the operation does not declare | | `validation.request.contentType.notAllowed` | the request Content-Type is not declared for the operation | | `validation.request.body.schema.{keyword}` | the request body violates a schema keyword (type, required, enum, format, pattern, minimum, maxLength, additionalProperties, oneOf, …) | | `validation.request.body.schema.invalidJson` | the request body cannot be parsed | | `validation.request.body.schema.processingError` | the request body schema cannot be compiled or the body cannot be read | | `validation.request.parameter.missing` | a required path parameter is missing | | `validation.request.parameter.query.missing` | a required query parameter is missing | | `validation.request.parameter.header.missing` | a required header parameter is missing | | `validation.request.parameter.cookie.missing` | a required cookie parameter is missing | | `validation.request.parameter.schema.{keyword}` | a parameter value violates its schema (type, enum, format, pattern, minimum, …) | | `validation.request.parameter.schema.invalidJson` | a JSON (content) parameter cannot be parsed | | `validation.request.parameter.collection.invalidFormat` | an array or object parameter is serialized in the wrong style | | `validation.request.parameter.collection.tooManyItems` | an array parameter has more than maxItems items | | `validation.request.parameter.collection.tooFewItems` | an array parameter has fewer than minItems items | | `validation.request.parameter.collection.duplicateItems` | an array parameter with uniqueItems repeats an item | | `validation.request.parameter.{in}.invalid` | any other parameter problem; {in} is `path`, `query`, `header` or `cookie` | | `validation.request.security.missing` | the credentials a security requirement needs are absent | | `validation.request.security.invalid` | credentials are present but do not match the security scheme | | `validation.response.status.unknown` | the response status is not documented for the operation (and there is no default) | | `validation.response.contentType.notAllowed` | the response Content-Type is not declared for the status | | `validation.response.body.missing` | the response declares a body schema but has no body | | `validation.response.body.unexpected` | a response to HEAD has a body | | `validation.response.body.schema.{keyword}` | the response body violates a schema keyword | | `validation.response.body.schema.invalidJson` | the response body cannot be parsed | | `validation.response.body.schema.processingError` | the response body schema cannot be compiled or the body cannot be read | | `validation.response.header.missing` | a required response header is missing | | `validation.response.header.schema.{keyword}` | a response header value violates its schema | | `validation.request.unknownError` | anything else the validator reports about the request | | `validation.response.unknownError` | anything else the validator reports about the response | Schema keywords use the draft-4 names: `const` is reported as `enum`, `exclusiveMinimum`/`exclusiveMaximum` as `minimum`/`maximum`, `unevaluatedProperties` as `additionalProperties`. When a scenario fails, its failure context (`rest`) shows the last request and response (headers, bodies truncated to 2 KB) and the OpenAPI findings. ## `rest.service` [Section titled “rest.service”](#restservice) ```gherkin Given the {word} service with the following properties: | ... | ... | ``` Register a REST service. The first service registered in a scenario is the default one. Properties (`${env:..}`/`${sys:..}` are expanded): * `url` (required): the base URL requests are sent to, e.g. `http://localhost:8080`. * `openapi`: the service’s OpenAPI 3.0 or 3.1 specification, as a URL or a file path (resolved against the `resources` roots). When set, every executed request and its response are validated against it, and content example payloads come from it. **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given the parcels service with the following properties: ``` ## `rest.openapi.levels` [Section titled “rest.openapi.levels”](#restopenapilevels) ```gherkin Given the OpenAPI validation levels[[ on {service}]] are: | ... | ... | ``` Override OpenAPI validation levels for this scenario (on the default or the named service). Each row is `validation key | level`; the level is `ERROR` (or its alias `FAIL`), `WARN`, `INFO` or `IGNORE`. A key covers every more specific key: `validation.request.body` relaxes `validation.request.body.schema.required` too, and the most specific configured key wins. Rows are merged over `openapi.levels` from axx.yaml. See the pack documentation for the keys. **Variants** (optional parts in `[[...]]` above): * `the OpenAPI validation levels are:` * `the OpenAPI validation levels on {service} are:` **Parameters:** `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the OpenAPI validation levels are: ``` ## `rest.request` [Section titled “rest.request”](#restrequest) ```gherkin Given a(n) {word} request to {word}[[ on {service}]] ``` Add a request with a method and a path (optionally with a query string, e.g. `/api/parcels?sender=kestrel-books`) to the default or the named service. This is the service’s first (default) request; add more with the ordered form. The path is appended to the service URL; an absolute URL replaces it. **Variants** (optional parts in `[[...]]` above): * `a(n) {word} request to {word}` * `a(n) {word} request to {word} on {service}` **Parameters:** `{word}` (one word, no spaces), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given a GET request to /api/parcels/PX-1001 Given a DELETE request to /api/parcels/PX-1001 on parcels ``` ## `rest.request.ordered` [Section titled “rest.request.ordered”](#restrequestordered) ```gherkin Given a {ordinal} ordered {word} request to {word}[[ on {service}]] ``` Add the Nth request of a service. Requests are numbered in the order they are added: the 1st ordered request is the default request, and the Nth can only be added once N-1 exist. **Variants** (optional parts in `[[...]]` above): * `a {ordinal} ordered {word} request to {word}` * `a {ordinal} ordered {word} request to {word} on {service}` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given a 2nd ordered GET request to /api/parcels/PX-1001 Given a 1st ordered POST request to /api/parcels on parcels ``` ## `rest.request.header` [Section titled “rest.request.header”](#restrequestheader) ```gherkin Given the request header {word} is {string}[[ for {ordinal} ordered request]] ``` Set a request header. `Content-Type` and `Accept` replace an earlier value; other headers may be added more than once and are all sent. Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the request header {word} is {string}` * `the request header {word} is {string} for {ordinal} ordered request` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given the request header Content-Type is 'application/json' ``` ## `rest.request.header.on` [Section titled “rest.request.header.on”](#restrequestheaderon) ```gherkin Given the request header {word} is {string} for[[ {ordinal} ordered]] request on {service} ``` `rest.request.header` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.header`. **Variants** (optional parts in `[[...]]` above): * `the request header {word} is {string} for request on {service}` * `the request header {word} is {string} for {ordinal} ordered request on {service}` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the request header Accept is 'application/json' for 1st ordered request on parcels ``` ## `rest.request.headers` [Section titled “rest.request.headers”](#restrequestheaders) ```gherkin Given the request headers[[ for {ordinal} ordered request]] are: | ... | ... | ``` Set request headers from a `name | value` table (a name may repeat). Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the request headers are:` * `the request headers for {ordinal} ordered request are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given the request headers are: ``` ## `rest.request.headers.on` [Section titled “rest.request.headers.on”](#restrequestheaderson) ```gherkin Given the request headers for[[ {ordinal} ordered]] request on {service} are: | ... | ... | ``` `rest.request.headers` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.headers`. **Variants** (optional parts in `[[...]]` above): * `the request headers for request on {service} are:` * `the request headers for {ordinal} ordered request on {service} are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the request headers for request on parcels are: ``` ## `rest.request.payload.empty` [Section titled “rest.request.payload.empty”](#restrequestpayloadempty) ```gherkin Given a request payload using a(n) {mimeType} empty content template[[ for {ordinal} ordered request]] ``` Start the request payload from an empty JSON object `{}`, to be filled with the payload property steps; no OpenAPI specification is needed. With `application/x-www-form-urlencoded` the properties are sent form-encoded (nested objects and arrays as JSON text). Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `a request payload using a(n) {mimeType} empty content template` * `a request payload using a(n) {mimeType} empty content template for {ordinal} ordered request` **Parameters:** `{mimeType}` (One of `application/json`, `text/json`, `application/problem+json`, `application/x-www-form-urlencoded`), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given a request payload using an application/json empty content template ``` ## `rest.request.payload.empty.on` [Section titled “rest.request.payload.empty.on”](#restrequestpayloademptyon) ```gherkin Given a request payload using a(n) {mimeType} empty content template for[[ {ordinal} ordered]] request on {service} ``` `rest.request.payload.empty` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.payload.empty`. **Variants** (optional parts in `[[...]]` above): * `a request payload using a(n) {mimeType} empty content template for request on {service}` * `a request payload using a(n) {mimeType} empty content template for {ordinal} ordered request on {service}` **Parameters:** `{mimeType}` (One of `application/json`, `text/json`, `application/problem+json`, `application/x-www-form-urlencoded`), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given a request payload using an application/json empty content template for request on parcels ``` ## `rest.request.payload.example` [Section titled “rest.request.payload.example”](#restrequestpayloadexample) ```gherkin Given a request payload using a(n) {mimeType} content example[[ named {string}]][[ for {ordinal} ordered request]] ``` Use a request body example of the service’s OpenAPI specification as the payload: with `named ''` the example of that name, otherwise the first example in document order (or the media type’s single `example`). The example is looked up under the operation that matches the request’s method and path, for the given media type. An example with an `externalValue` is read relative to the specification. Requires the service’s `openapi` property and a request added first. Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `a request payload using a(n) {mimeType} content example` * `a request payload using a(n) {mimeType} content example named {string}` * `a request payload using a(n) {mimeType} content example for {ordinal} ordered request` * `a request payload using a(n) {mimeType} content example named {string} for {ordinal} ordered request` **Parameters:** `{mimeType}` (One of `application/json`, `text/json`, `application/problem+json`, `application/x-www-form-urlencoded`), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given a request payload using an application/json content example named 'Standard parcel' ``` ## `rest.request.payload.example.on` [Section titled “rest.request.payload.example.on”](#restrequestpayloadexampleon) ```gherkin Given a request payload using a(n) {mimeType} content example[[ named {string}]] for[[ {ordinal} ordered]] request on {service} ``` `rest.request.payload.example` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.payload.example`. **Variants** (optional parts in `[[...]]` above): * `a request payload using a(n) {mimeType} content example for request on {service}` * `a request payload using a(n) {mimeType} content example named {string} for request on {service}` * `a request payload using a(n) {mimeType} content example for {ordinal} ordered request on {service}` * `a request payload using a(n) {mimeType} content example named {string} for {ordinal} ordered request on {service}` **Parameters:** `{mimeType}` (One of `application/json`, `text/json`, `application/problem+json`, `application/x-www-form-urlencoded`), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given a request payload using an application/json content example for 1st ordered request on parcels ``` ## `rest.request.property` [Section titled “rest.request.property”](#restrequestproperty) ```gherkin Given the request payload property {word} is {string}[[ for {ordinal} ordered request]] ``` Set a payload property (a JSONPath such as `weightGrams`, `recipient.postcode` or `$.recipient.name`). A value in double quotes inside the quotes (`'"42"'`) is always a string. Otherwise the value takes the type of the current value (string, boolean, integer, number, object or array, parsed from JSON text); a property that does not exist yet, or is null, gets the type the text reads as (`true`, `42`, `1.5`, `{...}`, `[...]`, else a string). Requires a payload step first. Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the request payload property {word} is {string}` * `the request payload property {word} is {string} for {ordinal} ordered request` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given the request payload property sender is 'kestrel-books' ``` ## `rest.request.property.on` [Section titled “rest.request.property.on”](#restrequestpropertyon) ```gherkin Given the request payload property {word} is {string} for[[ {ordinal} ordered]] request on {service} ``` `rest.request.property` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.property`. **Variants** (optional parts in `[[...]]` above): * `the request payload property {word} is {string} for request on {service}` * `the request payload property {word} is {string} for {ordinal} ordered request on {service}` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the request payload property serviceLevel is 'EXPRESS' for request on parcels ``` ## `rest.request.properties` [Section titled “rest.request.properties”](#restrequestproperties) ```gherkin Given the request payload properties[[ for {ordinal} ordered request]] are: | ... | ... | ``` Set payload properties from a `path | value` table, row by row, like the single-property step. `null` sets JSON null and `undefined` removes the property (any case); write `"null"` or `"undefined"` in double quotes for the strings. Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the request payload properties are:` * `the request payload properties for {ordinal} ordered request are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given the request payload properties are: ``` ## `rest.request.properties.on` [Section titled “rest.request.properties.on”](#restrequestpropertieson) ```gherkin Given the request payload properties for[[ {ordinal} ordered]] request on {service} are: | ... | ... | ``` `rest.request.properties` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.properties`. **Variants** (optional parts in `[[...]]` above): * `the request payload properties for request on {service} are:` * `the request payload properties for {ordinal} ordered request on {service} are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the request payload properties for 1st ordered request on parcels are: ``` ## `rest.request.property.null` [Section titled “rest.request.property.null”](#restrequestpropertynull) ```gherkin Given the request payload property {word} is null[[ for {ordinal} ordered request]] ``` Set an existing payload property to JSON null. Without an ordinal the step applies to the first (default) request of the service; `for 2nd ordered request` picks the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the request payload property {word} is null` * `the request payload property {word} is null for {ordinal} ordered request` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Given the request payload property recipient.street is null ``` ## `rest.request.property.null.on` [Section titled “rest.request.property.null.on”](#restrequestpropertynullon) ```gherkin Given the request payload property {word} is null for[[ {ordinal} ordered]] request on {service} ``` `rest.request.property.null` on a named service: `for request on ` addresses the service’s first (default) request, `for 2nd ordered request on ` its second one. Everything else works like `rest.request.property.null`. **Variants** (optional parts in `[[...]]` above): * `the request payload property {word} is null for request on {service}` * `the request payload property {word} is null for {ordinal} ordered request on {service}` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Given the request payload property recipient.street is null for request on parcels ``` ## `rest.execute` [Section titled “rest.execute”](#restexecute) ```gherkin When the[[ {ordinal} ordered]] request is executed[[ on {service}]] ``` Send a request and keep its response for the response steps. With an OpenAPI specification the request and the response are validated after sending: findings at level ERROR fail the step (all of them are listed with their keys), WARN and INFO are logged. The payload is sent as is, form-encoded for `application/x-www-form-urlencoded`; without a Content-Type header the payload’s media type is used. The request honors the step timeout. A request can be executed once. **Variants** (optional parts in `[[...]]` above): * `the request is executed` * `the request is executed on {service}` * `the {ordinal} ordered request is executed` * `the {ordinal} ordered request is executed on {service}` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin When the request is executed When the 2nd ordered request is executed on parcels ``` ## `rest.response.status` [Section titled “rest.response.status”](#restresponsestatus) ```gherkin Then the[[ {ordinal} ordered]] response status code is {int}[[ on {service}]] ``` Assert the HTTP status code of a response. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response status code is {int}` * `the response status code is {int} on {service}` * `the {ordinal} ordered response status code is {int}` * `the {ordinal} ordered response status code is {int} on {service}` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{int}` (a 32-bit integer), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response status code is 200 Then the 2nd ordered response status code is 201 on parcels ``` ## `rest.response.body.contains` [Section titled “rest.response.body.contains”](#restresponsebodycontains) ```gherkin Then the response body contains {string}[[ for {ordinal} ordered response]] ``` Assert that the response body contains the text. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response body contains {string}` * `the response body contains {string} for {ordinal} ordered response` **Parameters:** `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response body contains 'already registered' ``` ## `rest.response.body.contains.on` [Section titled “rest.response.body.contains.on”](#restresponsebodycontainson) ```gherkin Then the response body contains {string} for[[ {ordinal} ordered]] response on {service} ``` `rest.response.body.contains` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.body.contains`. **Variants** (optional parts in `[[...]]` above): * `the response body contains {string} for response on {service}` * `the response body contains {string} for {ordinal} ordered response on {service}` **Parameters:** `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response body contains 'already registered' for 2nd ordered response on parcels ``` ## `rest.response.header.is` [Section titled “rest.response.header.is”](#restresponseheaderis) ```gherkin Then the response header {word} is {string}[[ for {ordinal} ordered response]] ``` Assert that a response header (name matched case-insensitively) has the value; with repeated headers, one of them must. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response header {word} is {string}` * `the response header {word} is {string} for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response header Content-Type is 'application/json' ``` ## `rest.response.header.is.on` [Section titled “rest.response.header.is.on”](#restresponseheaderison) ```gherkin Then the response header {word} is {string} for[[ {ordinal} ordered]] response on {service} ``` `rest.response.header.is` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.header.is`. **Variants** (optional parts in `[[...]]` above): * `the response header {word} is {string} for response on {service}` * `the response header {word} is {string} for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response header Content-Type is 'application/json' for response on parcels ``` ## `rest.response.header.matches` [Section titled “rest.response.header.matches”](#restresponseheadermatches) ```gherkin Then the response header {word} matches {pattern}[[ for {ordinal} ordered response]] ``` Assert that a response header matches a regular expression (Java syntax; it must match the whole value). With repeated headers, one of them must match. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response header {word} matches {pattern}` * `the response header {word} matches {pattern} for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{pattern}` (A regular expression (Java syntax) without whitespace. It must match the whole value), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response header Content-Type matches ^application/json.*$ ``` ## `rest.response.header.matches.on` [Section titled “rest.response.header.matches.on”](#restresponseheadermatcheson) ```gherkin Then the response header {word} matches {pattern} for[[ {ordinal} ordered]] response on {service} ``` `rest.response.header.matches` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.header.matches`. **Variants** (optional parts in `[[...]]` above): * `the response header {word} matches {pattern} for response on {service}` * `the response header {word} matches {pattern} for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{pattern}` (A regular expression (Java syntax) without whitespace. It must match the whole value), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response header Content-Type matches ^application/json$ for 1st ordered response on parcels ``` ## `rest.response.header.missing` [Section titled “rest.response.header.missing”](#restresponseheadermissing) ```gherkin Then the response header {word} is missing[[ for {ordinal} ordered response]] ``` Assert that the response has no header with the name. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response header {word} is missing` * `the response header {word} is missing for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response header Content-Length is missing ``` ## `rest.response.header.missing.on` [Section titled “rest.response.header.missing.on”](#restresponseheadermissingon) ```gherkin Then the response header {word} is missing for[[ {ordinal} ordered]] response on {service} ``` `rest.response.header.missing` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.header.missing`. **Variants** (optional parts in `[[...]]` above): * `the response header {word} is missing for response on {service}` * `the response header {word} is missing for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response header X-Custom is missing for response on parcels ``` ## `rest.response.headers.are` [Section titled “rest.response.headers.are”](#restresponseheadersare) ```gherkin Then the response headers[[ for {ordinal} ordered response]] are: | ... | ... | ``` Assert response headers from a `name | value` table, each like the single-header step (a name may repeat). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response headers are:` * `the response headers for {ordinal} ordered response are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response headers are: ``` ## `rest.response.headers.are.on` [Section titled “rest.response.headers.are.on”](#restresponseheadersareon) ```gherkin Then the response headers for[[ {ordinal} ordered]] response on {service} are: | ... | ... | ``` `rest.response.headers.are` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.headers.are`. **Variants** (optional parts in `[[...]]` above): * `the response headers for response on {service} are:` * `the response headers for {ordinal} ordered response on {service} are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response headers for 1st ordered response on parcels are: ``` ## `rest.response.headers.match` [Section titled “rest.response.headers.match”](#restresponseheadersmatch) ```gherkin Then the response headers[[ for {ordinal} ordered response]] match: | ... | ... | ``` Assert response headers from a `name | regular expression` table (full match, Java syntax). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response headers match:` * `the response headers for {ordinal} ordered response match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response headers match: ``` ## `rest.response.headers.match.on` [Section titled “rest.response.headers.match.on”](#restresponseheadersmatchon) ```gherkin Then the response headers for[[ {ordinal} ordered]] response on {service} match: | ... | ... | ``` `rest.response.headers.match` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.headers.match`. **Variants** (optional parts in `[[...]]` above): * `the response headers for response on {service} match:` * `the response headers for {ordinal} ordered response on {service} match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response headers for response on parcels match: ``` ## `rest.response.headers.missing` [Section titled “rest.response.headers.missing”](#restresponseheadersmissing) ```gherkin Then the response headers[[ for {ordinal} ordered response]] are missing: | ... | ... | ``` Assert that the response has none of the headers named in the table’s first column. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response headers are missing:` * `the response headers for {ordinal} ordered response are missing:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response headers are missing: ``` ## `rest.response.headers.missing.on` [Section titled “rest.response.headers.missing.on”](#restresponseheadersmissingon) ```gherkin Then the response headers for[[ {ordinal} ordered]] response on {service} are missing: | ... | ... | ``` `rest.response.headers.missing` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.headers.missing`. **Variants** (optional parts in `[[...]]` above): * `the response headers for response on {service} are missing:` * `the response headers for {ordinal} ordered response on {service} are missing:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response headers for response on parcels are missing: ``` ## `rest.response.property.is` [Section titled “rest.response.property.is”](#restresponsepropertyis) ```gherkin Then the response payload property {word} is {string}[[ for {ordinal} ordered response]] ``` Assert a property of a JSON response (a JSONPath such as `status`, `recipient.postcode` or `[?(@.sender=='kestrel-books')].reference`; an indefinite path yields a list). The response must be JSON (`application/json`, `text/json` or any `+json` type, charset ignored). Values are compared with their JSON type: `'John'` or `"42"` (double quotes inside) are strings, `42` an integer, `42L` a long, `1.5` a number, `true`/`false` booleans, `{...}` and `[...]` JSON objects and arrays (compared regardless of member order). An integer never equals a decimal (`5` is not `5.0`). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is {string}` * `the response payload property {word} is {string} for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload property status is 'REGISTERED' ``` ## `rest.response.property.is.on` [Section titled “rest.response.property.is.on”](#restresponsepropertyison) ```gherkin Then the response payload property {word} is {string} for[[ {ordinal} ordered]] response on {service} ``` `rest.response.property.is` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.property.is`. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is {string} for response on {service}` * `the response payload property {word} is {string} for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{string}` (text in single or double quotes; the quotes are removed), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload property status is 'REGISTERED' for 1st ordered response on parcels ``` ## `rest.response.property.null` [Section titled “rest.response.property.null”](#restresponsepropertynull) ```gherkin Then the response payload property {word} is null[[ for {ordinal} ordered response]] ``` Assert that a response payload property exists and is JSON null. Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is null` * `the response payload property {word} is null for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload property lastLocation is null ``` ## `rest.response.property.null.on` [Section titled “rest.response.property.null.on”](#restresponsepropertynullon) ```gherkin Then the response payload property {word} is null for[[ {ordinal} ordered]] response on {service} ``` `rest.response.property.null` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.property.null`. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is null for response on {service}` * `the response payload property {word} is null for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload property lastLocation is null for response on parcels ``` ## `rest.response.property.undefined` [Section titled “rest.response.property.undefined”](#restresponsepropertyundefined) ```gherkin Then the response payload property {word} is undefined[[ for {ordinal} ordered response]] ``` Assert that a response payload property does not exist. (An indefinite path always exists: it reads as a possibly empty list.) Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is undefined` * `the response payload property {word} is undefined for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload property nonexistent is undefined ``` ## `rest.response.property.undefined.on` [Section titled “rest.response.property.undefined.on”](#restresponsepropertyundefinedon) ```gherkin Then the response payload property {word} is undefined for[[ {ordinal} ordered]] response on {service} ``` `rest.response.property.undefined` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.property.undefined`. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} is undefined for response on {service}` * `the response payload property {word} is undefined for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload property nonexistent is undefined for 1st ordered response on parcels ``` ## `rest.response.property.matches` [Section titled “rest.response.property.matches”](#restresponsepropertymatches) ```gherkin Then the response payload property {word} matches {pattern}[[ for {ordinal} ordered response]] ``` Assert that a response payload property is a string that matches a regular expression (Java syntax; it must match the whole value). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} matches {pattern}` * `the response payload property {word} matches {pattern} for {ordinal} ordered response` **Parameters:** `{word}` (one word, no spaces), `{pattern}` (A regular expression (Java syntax) without whitespace. It must match the whole value), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload property barcode matches ^PX[0-9]{11}$ ``` ## `rest.response.property.matches.on` [Section titled “rest.response.property.matches.on”](#restresponsepropertymatcheson) ```gherkin Then the response payload property {word} matches {pattern} for[[ {ordinal} ordered]] response on {service} ``` `rest.response.property.matches` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.property.matches`. **Variants** (optional parts in `[[...]]` above): * `the response payload property {word} matches {pattern} for response on {service}` * `the response payload property {word} matches {pattern} for {ordinal} ordered response on {service}` **Parameters:** `{word}` (one word, no spaces), `{pattern}` (A regular expression (Java syntax) without whitespace. It must match the whole value), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload property barcode matches ^PX[0-9]{11}$ for response on parcels ``` ## `rest.response.properties.are` [Section titled “rest.response.properties.are”](#restresponsepropertiesare) ```gherkin Then the response payload properties[[ for {ordinal} ordered response]] are: | ... | ... | ``` Assert response payload properties from a `path | value` table. `null` and `undefined` (any case) check for JSON null and absence; `"null"` in double quotes is the string. Every other value is compared like the single-property step. All rows are checked and every mismatch is reported. Values are compared with their JSON type: `'John'` or `"42"` (double quotes inside) are strings, `42` an integer, `42L` a long, `1.5` a number, `true`/`false` booleans, `{...}` and `[...]` JSON objects and arrays (compared regardless of member order). An integer never equals a decimal (`5` is not `5.0`). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload properties are:` * `the response payload properties for {ordinal} ordered response are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload properties are: ``` ## `rest.response.properties.are.on` [Section titled “rest.response.properties.are.on”](#restresponsepropertiesareon) ```gherkin Then the response payload properties for[[ {ordinal} ordered]] response on {service} are: | ... | ... | ``` `rest.response.properties.are` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.properties.are`. **Variants** (optional parts in `[[...]]` above): * `the response payload properties for response on {service} are:` * `the response payload properties for {ordinal} ordered response on {service} are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload properties for 1st ordered response on parcels are: ``` ## `rest.response.properties.match` [Section titled “rest.response.properties.match”](#restresponsepropertiesmatch) ```gherkin Then the response payload properties[[ for {ordinal} ordered response]] match: | ... | ... | ``` Assert response payload properties from a `path | regular expression` table (full match, Java syntax). Without an ordinal the step checks the response of the first (default) request; `for 2nd ordered response` the response of the second one. Without `on {service}` it uses the default (first registered) service. **Variants** (optional parts in `[[...]]` above): * `the response payload properties match:` * `the response payload properties for {ordinal} ordered response match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first) **Example:** ```gherkin Then the response payload properties match: ``` ## `rest.response.properties.match.on` [Section titled “rest.response.properties.match.on”](#restresponsepropertiesmatchon) ```gherkin Then the response payload properties for[[ {ordinal} ordered]] response on {service} match: | ... | ... | ``` `rest.response.properties.match` on a named service: `for response on ` addresses the service’s first (default) response, `for 2nd ordered response on ` its second one. Everything else works like `rest.response.properties.match`. **Variants** (optional parts in `[[...]]` above): * `the response payload properties for response on {service} match:` * `the response payload properties for {ordinal} ordered response on {service} match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{service}` (The name of a REST service registered in the scenario) **Example:** ```gherkin Then the response payload properties for response on parcels match: ``` # SQL > Seed, query and assert on relational databases. PostgreSQL is fully supported (including JSONB and trigger fault injection); MySQL/MariaDB, SQL Server and SQLite support seeds, selections, row counts and locks. JDBC URLs (jdbc:postgresql://...) are accepted as-is. Seed, query and assert on relational databases. PostgreSQL is fully supported (including JSONB and trigger fault injection); MySQL/MariaDB, SQL Server and SQLite support seeds, selections, row counts and locks. JDBC URLs (jdbc:postgresql://…) are accepted as-is. ## `sql.service` [Section titled “sql.service”](#sqlservice) ```gherkin Given a(n) {word} database with the following properties: | ... | ... | ``` Register a database. The first one registered in a scenario is the default. Properties: `url` (JDBC or native URL), `user`, `password` (all required; `${env:..}`/`${sys:..}` expanded), `schema` (optional). **Parameters:** `{word}` (one word, no spaces) **Example:** ```gherkin Given a parcels-db database with the following properties: ``` ## `sql.seed` [Section titled “sql.seed”](#sqlseed) ```gherkin Given a {filepath} db seed[[ on {dbService}]] ``` Insert the rows of a dataset file (resolved against `resources`). Formats by extension: `.yaml`/`.yml` (`schema.table:` → list of rows), flat XML (``), `.json`, `.csv` (a directory of `.csv` files with `table-ordering.txt`) and `.xlsx` (one sheet per table). Replacers: `[null]`, `[DAY,NOW]`, `[DAY,PLUS,1]`, `[UNIX_TIMESTAMP]`. Rows are inserted in one transaction and never deleted. **Variants** (optional parts in `[[...]]` above): * `a {filepath} db seed` * `a {filepath} db seed on {dbService}` **Parameters:** `{filepath}` (A file of the project, without whitespace: a path relative to the `resources` directories or to the directory of axx.yaml, or an absolute path. Editors link it to the file), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Given a seeds/manifest-kestrel.yaml db seed Given a seeds/dispatching.yaml db seed on parcels-db ``` ## `sql.lock` [Section titled “sql.lock”](#sqllock) ```gherkin Given the rows in the {word} table[[ on {dbService}]] are locked where: | ... | ... | ``` Lock matching rows with SELECT … FOR UPDATE on a separate connection, held until the locks are released or the scenario ends. Values in the table become `column = 'value'` conditions joined with AND; `null` becomes `IS NULL`. Values are escaped. **Variants** (optional parts in `[[...]]` above): * `the rows in the {word} table are locked where:` * `the rows in the {word} table on {dbService} are locked where:` **Parameters:** `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Given the rows in the parcels.parcels table are locked where: ``` ## `sql.unlock` [Section titled “sql.unlock”](#sqlunlock) ```gherkin Then the row locks[[ on {dbService}]] are released ``` Release row locks taken with the lock step. **Variants** (optional parts in `[[...]]` above): * `the row locks are released` * `the row locks on {dbService} are released` **Parameters:** `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then the row locks are released ``` ## `sql.select` [Section titled “sql.select”](#sqlselect) ```gherkin Then a[[ {ordinal}]] selection of rows is retrieved from the {word} table[[ on {dbService}]] where: | ... | ... | ``` Query rows (SELECT \* … WHERE) and keep the result as the next selection for later assertions. Selections are numbered in the order they are retrieved; `the selection` means the first. Values in the table become `column = 'value'` conditions joined with AND; `null` becomes `IS NULL`. Values are escaped. **Variants** (optional parts in `[[...]]` above): * `a selection of rows is retrieved from the {word} table where:` * `a {ordinal} selection of rows is retrieved from the {word} table where:` * `a selection of rows is retrieved from the {word} table on {dbService} where:` * `a {ordinal} selection of rows is retrieved from the {word} table on {dbService} where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then a selection of rows is retrieved from the parcels.parcels table where: ``` ## `sql.select.poll` [Section titled “sql.select.poll”](#sqlselectpoll) ```gherkin Then within {duration} a[[ {ordinal}]] selection of at least {int} row(s) is retrieved from the {word} table[[ on {dbService}]] where: | ... | ... | ``` Poll every 500ms until the query returns at least the given number of rows or the time is up. On timeout the last result (possibly empty) is kept, so assert on it with a row-count step. **Variants** (optional parts in `[[...]]` above): * `within {duration} a selection of at least {int} row(s) is retrieved from the {word} table where:` * `within {duration} a {ordinal} selection of at least {int} row(s) is retrieved from the {word} table where:` * `within {duration} a selection of at least {int} row(s) is retrieved from the {word} table on {dbService} where:` * `within {duration} a {ordinal} selection of at least {int} row(s) is retrieved from the {word} table on {dbService} where:` **Parameters:** `{duration}` (A duration in seconds or minutes, e.g. `5s` or `2m`), `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{int}` (a 32-bit integer), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then within 10s a selection of at least 1 row is retrieved from the parcels.manifest_lines table where: ``` ## `sql.select.jsonb` [Section titled “sql.select.jsonb”](#sqlselectjsonb) ```gherkin Then a[[ {ordinal}]] selection of rows is retrieved from the {word} table[[ on {dbService}]] where the {word} jsonb column contains: | ... | ... | ``` PostgreSQL: select rows whose JSONB column contains the given properties (`@>`). Dotted keys (`a.b`) build nested objects; every value is compared as a JSON string (`null` means JSON null). **Variants** (optional parts in `[[...]]` above): * `a selection of rows is retrieved from the {word} table where the {word} jsonb column contains:` * `a {ordinal} selection of rows is retrieved from the {word} table where the {word} jsonb column contains:` * `a selection of rows is retrieved from the {word} table on {dbService} where the {word} jsonb column contains:` * `a {ordinal} selection of rows is retrieved from the {word} table on {dbService} where the {word} jsonb column contains:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then a selection of rows is retrieved from the parcels.parcels table where the details jsonb column contains: ``` ## `sql.json.are` [Section titled “sql.json.are”](#sqljsonare) ```gherkin Then the {ordinal} row {word} property for the[[ {ordinal}]] selection[[ on {dbService}]] json properties are: | ... | ... | ``` Assert JSON properties (JSONPath) of a JSON column in the given row of a selection. Every scalar is compared as text; `null` means JSON null and `undefined` means the property is absent. **Variants** (optional parts in `[[...]]` above): * `the {ordinal} row {word} property for the selection json properties are:` * `the {ordinal} row {word} property for the {ordinal} selection json properties are:` * `the {ordinal} row {word} property for the selection on {dbService} json properties are:` * `the {ordinal} row {word} property for the {ordinal} selection on {dbService} json properties are:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then the 1st row details property for the 2nd selection json properties are: ``` ## `sql.json.match` [Section titled “sql.json.match”](#sqljsonmatch) ```gherkin Then the {ordinal} row {word} property for the[[ {ordinal}]] selection[[ on {dbService}]] json properties match: | ... | ... | ``` Like the `are` form, but each value is a regular expression that must match the whole property value (as text). **Variants** (optional parts in `[[...]]` above): * `the {ordinal} row {word} property for the selection json properties match:` * `the {ordinal} row {word} property for the {ordinal} selection json properties match:` * `the {ordinal} row {word} property for the selection on {dbService} json properties match:` * `the {ordinal} row {word} property for the {ordinal} selection on {dbService} json properties match:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario) **Example:** ```gherkin Then the 1st row recipient property for the 3rd selection json properties match: ``` ## `sql.rows.eq` [Section titled “sql.rows.eq”](#sqlrowseq) ```gherkin Then the[[ {ordinal}]] selection[[ on {dbService}]] has {int} row(s) ``` Assert that a selection has exactly the given number of rows. `the selection` means the first selection of the scenario. **Variants** (optional parts in `[[...]]` above): * `the selection has {int} row(s)` * `the {ordinal} selection has {int} row(s)` * `the selection on {dbService} has {int} row(s)` * `the {ordinal} selection on {dbService} has {int} row(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{dbService}` (The name of a database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has 2 rows Then the 2nd selection on parcels-db has 1 row ``` ## `sql.rows.gt` [Section titled “sql.rows.gt”](#sqlrowsgt) ```gherkin Then the[[ {ordinal}]] selection[[ on {dbService}]] has more than {int} row(s) ``` Assert that a selection has more than the given number of rows. `the selection` means the first selection of the scenario. **Variants** (optional parts in `[[...]]` above): * `the selection has more than {int} row(s)` * `the {ordinal} selection has more than {int} row(s)` * `the selection on {dbService} has more than {int} row(s)` * `the {ordinal} selection on {dbService} has more than {int} row(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{dbService}` (The name of a database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has more than 2 rows Then the 2nd selection on parcels-db has more than 1 row ``` ## `sql.rows.lt` [Section titled “sql.rows.lt”](#sqlrowslt) ```gherkin Then the[[ {ordinal}]] selection[[ on {dbService}]] has fewer than {int} row(s) ``` Assert that a selection has fewer than the given number of rows. `the selection` means the first selection of the scenario. **Variants** (optional parts in `[[...]]` above): * `the selection has fewer than {int} row(s)` * `the {ordinal} selection has fewer than {int} row(s)` * `the selection on {dbService} has fewer than {int} row(s)` * `the {ordinal} selection on {dbService} has fewer than {int} row(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{dbService}` (The name of a database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the selection has fewer than 2 rows Then the 2nd selection on parcels-db has fewer than 1 row ``` ## `sql.trigger.raise` [Section titled “sql.trigger.raise”](#sqltriggerraise) ```gherkin Given a(n)[[ {ordinal} ordered]] before insert trigger on the {word} table[[ on {dbService}]] will raise a(n) {sqlState} exception where: | ... | ... | ``` PostgreSQL: create a BEFORE INSERT trigger that raises the SQLSTATE for inserted rows matching the table (`column | value`, `null` for IS NULL), so you can test how your app handles database errors. The trigger is dropped when the scenario ends. **Variants** (optional parts in `[[...]]` above): * `a(n) before insert trigger on the {word} table will raise a(n) {sqlState} exception where:` * `a(n) before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table will raise a(n) {sqlState} exception where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario), `{sqlState}` (A five-character SQLSTATE code, e.g. `23505` (unique violation)) **Example:** ```gherkin Given a before insert trigger on the parcels.parcels table will raise a 40001 exception where: ``` ## `sql.trigger.raise.times` [Section titled “sql.trigger.raise.times”](#sqltriggerraisetimes) ```gherkin Given a(n)[[ {ordinal} ordered]] before insert trigger on the {word} table[[ on {dbService}]] will raise a(n) {sqlState} exception {int} time(s) where: | ... | ... | ``` PostgreSQL: create a BEFORE INSERT trigger that raises the SQLSTATE for inserted rows matching the table (`column | value`, `null` for IS NULL), so you can test how your app handles database errors. The trigger is dropped when the scenario ends. Only the first N matching inserts raise. **Variants** (optional parts in `[[...]]` above): * `a(n) before insert trigger on the {word} table will raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table will raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will raise a(n) {sqlState} exception {int} time(s) where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario), `{sqlState}` (A five-character SQLSTATE code, e.g. `23505` (unique violation)), `{int}` (a 32-bit integer) **Example:** ```gherkin Given a before insert trigger on the parcels.parcels table will raise a 40001 exception 1 time where: ``` ## `sql.trigger.insertRaise` [Section titled “sql.trigger.insertRaise”](#sqltriggerinsertraise) ```gherkin Given a(n)[[ {ordinal} ordered]] before insert trigger on the {word} table[[ on {dbService}]] will insert and raise a(n) {sqlState} exception where: | ... | ... | ``` PostgreSQL: create a BEFORE INSERT trigger that raises the SQLSTATE for inserted rows matching the table (`column | value`, `null` for IS NULL), so you can test how your app handles database errors. The trigger is dropped when the scenario ends. The row is still committed through a second connection (dblink) before the error, simulating a write that succeeded but reported failure. **Variants** (optional parts in `[[...]]` above): * `a(n) before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception where:` * `a(n) before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario), `{sqlState}` (A five-character SQLSTATE code, e.g. `23505` (unique violation)) **Example:** ```gherkin Given a before insert trigger on the parcels.parcels table will insert and raise a 40001 exception where: ``` ## `sql.trigger.insertRaise.times` [Section titled “sql.trigger.insertRaise.times”](#sqltriggerinsertraisetimes) ```gherkin Given a(n)[[ {ordinal} ordered]] before insert trigger on the {word} table[[ on {dbService}]] will insert and raise a(n) {sqlState} exception {int} time(s) where: | ... | ... | ``` PostgreSQL: create a BEFORE INSERT trigger that raises the SQLSTATE for inserted rows matching the table (`column | value`, `null` for IS NULL), so you can test how your app handles database errors. The trigger is dropped when the scenario ends. The row is still committed through a second connection (dblink) before the error, simulating a write that succeeded but reported failure. Only the first N matching inserts raise. **Variants** (optional parts in `[[...]]` above): * `a(n) before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table will insert and raise a(n) {sqlState} exception {int} time(s) where:` * `a(n) {ordinal} ordered before insert trigger on the {word} table on {dbService} will insert and raise a(n) {sqlState} exception {int} time(s) where:` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario), `{sqlState}` (A five-character SQLSTATE code, e.g. `23505` (unique violation)), `{int}` (a 32-bit integer) **Example:** ```gherkin Given a before insert trigger on the parcels.parcels table will insert and raise a 40001 exception 1 time where: ``` ## `sql.trigger.raised` [Section titled “sql.trigger.raised”](#sqltriggerraised) ```gherkin Then the[[ {ordinal} ordered]] before insert trigger on the {word} table[[ on {dbService}]] was raised {int} time(s) ``` Assert how many times a simulated trigger raised its exception. Triggers are numbered in creation order; the table must match the trigger’s table. **Variants** (optional parts in `[[...]]` above): * `the before insert trigger on the {word} table was raised {int} time(s)` * `the before insert trigger on the {word} table on {dbService} was raised {int} time(s)` * `the {ordinal} ordered before insert trigger on the {word} table was raised {int} time(s)` * `the {ordinal} ordered before insert trigger on the {word} table on {dbService} was raised {int} time(s)` **Parameters:** `{ordinal}` (A 1-based position such as `1st`, `2nd`, `3rd` or `4th`. Omitting an optional ordinal means the first), `{word}` (one word, no spaces), `{dbService}` (The name of a database registered in the scenario), `{int}` (a 32-bit integer) **Example:** ```gherkin Then the before insert trigger on the parcels.parcels table was raised 1 time ``` # FAQ > Common questions about Axx - the name, how it relates to Cucumber, what it needs, and how it fits with other testing tools. ### Why “axxeptance”? [Section titled “Why “axxeptance”?”](#why-axxeptance) It is *acceptance* testing, with a name of its own. Axx is the short form, and the command is `axx`. ### Do I need Java, Node or Python? [Section titled “Do I need Java, Node or Python?”](#do-i-need-java-node-or-python) No. Axx is a single static binary. Your service can be written in anything, and Axx itself needs nothing installed. Docker is only needed if your apps start with Docker Compose, and Go only if your project adds [custom packs](/guides/use-packs/). ### Is Axx Cucumber? [Section titled “Is Axx Cucumber?”](#is-axx-cucumber) Axx runs Gherkin, the language Cucumber defined, with its own executor built on the official Cucumber libraries for Go (the Gherkin parser, Cucumber Expressions, tag expressions and Cucumber Messages). Feature files are standard Gherkin, and reports use Cucumber formats. You do not write step definitions for the built-in capabilities: they ship with Axx. ### How is this different from Postman, Karate or REST Assured? [Section titled “How is this different from Postman, Karate or REST Assured?”](#how-is-this-different-from-postman-karate-or-rest-assured) Axx tests a whole service from the outside, not only its HTTP API: it seeds databases, publishes and consumes events, verifies calls to mocked dependencies, and validates everything against your OpenAPI document. It also manages the service’s lifecycle (start, wait, stop, clean up) and runs scenarios in parallel. Scenarios are plain Gherkin, readable by the people who wrote the acceptance criteria. ### Can I test services that are not HTTP? [Section titled “Can I test services that are not HTTP?”](#can-i-test-services-that-are-not-http) Yes. SQL databases, MongoDB and Kafka are built in, and anything else can be reached with a [custom step](/guides/write-custom-steps/), written as a Go pack. ### Does Axx run my unit tests? [Section titled “Does Axx run my unit tests?”](#does-axx-run-my-unit-tests) No. Keep unit and integration tests in your language’s test framework. Axx tests the running system through its public interfaces. ### Can Axx test a deployed environment? [Section titled “Can Axx test a deployed environment?”](#can-axx-test-a-deployed-environment) Yes. Point the service URLs at the environment (with a profile, say `--profile staging`) and run with `--no-start` so Axx does not try to start apps. Keep data isolation in mind: the environment is shared with everyone else. ### Why do my scenarios pass alone and fail together? [Section titled “Why do my scenarios pass alone and fail together?”](#why-do-my-scenarios-pass-alone-and-fail-together) They share data. Scenarios run in parallel against shared infrastructure, so each one needs its own ids, names and keys. See [Scenario isolation](/explanations/scenario-isolation/) and [`axx lint`](/guides/isolate-test-data/). ### Does Axx collect telemetry? [Section titled “Does Axx collect telemetry?”](#does-axx-collect-telemetry) No. Axx makes no network calls of its own; it only talks to the services, databases and brokers your scenarios name. ### What license is Axx under? [Section titled “What license is Axx under?”](#what-license-is-axx-under) Apache-2.0. Contributions are welcome with a DCO sign-off (`git commit -s`); see [CONTRIBUTING.md](https://github.com/nimbusxr/axx/blob/main/CONTRIBUTING.md). ### Where do I report a bug or a security issue? [Section titled “Where do I report a bug or a security issue?”](#where-do-i-report-a-bug-or-a-security-issue) Bugs and questions: [GitHub issues](https://github.com/nimbusxr/axx/issues). Security issues privately, as described in [SECURITY.md](https://github.com/nimbusxr/axx/blob/main/SECURITY.md).