OpenTelemetry

Check what your services report through OpenTelemetry: the spans of what a scenario did, the logs that came with them, and their metrics.

Your service exports OTLP, over gRPC or HTTP, to a real Collector in your test environment, and the Collector’s file exporter writes it where Axx reads it. You can run that Collector with no tests at all, and send it on to Jaeger or Grafana as well. Register what it writes, usually in the Background:

Given the parcels-api telemetry with the following properties:
  | url     | file://.axx/otel/telemetry.jsonl |
  | service | parcels-api                      |

• the trace is this scenario’s own, whatever runs beside it: once telemetry is registered, the requests the scenario sends carry one traceparent header, with a trace ID of the scenario’s own, and a service that follows W3C Trace Context puts its spans and logs in that trace. A request that sets its own traceparent keeps it.
• Metrics are shared. A scenario sees what was counted since it registered the telemetry, whether the metric is cumulative or delta, but scenarios running at the same time count together: give a check attributes that are its scenario’s own.
• Checks wait. A check that something arrived waits 10 seconds, or within {duration}. A check that nothing did looks once, so put it after one that waits. A count higher than the check’s fails at once.
• Metrics take their time: SDKs export them every 60 seconds by default, so set OTEL_METRIC_EXPORT_INTERVAL, in milliseconds, in the service’s test environment.

Add it to a project with axx pack add otel.

Steps

otel.telemetry

Given the {word} telemetry with the following properties:
  | property | value |

The table may start with the row naming its columns, as above, or go straight to its rows.

Register what a Collector’s file exporter writes.

• From here on, the requests the scenario sends carry this scenario’s traceparent header.
• Metrics are counted from here on.

Parameter	Takes	For example
{word}	one word, with no spaces	parcels, PX-4101

Property	Takes	Default
url	the exporter’s file: file://, relative to the directory of axx.yaml or absolute	required
service	the service.name whose metrics the steps that name this telemetry look at; the others look at every service’s

Example:

Given the parcels-api telemetry with the following properties:
  | url     | file://.axx/otel/telemetry.jsonl |
  | service | parcels-api                      |

Since 0.3.1.

otel.trace.span

Then [[within {duration} ]]the trace has a span where:
  | field | value |

The table may start with the row naming its columns, as above, or go straight to its rows.

Check that this scenario’s trace has a span with those values.

• The check waits for it: 10 seconds, or within {duration}.
• undefined as an attribute’s value is a span without that attribute.

Parameter	Takes	For example
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m

Field	Takes
name	the span’s name, all of it
status	OK, ERROR or UNSET
kind	SERVER, CLIENT, INTERNAL, PRODUCER or CONSUMER
service	the service that reported it
attribute <key>	an attribute’s value, of its type: its key after attribute

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Variants, the parts in [[...]] said or left out:

• the trace has a span where:
• within {duration} the trace has a span where:

Example:

Then within 10s the trace has a span where:
  | name                 | POST /api/parcels |
  | attribute http.route | /api/parcels      |

Since 0.3.1.

otel.trace.nospan

Then the trace has no span where:
  | field | value |

The table may start with the row naming its columns, as above, or go straight to its rows.

Check that this scenario’s trace has no span with those values.

• It looks once, at what has arrived: put it after a check that waits.

Field	Takes
name	the span’s name, all of it
status	OK, ERROR or UNSET
kind	SERVER, CLIENT, INTERNAL, PRODUCER or CONSUMER
service	the service that reported it
attribute <key>	an attribute’s value, of its type: its key after attribute

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Example:

Then the trace has no span where:
  | status | ERROR |

Since 0.3.1.

otel.trace.child

Then the {string} span is a child of the {string} span

Check that a span of this scenario’s trace has the other for its parent, waiting for them for 10 seconds.

Parameter	Takes	For example
{string}	text in double or single quotes, which the step leaves out	"Get a quote", 'Express'

Example:

Then the "store parcel" span is a child of the "POST /api/parcels" span

Since 0.3.1.

otel.trace.took

Then the {string} span took less than {duration}

Check how long a span of this scenario’s trace took, waiting for it for 10 seconds.

• Every span of that name in the trace must have.

Parameter	Takes	For example
{string}	text in double or single quotes, which the step leaves out	"Get a quote", 'Express'
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m

Example:

Then the "POST /api/parcels" span took less than 500ms

Since 0.3.1.

otel.trace.log

Then [[within {duration} ]]the trace has a log where:
  | field | value |

The table may start with the row naming its columns, as above, or go straight to its rows.

Check that this scenario’s trace has a log record with those values.

• The check waits for it: 10 seconds, or within {duration}.
• A record is in the trace when the service logged it with the span’s context.

Parameter	Takes	For example
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m

Field	Takes
severity	its severity: INFO, WARN, ERROR..
body	text its body contains
service	the service that reported it
attribute <key>	an attribute’s value, of its type: its key after attribute

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Variants, the parts in [[...]] said or left out:

• the trace has a log where:
• within {duration} the trace has a log where:

Example:

Then within 10s the trace has a log where:
  | severity | ERROR |

Since 0.3.1.

otel.counted

Then [[within {duration} ]]the {telemetry} telemetry counted {int} {word}[[ where:]]

Check what a counter has counted since the telemetry was registered, cumulative or delta.

• The check waits for that many: 10 seconds, or within {duration}; more fails at once.
• With 0 it looks once, or at the end of within {duration}.

Parameter	Takes	For example
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m
{telemetry}	the name of telemetry registered in the scenario	parcels-api
{int}	a whole number	200, 3
{word}	one word, with no spaces	parcels, PX-4101

Field	Takes
attribute <key>	the value of an attribute of the data points to count: its key after attribute
service	the service that reported them

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Variants, the parts in [[...]] said or left out:

• the {telemetry} telemetry counted {int} {word}
• the {telemetry} telemetry counted {int} {word} where:
• within {duration} the {telemetry} telemetry counted {int} {word}
• within {duration} the {telemetry} telemetry counted {int} {word} where:

Examples:

Then within 30s the parcels-api telemetry counted 1 parcels_registered where:
  | attribute sender | juniper-and-jay |

Then the parcels-api telemetry counted 0 parcels_rejected

Since 0.3.1.

otel.recorded

Then [[within {duration} ]]the {telemetry} telemetry recorded {int} {word} values[[ where:]]

Check how many values a histogram has recorded since the telemetry was registered.

• The check waits for that many: 10 seconds, or within {duration}; more fails at once.

Parameter	Takes	For example
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m
{telemetry}	the name of telemetry registered in the scenario	parcels-api
{int}	a whole number	200, 3
{word}	one word, with no spaces	parcels, PX-4101

Field	Takes
attribute <key>	the value of an attribute of the data points to count: its key after attribute
service	the service that reported them

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Variants, the parts in [[...]] said or left out:

• the {telemetry} telemetry recorded {int} {word} values
• the {telemetry} telemetry recorded {int} {word} values where:
• within {duration} the {telemetry} telemetry recorded {int} {word} values
• within {duration} the {telemetry} telemetry recorded {int} {word} values where:

Example:

Then within 30s the parcels-api telemetry recorded 1 parcel_weight values where:
  | attribute sender | juniper-and-jay |

Since 0.3.1.

otel.gauge

Then [[within {duration} ]]the {telemetry} telemetry's {word} gauge is {gaugeValue}[[ where:]]

Check the latest value a gauge reported since the telemetry was registered.

• The check waits for that value: 10 seconds, or within {duration}.
• The value keeps its type: 4 is an integer, and 4.0 a decimal.

Parameter	Takes	For example
{duration}	a duration in milliseconds (ms), seconds (s) or minutes (m)	5s, 2m
{telemetry}	the name of telemetry registered in the scenario	parcels-api
{word}	one word, with no spaces	parcels, PX-4101
{gaugeValue}	a gauge’s value, of its type: 4 an integer, and 4.0 a decimal	4, 4.5

Field	Takes
attribute <key>	the value of an attribute of the data points to count: its key after attribute
service	the service that reported them

Values are typed: 42 an integer, 1.5 a decimal, true and false booleans, null null, {...} and [...] JSON, "42" in double quotes the string, and other text a string; 42 is not "42", nor 2 2.0. undefined is no such property.

Variants, the parts in [[...]] said or left out:

• the {telemetry} telemetry's {word} gauge is {gaugeValue}
• the {telemetry} telemetry's {word} gauge is {gaugeValue} where:
• within {duration} the {telemetry} telemetry's {word} gauge is {gaugeValue}
• within {duration} the {telemetry} telemetry's {word} gauge is {gaugeValue} where:

Example:

Then within 30s the parcels-api telemetry's last_parcel_weight gauge is 1200 where:
  | attribute sender | juniper-and-jay |

Since 0.3.1.

otel.contains.not

Then the telemetry does not contain {string}

Check that nothing the services reported since the telemetry was registered contains that text: what must stay out of your telemetry, like a person’s name.

• It looks once, at what has arrived, and fails when nothing has: put it after a check that waits.

Parameter	Takes	For example
{string}	text in double or single quotes, which the step leaves out	"Get a quote", 'Express'

Example:

Then the telemetry does not contain 'Marta Keller'

Since 0.3.1.

Parameter types

Parameter	Takes	Values	For example
{gaugeValue}	a gauge’s value, of its type: 4 an integer, and 4.0 a decimal		4, 4.5
{telemetry}	the name of telemetry registered in the scenario		parcels-api