Instrumenting

Making the examples executable

This page explains instrumenting Markdown specifications. Click the toggle buttons above to choose other formats.

Overview

Concordion commands are added as links in Markdown documents.

Concordion commands are differentiated from other Markdown links by using a hyphen (-) for the URL:

[value](- "command")

As an alternative to inline links, reference style links are supported, for example:

[value][id]
    
[id]: - "command"

or

[value][]
    
[value]: - "command"

Reference style links can help improve readability of the Markdown document, especially for commands that are in table headers or are lengthy or repeated .

Let’s start with a really simple example…

assert-equals command

Create a file HelloWorld.md containing:

[Hello World!](- "?=getGreeting()")

When run with a fixture that implements the getGreeting() method to return Hello World!, the output specification will show:

successful specification

Properties support

In the example above, the call to getGreeting() can be simplified to greeting since Concordion’s expression language includes properties support.

[Hello World!](- "?=greeting")

Note that the ?= syntax is short for c:assert-equals

Further details: assert-equals command specification and Markdown grammar

set command

Given a specification like this:

The greeting for user Bob will be: Hello Bob!

We want the first name (Bob) to be a parameter and the greeting (Hello Bob!) to be verified against the result returned by the system.

To do this we place links around the two significant pieces of text in the document. When the specification is executed, these links will be changed into HTML attributes and will not show as links in the output specification.

The greeting for user [Bob]() will be: [Hello Bob!]()

Now we can instrument the document:

The greeting for user [Bob](- "#firstName") will be: [Hello Bob!](- "?=greetingFor(#firstName)")

When Concordion processes the document, it will set a specification variable #firstName to the value Bob and then call the greetingFor() method with that value and check that the result is equal to Hello Bob!.

Note that the # syntax is short for c:set=#

Further details: set command specification and Markdown grammar

example command

since: 2.0.0

To specify that a piece of the specification is an example, add a heading link with the name of the example set as the link title. For example:

## [Example 1](- "example1")

Example goes here

The example name is optional and will be generated from the heading text if not supplied. For example:

## [Check 3 items](-)

will create an example named check-3-items with the heading Check 3 items.

Each example is run and reported as a separate test. Any commands that are outside of named examples are executed in a single anonymous “outer” example that is run before the named examples.

Each row of a table can also be run as an example.

“before” examples

To specify that a piece of the specification should be run before each example, add a heading link with the keyword before as the link title. For example:

## [Per example setup](- "before")

Example goes here

A before example is intended to be used for functionality required by each example where adding it to the specification improves the communication aspect for the reader.

Where the functionality is not needed to clarify the specification, use the BeforeExample hook. This pushes the functionality down into the fixture, making the specification clearer and providing flexibility in implementation.

Any per-example teardown must be specified using the AfterExample hook. There is no after example available in the specification since tearing down context doesn’t add much to the communication aspect of specifications.

Closing an example

The example block continues until it is closed either implicitly or explicitly.

An example is implicitly closed on any of these conditions:

  • another example starts, or
  • a header is encountered that is at a higher level than the example header (eg. the example is a h3 and a h2 header is encountered), or
  • the end of file is reached.

To explicitly close an example, create a header with the example heading struck-through. For example:

## ~~Example 1~~

will close the example with the heading Example 1. The struck-through header will not appear in the output specification.

Implementation Status

As an alternative to setting the implementation status for the whole specification, you can set it for individual examples. This allows partially-implemented examples in the specification without breaking the build.

## [Example 3](- "example3 c:status=ExpectedToFail")

The status can be ExpectedToFail, Ignored or Unimplemented.

Further details: example command specification and Markdown grammar

execute command

The execute command has three main uses:

  1. Executing an instruction with a “void” result.
  2. Executing an instruction with an object result (to allow multiple properties of the object to be checked).
  3. Handling unusual sentence structures.

Executing an instruction with a void result

It can occasionally be useful to execute an instruction that sets up some system state. Every time you do this, however, alarm bells should ring in your head and you should question yourself to make sure that you are not inadvertently writing a script instead of a specification. E.g. a call to “clearDatabase()” would be a blatant misuse (see Hints and Tips for more on this topic).

As a rule of thumb, methods with a void result called from an execute should start with the word set or setUp. E.g. setUpUser(#username).

Take the following specification for example:

If the time is [09:00AM](- "#time") [ ](- "setCurrentTime(#time)")
then the greeting will say:
[Good Morning World!](- "?=getGreeting()")

We can actually remove the need for the set command by using the special variable #TEXT (which contains the text of the current element). The abbreviated instrumentation looks like this:

If the time is [09:00AM](- "setCurrentTime(#TEXT)") 
then the greeting will say:
[Good Morning World!](- "?=getGreeting()")

An alternative would be to change the getGreeting() method signature to allow the time to be passed in as a parameter. This is the approach you should normally take. An execute with no return value often indicates a “bad smell” - e.g. you’re writing a script or your specification contains too many variables and covers too many behaviours. However, the functionality is there if you need it.

(Note that the execute command is deduced by the absence of a #, ?= or c: prefix in the link title. It is equivalent to prefixing the command with c:execute=).

Executing an instruction with an object result

Sometimes you need to check more than one result of a behaviour. For example, here we want to check that both the first name and the last name are correctly extracted from the full name:

# Splitting Names

To help personalise our mailshots we want to have the first name and last name of the customer. 
Unfortunately the customer data that we are supplied only contains full names.

The system therefore attempts to break a supplied full name into its constituents by splitting around whitespace.

### [Example](- "simple-name")

The full name [Jane Smith](- "#result = split(#TEXT)")
will be broken into first name [Jane](- "?=#result.firstName")
and last name [Smith](- "?=#result.lastName")

The variable #result is going to be an object returned by the split() method. This object will have firstName and lastName properties.

Alternatively, to make it easier to return multiple values without creating a new object, you can return a MultiValueResult or return a Map result.

Handling unusual sentence structures

One of the great things about Concordion is that when you’re writing the specifications you do not have to worry about how you’re going to instrument it. You can just concentrate on making the document as readable as possible.

Most English sentences can be instrumented. If you can’t work out how to instrument it then you can always tweak the wording, but in general this should not be necessary. The execute command provides flexibility - however, you will need to embed HTML in your Markdown specification to achieve this, so will need to weigh that up against changing the wording.

For example, say we have the specification:

Upon login, the greeting for user [Bob]() will be: [Hello Bob!]())

This is easy to instrument:

Upon login, the greeting for user [Bob](- "#firstName") will be: [Hello Bob!](- "?=greetingFor(#firstName)")

But what if our specification was written like this:

The greeting "[Hello Bob!]()" should be given to user [Bob]() when he logs in.

In this case, the input parameter Bob occurs after the output greeting we want to check. We can solve this problem by changing this sentence to HTML, wrapping it in a <div> element and using an execute command on the outer element (the <p>).

<div>
    <p concordion:execute="#greeting = greetingFor(#firstName)">
        The greeting "<span concordion:assert-equals="#greeting">Hello Bob!</span>"
        should be given to user <span concordion:set="#firstName">Bob</span>
        when he logs in.
    </p>
</div>

How does this work? It works because the execute command is designed to process commands on its child elements in a special order. First of all it processes any child set commands then it runs its own command, then any child execute commands and finally any child assert-equals commands.

Further details: execute command specification and Markdown grammar

execute command on a table

When you want to show several examples of a behaviour, repeating the same sentence structure over and over again probably isn’t going to be very nice to read. It would be better to use a table.

For example:

How the table is displayed (nice and neat)

You can instrument this table, in a long-winded way, as follows:

### [Examples](- "simple-names")

| Full Name               | First Name      | Last Name |
| ---------------         | --------------- | --------------- |
| [John Smith][split]     | [John][first]   | [Smith][last] |
| [David Peterson][split] | [David][first]  | [Peterson][last] |

[split]: - "#result = split(#TEXT)"
[first]: - "?=#result.firstName"
[last]:  - "?=#result.lastName"

However, this is repetitive so Concordion provides a shortcut by placing commands on each table header column, with an additional execute command at the start of the first column. For each row of the table, the commands from the table headers are run, with any set commands being run before the execute command, followed by the assert commands.

For example:

### [Examples](- "simple-names")

| [split][][Full Name][full] | [First Name][first] | [Last Name][last] |
| ---------------            | ---------------     | ---------------   |
| John Smith                 | John                | Smith             |
| David Peterson             | David               | Peterson          |

[split]: - "#result = split(#fullName)"
[full]: - "#fullName"
[first]: - "?=#result.firstName"
[last]:  - "?=#result.lastName"

This instrumentation has identical behaviour to the previous example.

Run each row as an example

since: 2.1.0

To run each row of the table as an example (which will be run and reported as a separate test), add an ‘example’ command to a column header. The text in that column will be used for the example name.

For example:

| [split][][Description](- "c:example") | [Full Name][full]   | [First Name][first] | [Last Name][last] |
| ---------------                       | -------------       | ---------------     | ---------------   |
| Simple name                           | David Peterson      | David               | Peterson          |
| Double-barrelled name                 | Mike Cannon-Brookes | Mike                | Cannon-Brookes    |

[split]: - "#result = split(#fullName)"
[full]: - "#fullName"
[first]: - "?=#result.firstName"
[last]:  - "?=#result.lastName"

will run as two examples, named “Simple name” and “Double-barrelled name”.

Further details: execute command on a table specification and Markdown syntax

execute command on a list

since 1.4.6

The execute command has special behavior when placed on a list element (<ol> or <ul>). Instead of executing once, it executes every list item in the list (and all its sub lists) and transfers the commands from the list element to each list item element. This feature can for example be used to setup a hierarchical structure of test data.

There is currently no explicit support for this in the Concordion Markdown syntax. Instead the HTML version of this must be used, wrapped in a <div> element.

Further details: execute command on a list specification and Markdown syntax

verify-rows command

When you want to check the contents of a collection of results returned from the system, use the verify-rows command.

For example, while writing a user administration tool we might write a specification like this describing the behaviour of the search functionality:

Original Specification

The idea is that in the fixture code we’ll set up the users in the system, perform a search and then confirm that the right users (and only these users) were returned in the search results. If too many, too few, or the wrong users were returned we want the test to fail.

The instrumented source for the specification looks like this:

# Partial Matches

Username searches return partial matches, i.e. all usernames containing the search string are returned.

### [Example](- "beatles")

Given these users:

| [ ][setup] [Username][user]|
|------------------------------------------|
| john.lennon |
| ringo.starr |
| george.harrison |
| paul.mccartney |

[setup]: - "setUpUser(#username)"
[user]:   - "#username"

Searching for [arr](- "#searchString") will return:

| [ ][search] [Matching Usernames][match]|
|------------------------------------------|
| george.harrison |
| ringo.starr |

[search]: - "c:verify-rows=#username:getSearchResultsFor(#searchString)"
[match]: - "?=#username"

The syntax for a verify-rows command is:

#loopVar : expression

where expression returns an Iterable object with a predictable iteration order, (e.g. a List, LinkedHashSet or a TreeSet), and #loopVar provides access to the current object during iteration and allows the assert-equals method to check its value.

By default, the order of the items in the table being verified must match the iteration order of the items returned by the expression. You may need to sort the items to ensure they are in a known and consistent order. In our example, we are using alphabetical order (“george” before “ringo”). As an alternative, you can apply a match strategy to define how the rows are matched (since: 2.0.0).

When run with a fixture that returns an empty collection, we get:

Two missing rows

Two rows are missing because our search function is not implemented and returns an empty set.

After the feature is implemented, when we run it we get a success:

Success

Note that either verify-rows or verifyRows can be used for the command name.

Further details: verify-rows specification and Markdown grammar

run command

The run command lets you run another specification from this specification, and build up test suites. Executing the specification will automatically execute all its linked specifications (recursively), with the results aggregated up.

The format is:

[some link text](path/to/Spec.md "c:run")

where Spec.md is the name of the specification to be run, with the relative path path/to.

You can link to specifications with different file extensions (eg. .md, .html, .xlsx if the Excel extension is installed). Concordion will automatically update the href value to use the .html file extension in the output specification.

The specification to be run must have a corresponding fixture class appropriately named (eg. Spec, SpecTest or SpecFixture).

Further details: run command specification and Markdown grammar

assert-true and assert-false commands

These commands are useful for asserting boolean conditions.

They should be used sparingly, since on failure they can only report true or false. For example, when the specification:

The completion date should be set to [today](- "c:assert-true=isCompletionToday()")

is run with a isCompletionToday() method that returns false, the output shows:

specification showing false result

As an alternative, use the assert-equals command to show better error messages. Reworking our example above:

The completion date should be set to [today](- "?=getCompletionDay()").

When the completion date is today, return “today” from getCompletionDay() to show success:

specification showing today as success

When the completion date is not today, return the actual date from getCompletionDay() to show the actual date in the output specification:

specification showing the actual date as failure

Further details: assert-true command specification

echo command

The echo command evaluates an expression and inserts the result into the output HTML. One usage is to display environment details in a page footer.

For example:

Username:[ ](- "c:echo=#username")

Further details: echo command specification

The echo command allows you to insert the results of evaluating an expression as text. Should you wish to insert HTML into the document, you will need to use the embed extension.

Expression language

In order to keep your specifications simple and maintainable, Concordion deliberately restricts the expression language that is allowed when instrumenting specifications.

This can be overridden to allow complex expressions.

Codingchevron_right