Making the examples executable
This page explains instrumenting Markdown specifications. Click the toggle buttons above to choose other formats.
- assert-equals command
- set command
- example command
- execute command
- verify-rows command
- run command
- assert-true and assert-false commands
- echo command
- Expression language
Concordion commands are added as links in Markdown documents.
Concordion commands are differentiated from other Markdown links by using a hyphen (
-) for the URL:
As an alternative to inline links, reference style links are supported, for example:
[value][id] [id]: - "command"
[value] [value]: - "command"
Reference style links can help improve readability of the Markdown document, especially for commands that are in table headers or are lengthy or repeated .
Let’s start with a really simple example…
Create a file
[Hello World!](- "?=getGreeting()")
When run with a fixture that implements the
getGreeting() method to return
Hello World!, the output specification will show:
In the example above, the call to
getGreeting() can be simplified to
greeting since Concordion’s expression language includes properties support.
[Hello World!](- "?=greeting")
Note that the
?= syntax is short for
Given a specification like this:
The greeting for user Bob will be: Hello Bob!
We want the first name (
Bob) to be a parameter and the greeting (
Hello Bob!) to be verified against the result returned by the system.
To do this we place links around the two significant pieces of text in the document. When the specification is executed, these links will be changed into HTML attributes and will not show as links in the output specification.
The greeting for user [Bob]() will be: [Hello Bob!]()
Now we can instrument the document:
The greeting for user [Bob](- "#firstName") will be: [Hello Bob!](- "?=greetingFor(#firstName)")
When Concordion processes the document, it will set a specification variable
#firstName to the value
Bob and then call the
greetingFor() method with that value and check that the result is equal to
Note that the
# syntax is short for
To specify that a piece of the specification is an example, add a heading link with the name of the example set as the link title. For example:
## [Example 1](- "example1") Example goes here
The example name is optional and will be generated from the heading text if not supplied. For example:
## [Check 3 items](-)
will create an example named
check-3-items with the heading
Check 3 items.
Each example is run and reported as a separate test. Any commands that are outside of named examples are executed in a single anonymous “outer” example that is run before the named examples.
Each row of a table can also be run as an example.
To specify that a piece of the specification should be run before each example, add a heading link with the keyword
before as the link title. For example:
## [Per example setup](- "before") Example goes here
before example is intended to be used for functionality required by each example where adding it to the specification improves the communication aspect for the reader.
Where the functionality is not needed to clarify the specification, use the
BeforeExample hook. This pushes the functionality down into the fixture, making the specification clearer and providing flexibility in implementation.
Any per-example teardown must be specified using the
AfterExample hook. There is no
after example available in the specification since tearing down context doesn’t add much to the communication aspect of specifications.
Closing an example
The example block continues until it is closed either implicitly or explicitly.
An example is implicitly closed on any of these conditions:
- another example starts, or
- a header is encountered that is at a higher level than the example header (eg. the example is a
h2header is encountered), or
- the end of file is reached.
To explicitly close an example, create a header with the example heading struck-through. For example:
## ~~Example 1~~
will close the example with the heading
Example 1. The struck-through header will not appear in the output specification.
As an alternative to setting the implementation status for the whole specification, you can set it for individual examples. This allows partially-implemented examples in the specification without breaking the build.
## [Example 3](- "example3 c:status=ExpectedToFail")
The status can be
The execute command has three main uses:
- Executing an instruction with a “void” result.
- Executing an instruction with an object result (to allow multiple properties of the object to be checked).
- Handling unusual sentence structures.
Executing an instruction with a void result
It can occasionally be useful to execute an instruction that sets up some system state. Every time you do this, however, alarm bells should ring in your head and you should question yourself to make sure that you are not inadvertently writing a script instead of a specification. E.g. a call to “clearDatabase()” would be a blatant misuse (see Hints and Tips for more on this topic).
As a rule of thumb, methods with a void result called from an execute should start with the word set or setUp. E.g. setUpUser(#username).
Take the following specification for example:
If the time is [09:00AM](- "#time") [ ](- "setCurrentTime(#time)") then the greeting will say: [Good Morning World!](- "?=getGreeting()")
We can actually remove the need for the set command by using the special variable #TEXT (which contains the text of the current element). The abbreviated instrumentation looks like this:
If the time is [09:00AM](- "setCurrentTime(#TEXT)") then the greeting will say: [Good Morning World!](- "?=getGreeting()")
An alternative would be to change the getGreeting() method signature to allow the time to be passed in as a parameter. This is the approach you should normally take. An execute with no return value often indicates a “bad smell” - e.g. you’re writing a script or your specification contains too many variables and covers too many behaviours. However, the functionality is there if you need it.
(Note that the execute command is deduced by the absence of a
c: prefix in the link title. It is equivalent to prefixing the command with
Executing an instruction with an object result
Sometimes you need to check more than one result of a behaviour. For example, here we want to check that both the first name and the last name are correctly extracted from the full name:
# Splitting Names To help personalise our mailshots we want to have the first name and last name of the customer. Unfortunately the customer data that we are supplied only contains full names. The system therefore attempts to break a supplied full name into its constituents by splitting around whitespace. ### [Example](- "simple-name") The full name [Jane Smith](- "#result = split(#TEXT)") will be broken into first name [Jane](- "?=#result.firstName") and last name [Smith](- "?=#result.lastName")
#result is going to be an object returned by the
split() method. This object will have
Handling unusual sentence structures
One of the great things about Concordion is that when you’re writing the specifications you do not have to worry about how you’re going to instrument it. You can just concentrate on making the document as readable as possible.
Most English sentences can be instrumented. If you can’t work out how to instrument it then you can always tweak the wording, but in general this should not be necessary. The execute command provides flexibility - however, you will need to embed HTML in your Markdown specification to achieve this, so will need to weigh that up against changing the wording.
For example, say we have the specification:
Upon login, the greeting for user [Bob]() will be: [Hello Bob!]())
This is easy to instrument:
Upon login, the greeting for user [Bob](- "#firstName") will be: [Hello Bob!](- "?=greetingFor(#firstName)")
But what if our specification was written like this:
The greeting "[Hello Bob!]()" should be given to user [Bob]() when he logs in.
In this case, the input parameter Bob occurs after the output greeting we want to check. We can solve this problem by changing this sentence to HTML, wrapping it in a
<div> element and using an execute command on the outer element (the <p>).
<div> <p concordion:execute="#greeting = greetingFor(#firstName)"> The greeting "<span concordion:assert-equals="#greeting">Hello Bob!</span>" should be given to user <span concordion:set="#firstName">Bob</span> when he logs in. </p> </div>
How does this work? It works because the execute command is designed to process commands on its child elements in a special order. First of all it processes any child set commands then it runs its own command, then any child execute commands and finally any child assert-equals commands.
execute command on a table
When you want to show several examples of a behaviour, repeating the same sentence structure over and over again probably isn’t going to be very nice to read. It would be better to use a table.
You can instrument this table, in a long-winded way, as follows:
### [Examples](- "simple-names") | Full Name | First Name | Last Name | | --------------- | --------------- | --------------- | | [John Smith][split] | [John][first] | [Smith][last] | | [David Peterson][split] | [David][first] | [Peterson][last] | [split]: - "#result = split(#TEXT)" [first]: - "?=#result.firstName" [last]: - "?=#result.lastName"
However, this is repetitive so Concordion provides a shortcut by placing commands on each table header column, with an additional execute command at the start of the first column. For each row of the table, the commands from the table headers are run, with any set commands being run before the execute command, followed by the assert commands.
### [Examples](- "simple-names") | [split][Full Name][full] | [First Name][first] | [Last Name][last] | | --------------- | --------------- | --------------- | | John Smith | John | Smith | | David Peterson | David | Peterson | [split]: - "#result = split(#fullName)" [full]: - "#fullName" [first]: - "?=#result.firstName" [last]: - "?=#result.lastName"
This instrumentation has identical behaviour to the previous example.
Run each row as an example
To run each row of the table as an example (which will be run and reported as a separate test), add an ‘example’ command to a column header. The text in that column will be used for the example name.
| [split][Description](- "c:example") | [Full Name][full] | [First Name][first] | [Last Name][last] | | --------------- | ------------- | --------------- | --------------- | | Simple name | David Peterson | David | Peterson | | Double-barrelled name | Mike Cannon-Brookes | Mike | Cannon-Brookes | [split]: - "#result = split(#fullName)" [full]: - "#fullName" [first]: - "?=#result.firstName" [last]: - "?=#result.lastName"
will run as two examples, named “Simple name” and “Double-barrelled name”.
execute command on a list
execute command has special behavior when placed on a list element (
<ul>). Instead of executing once, it executes every list item in the list (and all its sub lists) and transfers the commands from the list element to each list item element. This feature can for example be used to setup a hierarchical structure of test data.
There is currently no explicit support for this in the Concordion Markdown syntax. Instead the HTML version of this must be used, wrapped in a
When you want to check the contents of a collection of results returned from the system, use the
For example, while writing a user administration tool we might write a specification like this describing the behaviour of the search functionality:
The idea is that in the fixture code we’ll set up the users in the system, perform a search and then confirm that the right users (and only these users) were returned in the search results. If too many, too few, or the wrong users were returned we want the test to fail.
The instrumented source for the specification looks like this:
# Partial Matches Username searches return partial matches, i.e. all usernames containing the search string are returned. ### [Example](- "beatles") Given these users: | [ ][setup] [Username][user]| |------------------------------------------| | john.lennon | | ringo.starr | | george.harrison | | paul.mccartney | [setup]: - "setUpUser(#username)" [user]: - "#username" Searching for [arr](- "#searchString") will return: | [ ][search] [Matching Usernames][match]| |------------------------------------------| | george.harrison | | ringo.starr | [search]: - "c:verify-rows=#username:getSearchResultsFor(#searchString)" [match]: - "?=#username"
The syntax for a verify-rows command is:
#loopVar : expression
expression returns an Iterable object with a predictable iteration order, (e.g. a List, LinkedHashSet or a TreeSet), and
#loopVar provides access to the current object during iteration and allows the assert-equals method to check its value.
By default, the order of the items in the table being verified must match the iteration order of the items returned by the expression. You may need to sort the items to ensure they are in a known and consistent order. In our example, we are using alphabetical order (“george” before “ringo”). As an alternative, you can apply a match strategy to define how the rows are matched (since: 2.0.0).
When run with a fixture that returns an empty collection, we get:
Two rows are missing because our search function is not implemented and returns an empty set.
After the feature is implemented, when we run it we get a success:
Note that either
verifyRows can be used for the command name.
run command lets you run another specification from this specification, and build up test suites. Executing the specification will automatically execute all its linked specifications (recursively), with the results aggregated up.
The format is:
[some link text](path/to/Spec.md "c:run")
Spec.md is the name of the specification to be run, with the relative path
You can link to specifications with different file extensions (eg.
.xlsx if the Excel extension is installed). Concordion will automatically update the
href value to use the
.html file extension in the output specification.
The specification to be run must have a corresponding fixture class appropriately named (eg.
assert-true and assert-false commands
These commands are useful for asserting boolean conditions.
They should be used sparingly, since on failure they can only report true or false. For example, when the specification:
The completion date should be set to [today](- "c:assert-true=isCompletionToday()")
is run with a
isCompletionToday() method that returns false, the output shows:
As an alternative, use the
assert-equals command to show better error messages. Reworking our example above:
The completion date should be set to [today](- "?=getCompletionDay()").
When the completion date is today, return “today” from
getCompletionDay() to show success:
When the completion date is not today, return the actual date from
getCompletionDay() to show the actual date in the output specification:
Further details: assert-true command specification
echo command evaluates an expression and inserts the result into the output HTML. One usage is to display environment details in a page footer.
Username:[ ](- "c:echo=#username")
Further details: echo command specification
The echo command allows you to insert the results of evaluating an expression as text. Should you wish to insert HTML into the document, you will need to use the embed extension.
This can be overridden to allow complex expressions.