Code Coverage with Behat
First published at Wednesday, 3 April 2013
Warning: This blog post is more then 11 years old – read and use with care.
Code Coverage with Behat
There is generally no point in having code coverage for Behat test cases because of their nature: The purpose of an acceptance test is to assert a certain behavior of an application, not to technically test a piece of code. Therefore, there is no point in checking for uncovered code pieces in order to write a Behat test for it.
That said, there is still a scenario where you want to peek at code coverage of Behat tests: When creating them as wide-coverage tests before starting to refactor legacy code. Behat in combination with Mink provides you with a great tool for such tests.
Before you can start with refactoring legacy code you need tests to ensure that you don't break working functionality. Web acceptance tests on basis of Behat and Mink are a great tool to realize these. But how can you detect if the code you are about to refactor is touched by at least one test? Code coverage can be of assistance there.
Preparation
Since Behat does not ship with code coverage (for very good reason), you need some hand work to get that done, but not much. In order to get started, you need to install the PHP_CodeCoverage library and phpcov, most probably via PEAR using:
$ pear config-set auto_discover 1
$ pear install -a pear.phpunit.de/phpcovCollecting Code Coverage
Since the Behat tests stimulate your application through external calls, it is not possible to generate code coverage right from the test code. Instead, you need to trigger the code coverage collection from your application code:
<?php
// ... forbid production access here ...
$calculateCoverage = file_exists("/tmp/generate-behat-coverage");
if ($calculateCoverage) {
require 'PHP/CodeCoverage/Autoload.php';
$filter = new PHP_CodeCoverage_Filter();
$filter->addDirectoryToBlacklist(__DIR__ . "/../vendor");
$filter->addDirectoryToWhitelist(__DIR__ . "/../src");
$coverage = new PHP_CodeCoverage(null, $filter);
$coverage->start('Behat Test');
}
// ... run your application here ...
if ($calculateCoverage) {
$coverage->stop();
$writer = new PHP_CodeCoverage_Report_PHP;
$writer->process($coverage, __DIR__ . "/../log/behat-coverage/" . microtime(true) . ".cov");
}At first the code detects if code coverage information should be gathered by checking if the file /tmp/generate-behat-coverage exists. You can touch and delete that one manually or from your test setup.
The next code block loads and initializes the code coverage collection, creates a filter for 3rd party code and starts the code coverage collection. After that, the comment indicates to run your application, which might e.g. be a Symfony2 kernel handling call.
The final lines write the code coverage information into a file for further processing. It will create a dedicated file for each request, where these files then need to be merged later.
Running Tests
With the shown code in place, you can trigger a Behat test run with code coverage using the following Ant code, for example:
<target name="behat-coverage" depends="clean, initialize">
<delete dir="${commons:logsdir}/behat-coverage" />
<mkdir dir="${commons:logsdir}/behat-coverage" />
<touch file="/tmp/generate-behat-coverage" />
<antcall target="behat" />
<delete file="/tmp/generate-behat-coverage" />
<exec executable="phpcov" failonerror="false" dir="${basedir}">
<arg value="--merge" />
<arg value="--html" />
<arg value="${commons:logsdir}/../coverage/behat" />
<arg value="${commons:logsdir}/behat-coverage" />
</exec>
</target>The Ant target first cleans up code coverage from previous runs. It then touches the file that indicates to the application to run code coverage, executes Behat and removes the trigger file again. Then the phpcov utility is executed to merge the results of all requests into a single coverage report and generate HTML from it.
Conclusion
Code coverage is completely out of scope for acceptance tests. However, if you abuse Behat to create wide-coverage tests before refactoring, it might be of help to you to see what is still missing before you start hacking.
Subscribe to updates
There are multiple ways to stay updated with new posts on my blog: