Qafoo GmbH - passion for software quality

Help you and your team benefit from new perspectives on cutting-edge quality engineering techniques and tools through the Qafoo team weblog.

By Tobias Schlitt, first published at Wed, 03 Apr 2013 07:52:04 +0200

Code Coverage with Behat

There is generally no point in having code coverage for Behat test cases because of their nature: The purpose of an acceptance test is to assert a certain behavior of an application, not to technically test a piece of code. Therefore, there is no point in checking for uncovered code pieces in order to write a Behat test for it.

Refactoring is an important skill for your team to make software maintainable and keep it like that. Base these capabilities on the sound basis of an individual Qafoo training for your team.

That said, there is still a scenario where you want to peek at code coverage of Behat tests: When creating them as wide-coverage tests before starting to refactor legacy code. Behat in combination with Mink provides you with a great tool for such tests.

Before you can start with refactoring legacy code you need tests to ensure that you don't break working functionality. Web acceptance tests on basis of Behat and Mink are a great tool to realize these. But how can you detect if the code you are about to refactor is touched by at least one test? Code coverage can be of assistance there.

Preparation

Since Behat does not ship with code coverage (for very good reason), you need some hand work to get that done, but not much. In order to get started, you need to install the PHP_CodeCoverage library and phpcov, most probably via PEAR using:

$ pear config-set auto_discover 1 $ pear install -a pear.phpunit.de/phpcov

Collecting Code Coverage

Since the Behat tests stimulate your application through external calls, it is not possible to generate code coverage right from the test code. Instead, you need to trigger the code coverage collection from your application code:

<?php // ... forbid production access here ... $calculateCoverage = file_exists("/tmp/generate-behat-coverage"); if ($calculateCoverage) { require 'PHP/CodeCoverage/Autoload.php'; $filter = new PHP_CodeCoverage_Filter(); $filter->addDirectoryToBlacklist(__DIR__ . "/../vendor"); $filter->addDirectoryToWhitelist(__DIR__ . "/../src"); $coverage = new PHP_CodeCoverage(null, $filter); $coverage->start('Behat Test'); } // ... run your application here ... if ($calculateCoverage) { $coverage->stop(); $writer = new PHP_CodeCoverage_Report_PHP; $writer->process($coverage, __DIR__ . "/../log/behat-coverage/" . microtime(true) . ".cov"); }

At first the code detects if code coverage information should be gathered by checking if the file /tmp/generate-behat-coverage exists. You can touch and delete that one manually or from your test setup.

The next code block loads and initializes the code coverage collection, creates a filter for 3rd party code and starts the code coverage collection. After that, the comment indicates to run your application, which might e.g. be a Symfony2 kernel handling call.

The final lines write the code coverage information into a file for further processing. It will create a dedicated file for each request, where these files then need to be merged later.

Running Tests

With the shown code in place, you can trigger a Behat test run with code coverage using the following Ant code, for example:

<target name="behat-coverage" depends="clean, initialize"> <delete dir="${commons:logsdir}/behat-coverage" /> <mkdir dir="${commons:logsdir}/behat-coverage" /> <touch file="/tmp/generate-behat-coverage" /> <antcall target="behat" /> <delete file="/tmp/generate-behat-coverage" /> <exec executable="phpcov" failonerror="false" dir="${basedir}"> <arg value="--merge" /> <arg value="--html" /> <arg value="${commons:logsdir}/../coverage/behat" /> <arg value="${commons:logsdir}/behat-coverage" /> </exec> </target>

Get a Qafoo expert on-site to support you with mastering challenges in various areas of high quality code, including software architecture, design, automated testing and more.

The Ant target first cleans up code coverage from previous runs. It then touches the file that indicates to the application to run code coverage, executes Behat and removes the trigger file again. Then the phpcov utility is executed to merge the results of all requests into a single coverage report and generate HTML from it.

Conclusion

Code coverage is completely out of scope for acceptance tests. However, if you abuse Behat to create wide-coverage tests before refactoring, it might be of help to you to see what is still missing before you start hacking.

Comments