By Kore Nordmann, first published at Wed, 17 Jul 2013 08:07:58 +0200
Download our free e-book "Crafting Quality Software" with a selection of the finest blog posts as PDF or EPub.
You can also buy a printed version of the book on Amazon or on epubli.
Talking to interesting people spawns ideas and spreads insight knowledge. Therefore, I talked to Johann Peter Hartmann about testing culture and how PHP projects should approach testing in 2013. Johann is the CTO of Mayflower, one of Germany's top technology service providers, developing leading web solutions for customers like Deutsche Telekom AG, ProSiebenSat.1 Media AG and Vaillant GmbH.
Johann, are you happy that PHP can by now keep up with other languages in terms of quality? Especially since we, PHP people, were well known as security-ignorant spaghetti coders?
Of course. We started our first Test Driven project in 2005. That was highly experimental back then, based on SimpleTest and without anything comparable to Continuous Integration (CI). Since then, basically everything has changed. By now we have virtually no project that does not run unit- and acceptance tests, a CI and the complete range of code quality tools. And in combination with recent IDEs that sums up to a workflow that is pleasant on the one hand and does not thwart efficiency on the other hand.
We have ultimately reached a level where we can easily compete with languages like Java, Python or Ruby, in terms of reliability of the code. Even though we do it our own way and don’t play the simple copy-cat.
Why not as a copy-cat? Aren’t the unit test frameworks extremely similar?
Of course, in most cases they even provide the same APIs. But languages are different. And that does not only apply to the syntax, but especially to the style of usage and the culture from which the developers originate. Some years ago we did a survey on why architectural decisions are made in favor of PHP and not for DotNet, Java, Ruby or Python. And the answers – not surprising for us as PHP developers - were low costs and quick results, as well as the potential for rapid change. If you decide to use PHP, you don’t want to do a lot of upfront planning and wait 6 months for the first release. You want to specify less, to see results fast and be able to react with adaption easily.
As it happens, these requirements coincide with those of the internet, where they are even stronger, because that is where innovative software happens and you don’t know upfront if anybody really wants to use the application at all. On top, there is open source and a variety of solutions for typical problems, like content management or e-commerce - and this provides you with the universal language to solve the problems of the internet.
But that comes with a change of your testing strategy. While a classical offline application can live for several years with a stable unit test, an online core workflow might change every other month and that does not only impact the business logic, but requires changing the unit tests as well. That means, a test must pay off much faster and therefore the whole requirement for software quality changes.
Get individual workshops and trainings on-site for your team around the full stack of software quality in PHP and JavaScript based web applications.
What does that mean in particular? Different styles of tests, less tests or even no unit tests at all?
No unit tests at all, after all PHPNuke was the most successful online software for years and did not have any tests ;-). No, our users would most probably not excuse that anymore. Instead, it means that test efficiency plays a important role. If I only use a software long enough in the same or a similar way, I want to write a test that ensures a stable API and protects the stability of the component across refactorings. The top layers are then covered by acceptance and regression tests, which stay essentially stable. But if I assume that everything can change - including the basic business hypotheses - the situation looks somewhat different. I still want reliability, but not at the expense of productivity.
I know that every other change requires me to touch the test because the changing business requirements simply do not abide to my abstraction cuts. And I don’t want that a two line change in the production code results in 20 changed lines in the test code. In that case I would most probably introduce 10 times more bugs into my test code than into the real one. The consequence would be that I could only trust my tests halfway.
So I look for a combination of tests that suit my changes best. Where I expect many changes, the confidence in my software must be ensured by tests - and that possibly works best on a higher level, where I ensure core processes. For example using Behat or Jasmine. I stick to the same rule for the levels below - making my test work where I gain the lowest cost for change but in exchange the highest stability.
Does that mean I should not work test driven?
No, working Test Driven is excellent and I can only recommend it. That is the best insurance you can buy against badly designed code, messed up dependencies and confusing APIs. But it only protects the piece of code which is in development right now. It does not imply that the combination of all pieces is correct as well. You could have built a completely different building with the metal pieces of the Eiffel tower: one that must instantly collapse and is massively defective in its whole.
Therefore, I definitely need tests that outreach a single unit to ensure proper combination and cooperation of the component. But you can also work Test Driven from outside in, that means Behavior Driven Development. That does not give you clean APIs but ensures that components are playing well together.
Nevertheless, these tests must also be efficient in respect to change. That’s a problem for many test frameworks, especially Selenium. In the end, there is no golden rule - this one would almost inevitably lead to too few or insane tests - but the point is to find the sweet spot for your individual project; the unique combination which minimizes test waste but still reliably detects errors in your core workflows.
So, how did Qafoo help you to reach that goal?
Toby did a workshop on-site and was so kind to craft that one around our individual problems. “How do I write sensible tests? How do I test efficiently?”. We have been doing testing since 2005, we don’t need someone to explain the syntax or the functionality of mock objects to us. But we wanted to know how to test in a sensible way. Where the sweet spot is, where you don’t have too many tests and not too few. And how to approach that place.
That was exactly the point where the workshop hooked in and helped. It imparted experience and - even more important - created new experience for the attendees. That way we get a feeling which approach makes sense in which situation. How we need to craft architecture and design in order to significantly reduce the cost for tests. How to escape from the trap of the inner-platform antipattern, where we need to provide all infrastructure and business models with an ugly, mocked twin. Simply: How to efficiently write tests? And how to avoid that everyone hates them within 10 months, so that they would rather deactivate them instead of fixing them. And that is what Tobias managed to do excellently in his typical amusing style.
What can we do besides that in order to get a well-tuned, efficient test setup?
Coding Dojos! Please google that immediately and try it out with your team. That is, apart from such a workshop, the perfect way to gather own expert knowledge. Or to introduce Test Driven Development or to create a good intuition for pair programming.
Oh, and don’t believe in a silver bullet that solves all your quality issues.
Johann, thanks for sharing your insights with us.
Stay up to date with regular new technological insights by subscribing to our newsletter. We will send you articles to improve your developments skills.