Joomla testing – With a little help from your friends

Testing software can be an arduous task. Every web environment comes with it's own PHP version, server configuration and quirks. For a project as big as Joomla, this can quickly become a problem.

Each new Joomla release is deployed on thousands of different environments around the world in a matter of hours. Some bugs might go unnoticed until end-users install on PHP versions that haven’t been tested. This can lead to a frustrating time for everyone involved.

Crowd sourced pre-release testing

Beginning of the year the Joomla project adopted a crowd sourced pre-release testing workflow, after problems with releases end of 2013.

“When a new version of Joomla is released, we sometimes run into a problem where not enough people test the release on real servers. Recently, this has led to some chaotic, back-to-back releases which causes stress everyone: end users, developers, and Joomla contributors.”

The adoption of a crowd-sourced testing flow is a great idea. However, the workflow is only as good as the tools used. At Joomlatools, the idea stuck and our team was keen to see if we could help out to make things easier for testers.

Putting things together

We already have a virtual site playground thanks to our Joomla in a box project. We just had to figure out a way to make it possible to install different versions of Joomla and PHP. Any good developer likes a challenge and so do we!

Installing Joomla versions was not that hard a nut to crack. To solve this we build a Joomla command line interface for our Joomla box that allows fetching any Joomla CMS branch or tag directly from GitHub.

Finding a way to easily and quickly install different PHP versions on the box posed to be a little bit harder. The solution : ‘phpmanager’, a custom script that can build different PHP versions and setup all the required modules.

Making testing easy and fun

With a little bit of help from awesome open source tools like Virtualbox, Vagrant and a copy of our Joomlatools Vagrant box anyone can now help test Joomla release before they go out, in true ‘Jumla’ style!

All these goodies are part of the 1.2 release of our Joomla Vagrant box, which has been released today. Just in time for the Joomladagen in the Netherlands next weekend. Johan will present the 1.2 version on Sunday and show you how to use it. Robin, Tom and Johan will also be participating in the Joomla Pizza Bugs and Fun event on Saturday! Need help setting up your own box, go and find them!

We love our little toolbox and we’re already using it extensively for our own extension development. We hope the Joomla community and testers find it useful as well. Ideas, suggestions, comments or patches are very welcome!

Give it a try right away by cloning our GitHub repository and ... happy Joomla testing!