Volunteering: Regression Testing Automation

OpenC++ Core Lib has a set of unit tests that exercise particular functionalities of the library. Similarily, OpenC++ has a suite of regression tests to exercie the whole framework.

We are currently trying to support multiple configurations (Linux/gcc-2.95, Linux/gcc-3.x, Solaris/gcc-2.95, Solaris/SunWsPro, MacOSX/gcc-3.x, Windows/cygwin, Windows/VC++), but this becomes difficult, as developers usually have access to one platform only.

As far as possible we use SourceForge Compile Farm, but requiring developers to run tests on all platforms by hand after each check-in puts huge burden on them, which could be avoided by test automation.

What are we looking for?

• We would like to have a mechanism that within, say, 24h after each checkin finds what branches were affected, checks them out, runs the regression tests on available CF platforms and e-mails developer either with confirmation or with error report.
• We would like to have the information about which branches pass tests to be available on the website.
• We would like the system to be gentle with Compile Farm, e.g. do not run multiple tests at once on the same machine.
• We would like to be able to keep track on what tests were run, on which machine, when and why.
• We would like to be able to extend this mechanism later, so that people can input information about test status on platform that cannot be automated on Compile Farm (e.g. Windows/VC++).

We are aware that implementations of similar functionality exist (e.g. bonsai/tinderbox), but we don't know an easy way to make them work for us. If you know how to do it, please contribute.

For more info send an e-mail to Grzegorz Jakacki. If you don't get any answer within 3 working days, please resend.

[Back to volunteering page.]