Part 1: UI Testing on Firefox OS using Marionette; a background


by Zac Campbell

While the main B2G team at Mozilla is developing Firefox OS, our support teams, Dev Tools, and the A-team are working hard to build the tools and framework to develop and test HTML5 applications.

Building these tools and framework are important to support not only the development of Firefox OS itself, but the apps that the Mozilla community will develop. We want solid, reliable apps that users will love and to support them we need reliable test tools.

On the Web QA team we are using these tools to write automated tests to cover the apps that are to be delivered with Firefox OS.

We’re doing this for the same reasons that we always do test automation like this; provide consistent, repeatable tests against daily builds and to spare the wear and tear on our manual testers’ fingers!

For automated UI testing of Firefox OS, we use a tool called Marionette. This has been developed by Mozilla’s A-team to enable us to run tests against Firefox OS.

Marionette allows us to control aspects of the user interface, for example clicking icons, keying in text and pressing buttons. Just a few quick examples of tests we have written so far are to take a photo using the camera, send and receive SMS messages, and play an MP3.

It’s based on the W3C WebDriver specification, so if you’re familiar with Selenium and WebDriver then you’ll have no problems adapting to Marionette and Firefox OS.

We’ve connected these tests to our Jenkins, some slaves and real phones. The tests really do make calls, send SMS and play music!

Enough about how we’re using it; over the next few weeks the Web QA team will publish a series of blog posts explaining the differences with testing on Firefox OS. Although they are HTML5 apps, their design and structure are different and so is the OS they’re running on. We’ll talk you through creating reliable tests and using the power of the WebAPIs.

You can learn from our experience to give you a head start when the time comes to write tests for your own app.

In part 2 we’ll have a look at how Firefox OS uses <iframe> for running apps and how we use Marionette to switch between the context of apps and the operating system.