I would like to first start by thanking Joel Maher and Geoff Brown for all the help they offered me and my team with creating Robocop tests.
Who am I and what I work on for Mozilla?
My name is Adrian Tamas and I am a member of the Softvision Mozilla Mobile team based in Cluj-Napoca, Romania. We are a team made up of 6 QA Engineers tasked with testing Firefox for Android. Our tasks include build validation, bug triage, investigation and verification, feature testing, test case creation and lately Robocop test case creation
What is Robocop?
Robocop is a Testing Framework created by Mozilla automation team used to test Firefox for Android. Robocop is based on the Robotium Test Framework and offers high level methods used to interact with the Firefox for Android UI.
Each Robocop test covers a full feature of Firefox for Android so the number of tests that are executed when running the Robocop testsuite is much larger then the actual number of tests.
For more information on Robocop, how to setup the test development environment and to wright tests you can check the Robocop MDN page.
How it started and where are we now?
I started in October of 2012 searching wiki pages and mdn pages for information on setting up the environment and how to wright/run Robocop tests. The information was scattered across multiple documents and not always up to date so I grouped it up in a ramp-up documentation for the rest of the team which evolved later into the Robocop MDN page.
When I started there were around 20 Robocop tests. Now the number has grown to 40 tests running on every branch of Mozilla Mobile. At this time we managed to cover every feature that can be automated for the Smoke testsuite. The team has created tests covering the functionalities:
- Add-ons Manager – Bug 777719
- Clear history – Bug 836341
- Awesomebar History Tab – Bug 777719
- Edit Bookmarks – Bug 817970
- Remove Bookmarks from the Context Menu – Bug 817970
- Settings Menu items – Bug 830755
- Detect device type and device OS – Bug 818390
- Back, Forward and Reload functionalities – Bug 747835
- Test doorhangers (password, geolocation and store offline data doorhangers) – Bug 732212
- Sharing Links – Bug 826261 and Bug 841377
- System Pages – Bug 822256
- Add a new Search Engine –Bug 846296
- Import from Android – Bug 833832
- Web Content Context Menu – Bug 846257
- Find in page – Bug 822259
What is next?
We will continue to create automated tests trying to cover all the features offered by Firefox for Android and also help to evolve Robocop by adding more methods that can be used reliably to do general actions. We have also started to look into intermittent fails of the Robocop tests and try to fix them in order to get the Robocop runs more reliable
Also recently a new feature has landed allowing the creation of JS tests using Robocop – Bug 870908 – which we will be looking at in the future to extend the number of MozTrap tests we can cover using Robocop