Exploratory Testing : Getting Started
This post is for those of you who want to start helping out- but aren’t familiar with exploratory testing. Exploratory testing is a great way to get introduced to doing QA.
Let’s start with this QMO job board posting for exploratory testing on Moztrap:
https://quality.mozilla.org/2012/08/webqajobboard1/
The purpose of exploratory testing is to find bugs. It’s a process of *exploring* the app as a user might. The term ‘exploratory testing’ has come to mean different things to different people. In this case it is a test approach where the tester seeks to find out how the software works, and asks questions on behalf of the users. The goal is to find bugs where the software does not work as expected, or does not work as a user might expect it to.
Let’s follow the job board posting to see what we can find!
Testing
* Most of Mozilla’s websites ask you to log in, in order to use the tools. You will need a Persona account to log in. You only need to create the account once, and then you can use it to log in wherever Persona is found. Create a Persona account.
* You can choose to run Moztrap Exploratory tests directly by clicking this link. NOTE: The build info is not important, fill in with anything you want: https://moztrap.mozilla.org/runtests/environment/2529/
1) Go to the URL for the Moztrap Staging environment: https://moztrap.allizom.org/manage/cases
Next we can dive into some exploratory testing. There are no strict rules to follow. Use the site as you think a user might. Try to find something. Click around to different pages or links. Keep your eyes open- is everything displayed correctly? Are you able to find what you are looking for?
2) Click different tabs and links such as:
- Click the arrow next to a test case to expand it
- Run searches – how is it different when searching by clicking enter vs. selecting name/tag/product?
- View the site on different browsers- is the display very different on Firefox vs. Safari?
- Click the Advance filtering button and run some advanced searches – are the results correct? Is it quick enough?
- Click on ‘Runs’ and try sorting the different columns of data
- Click ‘View Results’ and do some more exploring!
What next? The list above is not an exhaustive list of tests- it’s just a sample to give you some ideas of what to try. Can you see how this type of exploratory testing might be done on any website?
So far we’ve tested the site as a Guest. The next step will be to use the site as an authorized user.
3) Click the ‘Sign In’ link. You must sign in using Persona. If you do not have a Persona account, then you will need to create one. It is important to see how we are not testing Persona, but still use it to sign into MozTrap!
4) After Signing in to MozTrap there are a lot of options.
- Repeat the tests above- see how the permissions are different for an authorized user. Is the main behavior the same, or different?
- View Results – View results, sort column data, run some searches with and without advanced filtering
- Run Tests. This is one feature you can only use while signed in. Select the Product, Version and Run – and run some tests. Try passing a test case, or failing it. What about marking it invalid?
- Try these tests on a variety of Browsers and Operating Systems. Does it all work as expected?
There are a variety of permissions available to users of MozTrap. These tests cover the most basic role. There are many more features available with other roles.
If you would like to test some of those, you’ll need to talk to a MozTrap admin. This is easily done on IRC! You can find members of Web QA in the #mozwebqa IRC channel. Look at the job board posting to see the names of specific people who can help you with this task, or just ask anyone in the channel. If you don’t have an IRC client like LimeChat, mIRC, XChat, etc. you can click on this link: irc://irc.mozilla.org/#mozwebqa
The advanced features of MozTrap include such things as:
- Manage: Create a new testcase, edit a test case
- Creating new products, runs and suites of data
- Managing tags
- Editing or Deleting data
Filing a Bug
What happens when you find a bug? If you feel you have a bug that you would like to file, the next step is to open Bugzilla. You will need to sign in or create an account and sign in. The good part is that you only need to sign up once! After you are logged in, you can easily file a MozTrap bug by clicking or copying this link:
https://bugzilla.mozilla.org/enter_bug.cgi?product=Mozilla%20QA
Filing a bug requires entering as much data as you can! Select ‘MozTrap’ in the Product. Add a summary of the bug. Then enter a description of the issue, including reproduction steps. Be sure to add the ‘expected result’ of what you thought would happen and compare it to the ‘actual result’ of what really did happen. Feel free to add screenshots or video to help illustrate the issue. The click the ‘Submit Bug’ button and you are done!
Exploratory testing doesn’t follow a particular set of rules. Try entering too much data, or skip entering a field. Try to break the site by being creative! Of course there are product decisions that have been made already about how things should work- but we’d like to hear it if you disagree with them or if you have an idea how things might be improved. Finding out how users expected the application to work compared to what it does is a primary function of exploratory testing. You can always ask us questions if you aren’t sure if what you found should be entered as a bug.
Please let us know how your testing went. You may contact us in the ways listed above, or join the #moztrap channel in IRC. We’d love to hear from you!
Thanks for testing Moztrap! If you’d like to try another Exploratory test, click this link: