Here is an outline of notes I have in regards to exploratory testing for new comers that want to QA the b2g project. I will probably get flack for this but oh what the heck. I’ll share my general guideline in exploratory testing:
- No right or wrong… perhaps “better” as in more efficient or results are more satisfying… or ZUL. I forget which.
- Be curious:
Ask How/Why questions alot:
– Why does the feature work the way it does?
– How does the feature work the way it does?
– How many ways into the feature are there?
– How many ways out of a feature are there?
Why : Chances are a dev or triage team will ask you the question in a bug of what happens if you go through a different way to get to the same feature… Short cut by testing and placing in notes what does and doesn’t work; and provide as much info so the dev could try to find a fix in a faster amount of time.
Example of Dev asking for more details : https://wiki.mozilla.org/Gaia/Email/RequiredBugInfo
- Bugs occur when :
– There’s an oversight
– other people coding
– forgetting a case
– There’s a mistake
– simple syntax
– logic flow, etc.
- Get to understanding how things work
– Understanding the architecture, how things are built, how they are built… what they are capable of, what the limitations are… what design pattern they used in order to build… the more you understand and understand the context that the dev built it helps you find bugs faster/better. Better in that the dev will understand how to fix it faster. Being able to talk the same lingo helps out a lot as well.
- Model View Controller (MVC) : ( http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller )
– I use this model to test even though the software itself isn’t implemented this way:
– Reason: It’s easier to conceptualize
– UI (View)
– l10n bugs
– graphic icons
– glitches
– accessibility issues
– Data (Model)
– special edge cases such as 0, 1, null, etc
– Logic/functionality (Controller)
– Business logic (ie functionality issues)
– User Flow logic (ie Usability issues)
- Be aware of States :
– In comp sci: I was taught to comment pre, post, and expected results in a function to comment what a function does. It’s quite applicable in QA.
– in a micro sense we could potentially figure out various states of data going into a function and going out
– if you plot out what things you are doing and testing, and remember what you are doing in detail (ie what buttons, ui you touched) then chances are steps to reproduce (STR) are going to be easy to figure out.
– moving outwards, it still applies to functionality of a feature:
example :
– music app is turned on while dialer is running
– if the music app being turned on is a state different from nothing else running and you could potentially run into a bug.
– outlines help
– setting up test cases to a certain order helps
– caveat: test in various ways… don’t do the same thing the same way each time.
- Check the checked in code to see where to test:
– “Insanity: doing the same thing over and over again and expecting different results.” – Albert Einstein
– If nothing had changed in that part of the code, and the code doesn’t affect that area, and you have already tested that area… do you really expect to find a bug (easily)?
– looking at where the change takes place is most likely the easiest and fastest way to find bugs… and checking “all” ways of impact:
Example :
Note: the examples are not to call out the fixers of the bug; they are excellent folk… it’s more to show that you can find bugs faster.
- bug 820147 ( https://bugzilla.mozilla.org/show_bug.cgi?id=820147 )
- bug 813275 ( https://bugzilla.mozilla.org/show_bug.cgi?id=813275 )
- – Checkin webpages :
- l10n: https://hg.mozilla.org/gaia-l10n
- gaia: https://github.com/mozilla-b2g/gaia/commits/master
- gecko: https://hg.mozilla.org/releases/mozilla-b2g18/
- (you guys don’t have to worry so much .. ) UXbranch: https://github.com/gordonbrander/gaia
- Be curious about available tools to help make your job easier and you get done faster.
– if you ask the question : why isn’t there some sort of tool to do x task?
– you may want to check to see if there already is one.
- Automate if possible, esp test cases. Once a test case is run, it turns into regression testing. Granted there has been a lot of regression bugs occuring in this project (due to back end change and front end ui not accounting for that change) and things of that nature… generally speaking regression bugs should be a lot fewer in most other projects that don’t have back end changes happening too often. The time it takes to do regression testing is time that could be done doing exploratory testing and uncovering new ground/more bugs.
– If you can’t automate, well… that sucks. do it manually.
[As a background, I’m a comp sci major from Univ. of Michigan with ~ 6 years of QA experience and ~ 9 years in tech support. 4 of my QA experience was spent learning from a 15 year exploratory QA veteran & 14 year automation QA veteran. They were my mentors for the first year I started QA and have learned quite a lot from them.]
