Some people wonder how I find bugs. It’s not just a matter of running a test case, but to test around the bug. (I explained how to test around a bug in an earlier blog). Thinking of each of these items as a state, and running various permutations in which is likely to find bugs is what I do. It is potentially possible to automate this type of testing, when I stopped to think about it. It’s similar to the test fuzzing, but just moving steps around and such. Someone would still have to run a verification on the fuzz tests, though. Anyways…
I posted this previously on my blog with little explanation about this list. There are one off bugs within Fennec that you can do from a black box perspective. (ie not knowing anything about how things are coded). Here, I’d like to take the time out to show that one small action can make the difference between finding/reproducing a bug and missing a bug. Below is a list of things that you can try to test around a bug. ie, follow the steps that you would do + try one other different thing such as rotate the device or place one of these steps instead or insert the step before another step. Try combining it with the test web page list that I have in a previous entry.
I don’t look at just what the target of the tests are; I look at the whole device from start to finish while I running the tests. Why? Because even though I am testing to verify the tests, something else may occur in the meantime while running the tests. It has to do with Inattentional blindness which is best explained ( http://www.psychologytoday.com/blog/and-all-jazz/201009/must-read-the-invisible-gorilla ) The more you can remove your inattentional blindness, the more you can find bugs. Cognitive behavior plays a huge part in designs for ui as well. It’s good to know the science behind the ui/artwork. I admire good ui designers. Mozilla is very fortunate to have talented ui designers like Madhava and iBarlow. (Ya, I like saying iBarlow… though he isn’t a product of Apple). 
But anyhow that’s a side story.
One off bugs:
– Move to SDcard; stored on device *
– change orientations *
– difference between URLbar text typing and text field typing *
– URLbar is an android widget and will show difference in behavior
– different VKBs : simeji (double byte/commit), swype, swiftkey, default keyboard
– these vkbs have shown differences
– chrome versus content *
– flash versus webm *
– iframes *
– viewport versus non-viewport
– global history versus imported history *
– Sync *
– CPU instruction set : omap vs neon vs tegra *
– Arm version (6 versus 7) *
– Android OS versions 2.2, 2.3, 3.0, 4.0 *
– Hardware keyboard vs VKB (virtual keyboards)
– RTL
– double byte characters
– international date/time
– new tab versus current tab *
– local page (about: pages) versus remote pages (other urls) *
– uri (file:// versus http versus youtube versus https ) *
– web gl *
– quit * vs force quit (quitting will not restore the tab, force quit will restore the tab)
* asterisked are some actions that may make a difference in terms of crashing.
TIPS : When you try to go to a web site, try one of these small things and try to pay attention to what you do when you are testing. Sometimes it’s hard to remember every thing you do, I tend to either reboot my device or quit and restart fennec to get me to a known state.
When I do find a bug, I try to reproduce the bug right away; but I restart the fennec and device to verify that I can reproduce from scratch. Once I find the bug, I sometimes try to see what I can remove out of the steps to try to get to the shortest path to the bug.
Other information that helps are logcat, as well as crash ids from about:crashes, screenshots, and any other information you can provide such as version number and mobile device name.
Also other things I do is to try to see if the bug already exists, etc. etc. It’s not always easy to do all these things and at times I do fail to find duplicates, or another thing listed in here. I try the best that I can in the given amount of time I have. Hopefully these tips will help you find bugs within fennec and help us improve the product! Good luck and hope to read your bugs in bugzilla!
Filed under: mobile, Planet, QMO, Uncategorized 
