Why Regression Windows are so Important
Another great thing that QAs sometimes help developers with is finding regression windows for bugs. To be short, that is the day that the bug was introduced (the buggy code landed in the product). With this information we can take a look at the check-in list and, through the process of elimination, see what change/patch caused the problem. It’s always important to find the regression window because without it, a developer may apply a band-aid fix instead of finding the true source of the problem.
The Process of Finding a Regression Window
The work done to search for a regression window can be very quick if the person has been viewing the bug list for more than a couple of months. But, if you’re reading this page, then that probably means you haven’t been. So, I’ll just tell you right now not to get too frustrated if it’s taking a long time for you or you just aren’t able to find it. If you do become too flustered to really find it yourself, go over to the #qa channel on irc.mozilla.org and ask for some help! Anyway, without further ado, here’s is a general step-by-step approach to finding these windows:
- Download the previous version to the one from the bug from here and try to reproduce the issue on it (e.g. if the bug was found on Firefox 26, download Firefox 25). If the bug doesn’t reproduce on the older version, it means it regressed in the version where it was found. If it does reproduce, go back one version and try again. Keep doing this until you find a Firefox version without the bug.
- Let’s say you found version X has the bug and version X-1 doesn’t. That means the bug regressed in version X, i.e. it landed in Firefox X sometime during the time it was Nightly and up until it got released. Now you need to find the exact build it landed it. You can use MozRegression for this.
- Install MozRegression and use it the –good date, the date Firefox X got to Nightly (18 weeks before its release) and the –bad date, the date Firefox X got released. We know from the previous step that the bug appeared sometime between these two dates.
- MozRegression will compute the date that is the middle of the given time period and will download the build from that day. Try to reproduce the bug on that build and enter your results in the console. Good means the bug didn’t reproduce, while bad means it did reproduce.
- MozRegression will adjust the time period according to your results and will download another build. Try to reproduce the bug on it, then enter your results in the console again. Keep doing this until you receive a final result that will look like this:
Last good nightly: 2010-09-08 First bad nightly: 2010-09-09 Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=36f5cf6b2d42&tochange=8e0fce7d5b49
- The result you obtained points out the first build with the bug (first bad nightly) and the pushlog for that buid. One of the changes in the given pushlog is the cause for your bug. Post the result in the bug report and the developers will take it from there.