It seems that my recent post on requesting retest on NEW bugs showed a divide in the community. There are certain community members who agree that old bugs should be pinged, while others think that it is an annoyance rather than a help. And I agree it is a fine line that needs to be walked. There are pros and cons, and I think we as a community need to determine where to draw the line. Closing bugs as INCOMPLETE hurts, but it helps too.
Cons of closing bugs:
- It can be annoying. Getting thousands of bugspam emails at once is pretty annoying (I know, I get them too). And if your bug has slipped into a query it can be annoying to be asked if you’ve reproduced a bug in a newer version of Firefox when you already have. It happens.
- It can appear that Mozilla does not care about users and their Bugs. When a user files a bug in 2006, and it never gets touched until a comment is placed on it in 2011 asking the user to test again in the most recent Firefox version, reporters can understandably feel unappreciated.
- Real bugs can be closed in the process. In closing bugs, we run a slight chance of accidentally closing a bug that is an edge case and has not had any dupes filed.
Pros of closing bugs:
- It assists with finding bugs that will actually be fixed. If a developer has a list of 1000 bugs covering multiple versions of Firefox, it is a bit hard to find a bug with enough information to fix. However, if Triage has gone through that list, weeded out the ones that do not have active reporters anymore and closed them, that list could potentially go down to 300 bugs. All 300 with recent comments from their reporters or other users who have reproduced the bug, giving recent relevant information. A list of 300 up-to-date bugs is much better than 1000 of varying quality.
- It enlists the help of reporters to reproduce bugs. There are simply too many bugs for the Triage team and QA team to reproduce all of them at once. By asking reporters to reproduce, we can help enlist our community more.
- We have nothing to lose. These bugs are sitting there stagnate anyway. Nothing will change if we don’t go after them. If we do comment on the bugs, the chance that we will get valuable information is significantly higher than if they just sat. We have nothing to lose, and plenty to gain.
Now, there are some ways we can improve:
- Write better follow up comments. Possibly, something like David Eaves suggested in his blog post. Prettified emails. Perhaps Bugzilla could be modified to send an mmail saying “Since you reported this bug, two new versions of Firefox have been released, with many changes in x y z. You can read the release notes of these updates at mozilla.com. The Mozilla community would appreciate it if you would take the time to retest your bug XXXXXX with the latest version using a fresh profile. And update your bug accordingly.” Or something like. I’m not sure how feasible this is, but we definitely need to make our bug replies more friendly.
- Be better at following up ourselves. I mean come on, we ask the reporter to follow up on their bug, then we as Mozilla let the bug sit, again. It is disheartening to a reporter to be like “They asked me to follow up, maybe they are finally going to fix the darn thing.” Then it sits, again, for years because as a community we dropped the ball. I think that is what is more of a discouragement to our community of bug reporters. That they report a bug that never gets fixed. They spent time that they didn’t have too creating a bugzilla account, reporting a bug, retesting a bug (perhaps multiple times) then it sits. We really really need to pick up the game here. And start to focus on our community a little more.