Future of Bugdays: Accessibility, Productivity, and Recognition
Anthony Hughes
I’m back from an exciting week at the Mozilla Summit 2010 in Whistler, BC. It was a whirlwind — frankly, I’m still a bit overwhelmed by it all. My brain is still digesting many of the talks and tech demos. I’d like to thank everyone who came out and contributed ideas during my Bugday Brainstorm breakout session. I received a lot of excellent feedback and ideas from various contributors (testers, localizers, developers, and MozQA Engineers).
Organizing this brainstorm session was the second phase of “making bugdays better”. We need to make bugdays more accessible and productive, and give contributors more recognition. The next phase is to try out as many of the ideas and see what works. I plan to “experiment” over the next few months with as many ideas as possible. There will most certainly be some failures, but there will also be some successes. Going into the following quarter, we’ll do more of what works and less of what does not.
At this point, I’d like to summarize some of the ideas that came out of the brainstorm session and the discussion leading up to it.
Bugday Format
One point of order was that there is some breakage in the current format of Bugdays. Meaning, bugdays which cover broad topics are both ineffective for Mozilla and daunting for contributors. There is also a major flaw in the “availability” of bugdays. They exist in a small 8-hour PDT bi-monthly window. While this is ideal for North American contributors, this essentially closes the door on many European and Asian contributors. Going forward, we need to try out some ideas which will increase global accessibility to these events.
Communication
We also discussed communication; both the medium and the message. Speaking to the message, we do a good job of communicating what we want to accomplish, but not how or why. We also do a good job of communicating on various channels, however they are really North-America-centric and Mozilla-centric. To this end, we need to communicate a better message of learning, doing, having fun, getting recognized, and seeing how contributions benefit Firefox; and communicate that message on a broader, more global, set of communication channels.
Stakeholders
Probably the most important point of discussion was about identifying the stakeholders of bugdays. I don’t believe we identified ALL of our stakeholders, but I believe we were able to identify some of the most important stakeholders. At a very high level, they are Firefox, QA, Project Managers, Developers, and Testers (new and old). Of course, every stakeholder has needs, and this was an important part of this discussion as well. Basically, we identified the needs as being the reduction and validation of bugs, reduction of noise, fostering new and more productive relationships through education and tasks, and effective/timely/correct response to bugs.
Success
Finally, we discussed the success of a bugday; both in terms of metrics and contributing factors. In terms of metrics, we found that the percentage of bugs acted on is a useful metric. However, the number of active testers (total number and number of returning contributors) is also very useful. In terms of contributing factors, we identified several: broad coverage of test environments, recognition of contribution, representation of all stakeholders, and giving new testers small targeted tasks.
What’s Next?
So what do we do now? We have identified several issues, identified several improvements, and gathered a lot of great ideas. There are three common threads I’ve discovered strung through these discussions. Our bugdays are not globally accessible, they need to be more productive, and we do not do a good enough job of recognizing the efforts of our contributors. Going forward, here are some ideas I’d like to try:
1. Span bugdays over a longer period of time, perhaps a week in duration.
2. Have a primary driver for each bugday in each major geographic region. At the very least, I’d like to see one person in North America, Europe, and Asia driving each bugday.
3. Increase the frequency of Bugdays by creating a “revolving-door”. In other words, as one bugday ends, another one begins.
4. Make bugdays far more specific and targeted; make them about accomplishing smaller tasks toward a goal. Perhaps tiered tasks, goals, and rewards.
5. Have some bugdays about learning to do and teaching the methods we use to test and triage
6. Record bug numbers and contributor numbers in finer detail
7. Collect contributor data (platform, hardware, experience, locale, etc)
8. Find ways to recognize contributions and make that recognition more publicly visible
9. Reach out to participants in all the stakeholder groups
10. Post to social networks which are popular in other locales
11. Post to community forums for non-Mozilla development (web-apps, flash, games, video, etc)
12. Engage students through the Campus Reps program
13. Create a bugday forum on QMO
14. Proactively engage Mozilla developers to meet their needs and to solicit participation
15. Solicit more engagement from the QA team
16. Define a specific set of quarterly goals for “Bugdays”. In other words, treat it like any other project.
As I said earlier, I don’t expect all of these ideas to be successful. I’m a scientist at heart. I enjoy the experimentation process. I’m really looking forward to looking back on this blog post in a few months to see how far we have come and what we have learned.
Finally, I’d like to make a point of saying this does not signal the end of the discussion. I’ll be starting a thread, somewhere, in the near future to continue this discussion. I hope this is just the beginning. Over time, I expect to discuss the successes and failures, and to gather more ideas. Once I’ve set up the discussion thread, I will blog about it here. Stay tuned to my blog or planet.
Thanks everyone for your contributions.
