Background
If there’s one thing that Firefox Input has taught Mozilla, its that our community is really split up into two types of folks: passive and active. The latter are contributors. They’re our SUMO, QA, Marketing, Developer, Creative and Campus Rep volunteers who make Mozilla run. We know about them and we try their best to make sure they’re well taken care of and offered opportunities that they’ve rightly earned. The other type are passive community members. These are people who follow the things we do here at Mozilla and have opinions about the actions taken and usually only go so far as to reply to e-mails within newsgroups or file a bug or two. They’re not contributors in this definition, but they definitely are stakeholders.
Mozilla, historically, has been modestly successful at fulfilling the needs of our active community members, but have failed repeatedly in finding the right approaches to give opportunities to our passive community members. Input succeeded there; it has been able to collect close to 2 Million pieces of feedback over Firefox 4′s development cycle and those pieces of feedback have resulted in numerous bugs filed and **much** greater insight and understanding into what our passive community thinks of the actions we perform to help make Internet better.
The thing is, we can do better. Input was a late-breaking feature and something that had no semblance of typical strategic plans. There were plenty of things the team wanted to get accomplished, but neither had the time nor manpower to do it. Fortunately, we have it now and we’re planning to break the bank to show the world the real power of user feedback as a strategy for product development. The following post will outline what we plan to execute on over the next 3-6 months.
Objectives
- Evolve User Feedback into a platform for product and community development.
- Support all channels of Firefox’s development process.
- Marry Input with Big Data approaches.
Strategies and Outcomes
- Build an API and work with existing Mozilla’s product services to create features that integrate with Input’s feedback database. We’ve received buy-in for the following outcomes we plan to implement within Input and other services in Mozilla.
- Product Drivers – Input will translate spikes (i.e. 1-2 magnitude’s higher than the average over a version) over specific search terms and URLs into a newly filed bug in Bugzilla. The tool would most likely poll for data over a manually-set list of terms.
- SUMO – Support volunteers will be able to find Input’s clustered opinions that are similar to support thread topics using an easy-to-use button per thread.
- RRRT – Input will offer triagers the option to correlate messages per version within “broken”, “sad” and “ideas” feedback types with Bugzilla bug summaries to find matches. (and possibly increment “votes”). Just like SUMO, crash-stats triagers will be able to correlate messages and URLs per version specific feedback types with comments and URLs in crash-stats’ reports over the same version.
- AMO – Add-ons developers should be able to retrieve feedback for their Add-on’s name over the “happy”, “sad”, “ideas” (release and beta versions) feedback types using a simple “see feedback” button in the Developer Hub. Another option will be to create a running list of top suggestions by users that could be solved by the development of add-ons within AMO.
- Develop submission support for Nightly users and a dashboard that supports the new channels in Firefox’s development cycle. Here is a list of what each feedback type (within each channel) will allow our community to give us:
- Nightlies – Users on this channel are likelier to file bugs into Bugzilla and are adept at understanding regressed issues. Why force them to search through Bugzilla to find out their issue is already filed and lose that piece of additional regression information (i.e. environment, steps to reproduce and additional occurrences)? The Input team will add support to nightly users to file reports to our dashboard allowing our active community members to turn them into bugs or add additional information into already filed ones.
- File a Report – Help track down regressions and possible stability/performance concerns by filing simple reports that allow us to cluster the information.
- Experimental – The purpose of this channel is to determine the future of whether features will be added to a release. The crowd is still rather technical, but more likely to give UX or feature related feedback rather than regressions. Therefore, we’ll need to use focus group type of data to better understand their viability. Fortunately, we’ll be able to use the same happy/sad/issues feedback that we used in the betas and continue it here.
- Praise – Tell the development team what they’re doing right on specific features product characteristics.
- Issues – Tell the development team about specific problems/bugs/concerns you’re facing with new features that are offered on your experimental build.
- Ideas – Offer Ideas on how new features, listed over whats new/firstrun/Input pages, can better fit what you would expect out of that feature.
- Beta – The purpose of this channel is to determine the stability, performance and quality criteria is met in order to push a build to release. The crowd is still less technical, but still likely to give us enough constructive feedback to better aid our development team. We’ll continue to use the same happy/sad/issues feedback here as well.
- Praise – Tell the development team what they’re doing right on specific features product characteristics.
- Issues – Tell the development about specific problems/bugs/concerns you’re facing with new features that are offered on your beta build.
- Share an Idea – Offer your suggestion on how Firefox could be better.
- Release – Users are really just looking for a good browser to use rather than helping to build one. Therefore, we’ll use some higher level and deeper niche pieces of user feedback that offer support for these users to give us feedback in a way that will be constructive to the development team.
- Rate your Experience – Give us a quick and easy rating on how we are doing over key aspects of Firefox ranging from start-up time to features
- Report a Site Issue – Report issues on websites looking and behaving incorrectly on Firefox. We look for concise details here.
- Share an Idea -Offer your suggestions on how Firefox could be better.
- Build world-class, open-source text clustering services and data modules.
- Scale to cluster over hundreds of millions of pieces of feedback. Actually, we’re already doing this.
- Retrieve feedback per locale and feedback type (i.e. “happy” and “sad”) per beta version to create a world map visualization of user sentiment per locale
- Experiment with locale-based feedback to create possible stop-words for multi-language clustering
- Run Open Data experiments to our community and educational institutions to gain better insight into commonly asked questions about our userbase and the development of Firefox.