I’m writing today to report on the landscape of testdays through 2012. These metrics are fairly unscientific, in that they are just the summation of numbers I’ve been plotting into a spreadsheet throughout the year. However I think they are somewhat indicative of a slowly growing community. I would first like to thank some people before going into the metrics.
I would like to thank David Burns for setting up an IRC bot to assist in the gathering of metrics throughout the year. This would have been a much more manual process had he not set this up.
I would also like to thank the many of you who came out to help during the testdays this year. The standouts in the community are Gabriela Montagu for participating in 21 testdays (more than anyone else), Aleksej for not only participating but stepping up to moderate and help guide some newcomers during many testdays, and Swarnava for reporting over 90 bugs during testdays (21 of which have now been fixed).
Now, on with the metrics…
Bugs triaged or reported during testdays in 2012 (+7%)
This first set of metrics measures the growth of “contributions” to testdays. It shows pretty good growth in the amount of Bugzilla traffic coming out of testdays for the first half of the year, but this unfortunately falls off later in the year. I’ve come to realize that looking at Bugzilla alone is a fairly narrow view of contribution. Many testdays include running test cases in MozTrap (or elsewhere), or checking in code to our automated test suites. I would like to find ways to better measure (and reward) the contributions which come out of testdays in 2013.
Volunteer attendance to testdays throughout 2012 (+130%!!!)
Paid and volunteer participation during testdays in 2012 (+13%)
These metrics tell a story that we made great strides in getting community involved in testdays; volunteer community participation is up 130% after a year. Unfortunately, once you factor in participation by paid contributors, growth rate plummets to 13%. While I’m happy to see an increase in participation here, I think there is a lot of room for improvement. I will be working in 2013 to make testdays more accessible to all and perhaps more appealing to paid contributors.
Naturally, it’s great to see overall growth and I am truly thankful for the effort many of you have put in this year. However, there’s a lot of work to do — we need to do better. I will be working hard to continue to grow the community and make testdays more accessible to everyone through 2013. I have some new ideas I want to try out which will hopefully accomplish this goal.
Thank you, and see you in 2013!