Contributor spotlight: Swarnava

Mozilla, and specifically SUMO, relies on the talents of our contributors to make a difference. This month we’d like to give a big shout out to one of our Top Ten Contributors: Swarnava Sengupta!

Swarnava has provided close to 2800 answers in our forums- that’s a lot of help! Even better, most of those are ‘first responses’- which are the crucial first steps to giving our users the help they need. Even more impressive is that he’s done all of this and entered the Top Ten in under one year.

He also helps translate SUMO Knowledge Base articles in the Bengali language. He hopes to become a forum moderator someday, but is already the Bengali Locale Leader.

User questions come for all types of Firefox problems- and Swarnava specializes in crash related issues and browser hijacking problems. His advice to new contributors is that you don’t need to be an expert to help- which is so true! He points out that there is no need to feel shy, as he isn’t a developer or an expert. It’s just a matter of showing people Knowledge Base articles that help with their problem- that’s it. He has a really good point, as no one starts out as an expert-and the more help you give, the easier it gets.

Swarnava works from Kolkata, India. He puts in online 24/7 support via mobile & also uses Twitter to help with Army of Awesome. This sounds like a lot- but even with the big time difference between Mountain View, CA and Kolkata we see him online all the time! When not online helping Swarnava is in school. In addition to SUMO & Army of Awesome, Swarnava helps test beta products for Kaspersky Labs and Symantec Corporation. He maintains Bengali translations on Utorrent, Malwarebytes, and for some Firefox Addons. This is pretty impressive for someone who is 17 years old! He also chats with Friends using texts or Facebook. It’s not surprising that his favorite hobby is using the computer- especially as he aims to become a Computer Engineer. I’d say he’s well on his way to a successful career! If you’d like to hear more from Swarnava you can find him [online, of course] at:

facebook.com/heavengod

twitter.com/h34v3ng0d

Bug Verification Day

Hello Mozillians!

Wednesday, April 16th, we will be holding our weekly Bug Verification Day. This event is held on the #testday IRC channel and it’s addressed to everyone willing to get involved in improving Firefox.

You don’t need to have any previous experience in working with bugs. Ask for help on the #testday channel and someone there will give you assistance. More details are available in the wiki page we set up for you.

If you’re unable to attend these meetings and still want to get involved, you can verify bugs on your own time. Just add the [bugday-20140416] tag to the whiteboard or a comment for every bug you work on, so we know you participated to this event.

Join us and help make Firefox better!

Powerful tools for developing Web Apps

In the recent years, web development changed drastically. The emergence of the mobile web and the new form factor of smart phones created the demand for different solutions than the former desktop-only web.

Since then a lot of frameworks and tools have been created, with new ones being added almost weekly. Now, we web developers are faced with a different problem: for every development concern, there are multiple options to consider, without clear pros or cons. It is easy to feel intimidated not only by the choices available, but also by how similar those choices are.

Every day, web developers have to successfully overcome this issue and turn this diversity from a daunting proposition into an empowering one.

But why reinvent the wheel all the time?

A set of solid recommendations for app developers

Mozilla is putting together a core set of tools and recommendations that we believe are the most useful for making Web apps.

The key considerations for what we might recommend are:

  • Sufficiently well-documented and straightforward to use for an average developer (we will concisely document the required knowledge one needs in order to engage with the technology).
  • Loosely coupled and as modular as possible (so you can follow one recommendation but not another if you are so inclined).
  • Tested on Mozilla products (i.e. the UI components will perform well on Firefox OS, etc), but with cross-platform apps in mind

The upcoming, initial set of recommendations involves a toolchain that’s core to any modern web app, like a JavaScript framework, templating interface, UI framework and task runner. We will employ existing solutions wherever possible and write libraries or utilities to fill in the gaps.

On an ongoing basis, we’ll expand this systematically across the different parts of the development experience, such as offline handling or the use of various Web APIs.

All of this will be delivered in one central spot: The App Center on the Mozilla Developer Network (MDN), our established, renowned resource for web app development.

But I already have it all figured out!

We encourage you to share your success story with us. Heck, share your failures too. We need all the feedback we can get from everyday web developers. If you have a tool chain of your own, with your favorite JavaScript framework, etc., we’re not interested in converting you to something else. We’re trying to help developers who aren’t sure how to go about making these kinds of decisions.

Come join us!

In the Mozilla tradition, this is a community-driven process. This means your input is encouraged and appreciated, and we would like your help to make this a successful initiative!

The main discussions around this will happen in the following places:

For the initial tool chain recommendations, I started a thread on the mailing list already, go ahead and weigh in.

If you have ideas for topics worth exploring in future iterations, don’t be shy and open a new thread to get the discussion started.

What’s next?

If this whets your appetite, then great! 2014 is an exciting year to be a web app developer. We’ll keep you updated here on Hacks, as well as the MDN App Center over the coming months.

Bug Triage Day

Hi everyone!

Monday, April 14th we will be holding our weekly Bug Triage Day. Join us on the #testday IRC channel and get involved!

You don’t need to have any previous experience in working with bugs. Ask for help on #testday and someone there will offer you assistance. Details are also available in this event’s wiki page.

If you aren’t able to attend this event but still want to get involved, you can triage bugs on your own time.  Just add the [bugday-20140414] tag to the whiteboard or a comment for every bug you work on, so we know you participated to this event.

Join us and help make Firefox better!

Firefox 29 Beta 7 testday results

Hello everyone!

Friday, April the 11th, we held another Testday for Firefox 29 Beta 7. A big thank you is in order for all of you helping us running tests on multiple platforms, doing exploratory testing and updating bugs. More details about the work done can be found in the etherpad.

Special thanks go out to: florin_nechita, madamezou, tiziana, emerson, lakshmi, bsupreeth, sudharani, Meghraj, cbaba20, and to all our moderators! Your work is always appreciated.

We look forward to seeing you at the next Testday, so follow us on QMO for details!

Bug Verification Day

Hi everyone!

Wednesday, April 23rd, we will be holding our weekly Bug Verification Day. This event is held on the #testday IRC channel and it’s addressed to everyone willing to get involved in improving Firefox.

You don’t need to have any previous experience in working with bugs. Ask for help on the #testday channel and someone there will give you assistance. More details are available in the wiki page we set up for you.

If you’re unable to attend these meetings and still want to get involved, you can verify bugs on your own time. Just add the [bugday-20140423] tag to the whiteboard or a comment for every bug you work on, so we know you participated to this event.

Join us and help make Firefox better!

Firefox 29 Beta 7 Testday, April 11th

Greetings Firefox friends,

We are happy to announce that Friday,  April 11th, we are organizing the Firefox 29.0 Beta 7 Testday. We will be testing a new beta build with focus on recent changes and fixes. You can see all the details in this etherpad.

No previous testing experience is needed. Just join us on the #testday IRC channel and our moderators will offer you guidance and answer your questions.

Join us and let’s make Firefox better together!

Firefox 27 Bug Statistics

I’m writing today to present the bug statistics for Firefox 27. My apologies for the tardiness of this blog post; too many things have got in my way recently. I try to get these posts out at the end of life of the respective Firefox version as that allows me to present the statistics across the entire life-cycle of a Firefox version. For Firefox 27, this should have coincided with Firefox 28′s release a few weeks ago. Again, my apologies for getting this out later than usual.

The first story I want to tell is about the high-level breakdown of all tracked bug in this release. As you can see below there was a marked drop in the total bug volume in Firefox 27. Perhaps unsurprisingly this allowed us to focus a bit more which resulted in a smaller amount of unresolved and unconfirmed bugs being shipped in this release. The numbers are still much higher than we would like but it is a small victory for the overall quality of Firefox if these numbers continue to trend downward.

Firefox27_TotalBugs

The second story I want to tell is about the percentage of incoming bugs confirmed. This is typically an indication of the effectiveness of our incoming bug triage practices. As the volume of incoming bugs decreases we like to see the number of confirmed bugs increase. Unfortunately we have been trending the opposite direction for some time. Previously I had attributed this to the ever increasing volume of bugs but I can no longer rely on this excuse. Looking forward to Firefox 28 I can say that we’ve made remarkable improvement in this area in an effort to reverse this trend. I’ll share more on that in a few weeks.

Firefox27_Confirmed

The third story I’d like to share is that of when fixes landed for Firefox 27. The following chart I’ve plotted the average time-line for the past few releases along with Firefox 27′s time-line. In general we expect to see an ever increasing curve toward through the Nightly cycle, trailing off as we proceed through Aurora and Beta, with spikes in the first half of these cycles.

Firefox 27 appeared to be trending higher than average as we approached the end of each cycle. While these numbers are not completely out of control it does put a bit of extra strain on QA. After all, the later a fix lands, the less time we have to test it. Ultimately this creates risk to the quality of the product we ship, but as long as we recognize that we can try to plan for it accordingly.

Firefox27_Fixes-by-Date

The fourth story I want to tell is about the number of bugs reopened. We typically reopen a bug when something is fundamentally flawed with the initial implementation and/or if a patch needs to be backed out. Even in cases where a regression is found, we tend to leave the bug closed and deal with the regression in its own bug report. As such, a high volume of bugs being reopened is usually indicative of a release that saw much churn and may point to quality issues in release.

Unfortunately Firefox 27 continues the story of many of the version before it and represents a marginal increase in the number of bugs reopened. Of course, the other side of this story may be that testing was more effective. It’s hard to say concretely just looking at the bug numbers.

Firefox27_Reopened

The fifth story I want to tell is one of stability. The following chart shows the number of topcrash bugs reported against Firefox 27 as compared to previous releases. For those unaware, a topcrash bug are those crashes which show up most frequently in the wild and present the greatest risk to quality and security for our users. The unfortunate story for Firefox 27 is that we’ve seen an end to the downward trend that we saw started with Firefox 25 and continued with Firefox 26. The volume of topcrashes puts Firefox 27 in the same ballpark as the rash of point-releases we saw in Firefox’s teens.

Of course there’s two sides to every story. The other side of this may very well be that we got better at reporting stability issues and that resulted in a higher volume of known bugs. It’s hard to say for sure.

Firefox27_Topcrashes

The final story I want to tell today is about the percentage of regressions reported post-release. As we hone our processes, bring on more engineers, and get assistance from more contributors, we’ve been getting better at finding and fixing regressions. It’s inevitable that the more code landing in a release increases the potential for regression. Naturally this leads to an increase in the total number of regressions reported. Firefox 27 was no different so I thought I’d look at regressions a little differently this time around.

The following chart shows the ratio of regressions reported before release to regressions reported after release. A release with a high-volume of post-release regressions is a failure from a QA perspective because it means many bugs slipping through our fingers. I wouldn’t expect the number of post-release regressions to ever be 0 but we need to strive to always be better.

Firefox 27 represents a huge victory on this front. We saw a huge drop in the number of Firefox 27 regressions reported post-release. For months we’ve sought to improve our triage processes, engage more with developers, and work harder to involve volunteers in our day to day efforts. It’s nice to see these efforts finally paying off.

Firefox27_Regressions

That’s Firefox 27, in a nutshell, from a QA perspective. I think it’s useful to be able to reflect on the bug numbers and see what kind of an impact our efforts are having on the product. I really do enjoy visualizing the data and talking about our “victories”, but it’s just as interesting seeing what the data is telling us about where we may have failed. I believe that learning from failures has far more impact than building on successes and acts as a great motivator. What we want to avoid is those crippling failures. I think Firefox 27 is a nice iterative step forward.

Firefox 30 Aurora Testday, April 18th

Greetings mozillians,

We are happy to announce that Friday,  April 18th, we’re going to hold the Firefox 30.0 Aurora Testday. We will be testing the latest Aurora build, with focus on recent changes and fixes. Detailed instructions on how to get involved can be found in this etherpad.

No previous testing experience is required so feel free to join via #testday IRC channel and our moderators will offer you guidance and answer your questions.

Join us next Friday and let’s make Firefox better!

Google RMA, or how I finally got to use Firefox OS on Wind Mobile

The last 24 hours have really been quite an adventure in debugging. It all started last week when I decided to order a Nexus 5 from Google. It arrived yesterday, on time, and I couldn’t wait to get home to unbox it. Soon after unboxing my new Nexus 5 I would discover something was not well.

After setting up my Google account and syncing all my data I usually like to try out the camera. This did not go very well. I was immediately presented with a “Camera could not connect” error. After rebooting a couple times the error continued to persist.

I then went to the internet to research my problem and got the usual advice: clear the cache, force quit any unnecessary apps, or do a factory reset. Try as I might, all of these efforts would fail. I actually tried a factory reset three times and that’s where things got weirder.

On the third factory reset I decided to opt out of syncing my data and just try the camera with a completely stock install. However, this time the camera icon was completely missing. It was absent from my home screen and the app drawer. It was absent from the Gallery app. The only way I was able to get the Camera app to launch was to select the camera button on my lock screen.

Now that I finally got to the Camera app I noticed it had defaulted to the front camera, so naturally I tried to switch to the rear. However when I tried this, the icon to switch cameras was completely missing. I tried some third party camera apps but they would just crash on startup.

After a couple hours jumping through these hoops between factory resets I was about to give in. I gave it one last ditch effort and flashed the phone using Google’s stock Android 4.4 APK. It took me about another hour between getting my environment set up and getting the image flashed to the phone. However the result was the same: missing camera icons and crashing all over the place.

It was now past 1am, I had been at this for hours. I finally gave in and called up Google. They promptly sent me an RMA tag and I shipped the phone back to them this morning for a full refund. And so began the next day of my adventure.

I was now at a point where I had to decide what I wanted to do. Was I going to order another Nexus 5 and trust that one would be fine or would I save myself the hassle and just dig out an old Android phone I had lying around?

I remembered that I still had a Nexus S which was perfectly fine, albeit getting a bit slow. After a bit of research on MDN I decided to try flashing the Nexus S to use B2G. I had never successfully flashed any phone to B2G before and I thought yesterday’s events might have been pushing toward this moment.

I followed the documentation, checked out the source code, sat through the lengthy config and build process (this took about 2 hours), and pushed the bits to my phone. I then swapped in my SIM card and crossed my fingers. It worked! It seemed like magic, but it worked. I can again do all the things I want to: make phone calls, take pictures, check email, and tweet to my hearts content; all on a phone powered by the web.

I have to say the process was fairly painless (apart from the hours spent troubleshooting the Nexus 5). The only problem I encountered was a small hiccup in the config.sh process. Fortunately, I was able to work around this pretty easily thanks to Bugzilla. I can’t help but recognize my success was largely due to the excellent documentation provided by Mozilla and the work of developers, testers, and contributors alike who came before me.

I’ve found the process to be pretty rewarding. I built B2G, which I’ve never succeeded at before; I flashed my phone, which I’ve never succeeded at before; and I feel like I learned something new today.

I’ve been waiting a long time to be able to test B2G 1.4 on Wind Mobile, and now I can. Sure I’m sleep deprived, and sure it’s not an “official” Firefox OS phone, but that does not diminish the victory for me; not one bit.

 

Introducing PredictionIO

PredictionIO is an open source machine learning server for software developers to create predictive features, such as personalization, recommendation and content discovery. Building a production-grade engine to predict users’ preferences and personalize content for them used to be time-consuming. Not anymore with PredictionIO’s latest v0.7 release.

We are going to show you how PredictionIO streamlines the data process and make it friendly for developers and production deployment. A movie recommendation case will be used for illustration purpose. We want to offer “Top 10 Personalized Movie Recommendation” for each user. In addition, we also offer “If you like this movie, you may also like these 10 other movies….”.

Prerequisite

First, let’s explain a few terms we use in PredictionIO.

Apps

Apps in PredictionIO are not apps with program code. Instead, they correspond to the software application that is using PredictionIO. Apps can be treated as a logical separation between different sets of data. An App can have many Engines, but each Engine can only be associated with one App.

Engines

Engines are logical identities that an external application can interact with via the API. There are two types of Engines as of writing: item-recommendation, and item-similarity. Each Engine provide unique settings to meet different needs. An Engine may have only one deployed Algorithm at any time.

Algorithms

Algorithms are actual computation code that generates prediction models. Each Engine comes with a default, general purpose Algorithm. If you have any specific needs, you can swap the Algorithm with another one, and even fine tune its parameters.

Getting Hands-on

Preparing the Environment

Assuming a recent 64-bit Ubuntu Linux is installed, the first step is to install Java 7 and MongoDB. Java 7 can be installed by sudo apt-get install openjdk-7-jdk. MongoDB can be installed by following the steps described in Install MongoDB on Ubuntu.

Note: any recent 64-bit Linux should work. Distributions such as ArchLinux, CentOS, Debian, Fedora, Gentoo, Slackware, etc should all work fine with PredictionIO. Mac OS X is also supported.

Java 7 is required to run PredictionIO and its component Apache Hadoop. Apache Hadoop is optional since PredictionIO 0.7.0.

MongoDB 2.4.x is required for PredictionIO to read and write behavioral data and prediction model respectively.

Installing PredictionIO and its Components

To install PredictionIO, simply download a binary distribution and extract it. In this article, we assume that PredictionIO 0.7.0 has been installed at /opt/PredictionIO-0.7.0. When relative paths are used later in this article, they will be relative to this installation path unless otherwise stated.

The installation procedure is outlined in Installing PredictionIO on Linux.

To start and stop PredictionIO, you can use bin/start-all.sh and bin/stop-all.sh respectively.

Fine Tuning Apache Hadoop (Optional)

PredictionIO relies on Apache Hadoop for distributed data processing and storage. It is installed at vendors/hadoop-1.2.1. The following outlines some optimization opportunities.

vendors/hadoop-1.2.1/conf/hdfs-site.xml

dfs.name.dir and dfs.data.dir can be set to point at a big and persistent storage. Default values usually point at a temporary storage (usually /tmp), which could be pruned by the OS periodically.

vendors/hadoop-1.2.1/conf/mapred-site.xml

mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum control the maximum number of mapper and reducer jobs (data processing jobs). It is a good start to set the first value to be the number of CPU cores, and the second value to be half the first value. These should be reduced slightly to provide margin when you serve prediction in production settings if you do not run Hadoop on a separate machine.

mapred.child.java.opts controls the Java Virtual Machine options for each mapper and reducer job. It usually is a good idea to reserve a good amount of memory even when all mapper and reducer jobs are running. If you have a maximum of 4 mappers and 2 reducers, a 1GB heap space (-Xmx1g) for each of them (a total of 6GB max) would be reasonable if your machine has more than 10GB of RAM.

io.sort.mb controls the size of the internal sort buffer and is usually set to half of the child’s heap space. If you have set to 1GB above, this could be set to 512MB.

Creating an App in PredictionIO

This can be done by logging in PredictionIO’s web UI located at port 9000 on the server.

The following is the first screen after logging in and clicking on the “Add an App” button. To add an app, simply input the app’s name and click “Add”.

Importing Data to PredictionIO

Before importing data, the app key of the target app must be obtained. This was done by clicking on “Develop” next to an app. The screen below is what can be seen afterwards.

The app key for the “ml100k” app is NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM, as shown above. The app key is unique and your app key should be different.

To parse the MovieLens 100k data set and import to PredictionIO, one can use the import_ml.rb Gist. It requires the PredictionIO Ruby gem, which can be installed by sudo gem install predictionio.

Importing the whole set of data is simple:

ruby import_ml.rb NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM u.data

u.data is a file from the MovieLens 100k data set, which can be obtained from the GroupLens web site.

Adding Engines

Two engines can be added by simply clicking on the “Add an Engine” button. In our case, we have added “itemrec” and “itemsim” engines. Once they are added, they will commence training automatically with a default hourly schedule.

Getting Prediction Results

At the moment, you may access prediction results from these two sample URLs (API server at port 8000):

“Top 10 Personalized Movie Recommendation”:

http://localhost:8000/engines/itemrec/itemrec/topn.json?pio_appkey=NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM&pio_n=10&pio_uid=1

“If you like this movie, you may also like these 10 other movies….”:

http://localhost:8000/engines/itemsim/itemsim/topn.json?pio_appkey=NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM&pio_n=10&pio_iid=1

You may change the pio_uid and pio_iid parameters above to see results for other users/items. You are encouraged to take advantage of these endpoints to verify these results. You can also take advantage of our official and community-powered SDKs to simplify this process. Contributed SDKs are also available.

What’s News in PredictionIO v0.7

With the new extended support to GraphChi – a disk-based large-scale graph computation framework, developers can now evaluate and deploy algorithms in both GraphChi and Apache Mahout on one single platform!

Enjoy!