Posts by Aaron Train

James ‘snorp’ Willcox has landed support for hardware decoding via the public MediaCodec Java class in Nightly for Android for devices running Android 4.1+ (Jellybean). This is replacing OMXCodec & the Stagefright library which was introduced as a replacement for OpenCore for media decoding. This relatively new public Java class is used for decoding H.264/AAC in MP4 for playback in the browser with the benefit of allowing for direct access to the media codecs on the device through a “raw” interface.

This should correct a number of playback issues which have been reported to us regarding problems on Android 4.1+ devices.

Victory!

How to Help

• Install Nightly (available on 10/21/2014’s build)
• Test video playback on your Android 4.1+ device (e.g, test page)
• Talk to us on IRC about your experience or problems

Continue reading

At last week’s Mozilla QA Meetup in Mountain View, California I demonstrated how to effectively profile Gecko in Firefox for Android on a device. By targeting a device running Firefox for Android, one can measure responsiveness and performance costs of executed code in Gecko. By measuring sample set intervals, we can focus on a snapshot of a stack and pinpoint functions and application resources produced by a sampling run. With this knowledge, one can provide a valuable sampling of information back to developers to better filed bug reports. Typically, as we have seen, these are helpful in bug reports for measuring page-load and scrolling and panning performance problems, frame performance and other GPU issues.

How to Help

• Install this (available here) Gecko Profiler add-on in Firefox on your desktop (it is the Gecko profiler used for platform execution), and follow the instructions outlined here for setting up an environment
• If you encounter odd slowdown in Firefox for Android, profile it! You can save the profile locally or share it via URL
• Add it to a (or your) bug report on Bugzilla
• Talk to us on IRC about your experience or problems

Here is an example bug report, bug 1047127 (panning stutters on a page with overflow-x) where a profile may prove helpful for further investigation.

Detailed information on profiling in general is available on MDN here.

Continue reading

Recent work on standing up a proxy server for web browsing in Firefox for Android is now ready for real world testing. Eugen, Sylvain, and James, from the mobile platform team have been working towards the goal of building a proxy server to ultimately … Continue reading

As Jeff Beatty describes, language switching in Nightly is now available. Ultimately, this linguistic enhancement allows one to select from all officially shipped localizations from within the browser independent from available language selection provided by Android. As Jeff calls out, “Languages no longer have to be among the list of Android supported languages to become official localizations of the browser.”

Firefox for Android QA is looking for help from others for discovering issues found when trying this feature out.

The developer of the feature, Richard Newman, calls out the following to look for in Bug 917480 when testing this feature on Nightly:

Note: the option to control this is in Menu > Settings > Language > Browser language

• Nightly should obey one’s selection as their preferred Android system-provided language. Firefox has obeyed this in previous and current releases

• Nightly should use one one of the languages we ship, regardless of system language

• Testing this feature involves verifying that a change is immediately applied, and that all entry points into the application reflect the selected language

• Entry points to check:
  — Data reporting notification. This launches Settings in the “Mozilla” section. Titles should be correct: on tablet, for example, you should see “Settings” in the top left, and “Mozilla” as a heading. You only get this on first run, so you’ll need to Clear Data to get back to a clean slate and test this out
  — Launching the browser. Top sites should be in the correct language, as well as other UI elements
  — Clicking a link from another application
  — Installable Firefox Market Web applications

Other areas affected by language change:

• Sync settings when accessed via Nightly and via Android Settings > Accounts & Sync > Firefox

• Sent tabs from other devices: the launched notification should be in the last language you picked

Notes of interest:

• Language selection changes should be persistent across browser sessions and restarts

• All chrome content, such as error pages should be in the correct selected language

• Setting the browser language has the side effect of changing your Accept-Language header. You should get pages in non-phone languages sometimes; depends on the site

• Verify that switching to Android Settings and changing the system language does the right thing if “System default” is selected, and does the different right thing if a specific language is selected

If you discover any issues, please file a bug on Bugzilla

References:

Bug 917480

Try it out on Nightly today

Continue reading

This upcoming Friday, we would love for you to solicit your support towards testing and contributing to the process of qualifying a new developer-release version of Firefox for Android and Firefox on Desktop both on the Nightly channel as part of our rapid release testing cycle. We will be focusing on the process of discovering, using and testing WebRTC.

WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities via simple Javascript APIs. The WebRTC components have been optimized to best serve this purpose. The WebRTC mission is to enable rich, high quality, RTC applications to be developed in the browser via simple Javascript APIs and HTML5. The WebRTC initiative is a project supported by Google, Mozilla and Opera.

This QA event, this Friday, is open to all those interested: newcomers, experienced testers, developers, and anyone interested in testing early builds of Firefox on the Nightly channel.

We would like for you to use the new version of Firefox on your Android phone and desktop or laptop machine, and take a close look at the latest Nightly builds in order to assist us in identifying any noticeably major issues found with our WebRTC implementation, and ensure that all feature functionality that is included in this upcoming release is on its way to a feature and testing complete state.

To cover the work throughout the whole day we have created
an EtherPad test-plan. Please feel free to read and use it as a companion during the day’s event. There will be moderators at hand in the IRC channel: #testday, to answer any questions you have.

Together with your help we want to make this event a success and ensure the high quality of WebRTC support in Firefox for all of our users world-wide. If you have time on Friday June 21st, 2013, please join us on IRC, we will have Mozilla community and testers on hand to help answer any of your questions.

Your testing and feedback is highly valuable, and we hope to see you attend our test day event.

Continue reading

[For more information on Robocop Automation, see this Mozilla Wiki entry]

Short update of where we (Mozilla QA) are at with our recent automation efforts in converting our smoke-tests to automated tests. As of this month there are now approximately forty-five available Mochitest-Robotium (Robocop) tests with approximately thirty currently enabled and running per-checkin (see example run here). Each individual test is ran (see Firefox – Tinderbox Pushlog) in-part of a growing suite of automated user-interface tests that exercise in probing and injecting various native events to the Java front-end of Firefox for Android. In-turn, these provide automation for our list of identified browser smoke-tests (manual testing). While we have steady progress henceforth, we are always looking for more interested to-be test-authors who wish to help us continue to grow our automation efforts.

Short summary on some of the blocking situations we have recently ran into and some details on their solutions:

• Due to a number of user-interface differences between various Android devices, it was deemed necessary to create a Device and Navigation support class in BaseTest. Further investigation revealed that was it also necessary to map navigation (Back, Forward and Reload) due to different types of Android devices (i.e, tablets vs phones and varying Android platform versions) see bug 747835.

• Test stablization on the Panda and Tegra boards has been tricky. After further investigation we found out that the Tegra boards run Android 2.3 (API Level 11), and Firefox for Android serves the small-screen mobile user-interface for phones running Android 2.3. The Panda boards run Android 4.0.4 (API Level 15) and Firefox for Android serve the larger-screen mobile user-intreface optimized for 7-inch tablets running Android 4.0+. This split environment was rather sub-optimal.

• On Android 3.0+ (API Level 12+), the system Cancel and Ok buttons were switched around (gee, thanks Google!) so it was deemed necessary to use a clickOnButton(name); function instead of clickOnButton(index);. This was causing some failures early on.

• Again on Android 3.0+ (API Level 12+) pop-up notification buttons were not accessible at times due in part to the to an invocation of the system virtual keyboard, which would overlay cover the testable view. This was a result of having prior, entering text via the sendKeys() function. Thus this requires dismissing the virtual keyboard first and foremost before trying to click any buttons see ongoing investigation in bug 838596.

• Depending on the Android device, platform and screen-size we have varying user-interfaces thus it was required to check for proper navigation depending on the environemtn (e.g, we have a function to check for the “More” and “Tools” submenus in order to access items), see bug 830755.

• We had to hack around finding a solution for traversing the number of views in Firefox for Android’s settings user-interface. In detail, we had to dynamically fetch the number of checkboxes available to ensure that if further items are to be added, these will not affect the running test. This requires disecting the settings menu and asserting if each component is a checkbox, see ongoing investigation in bug 830834.

Current work and areas we need help in

• Expanding our set of tests, see existing coverage here
• Stabilize set of problematic tests (intermittent failures and timeouts), _see mozilla-central’s test manifest; specifically, bug 825152, bug 746876, bug 824067, 813107, bug 757475, bug 738890

How you can help

• Run our manual set of smoke-tests against Nightly builds of Firefox on your Android phone or tablet
• Contribute towards writing additional tests (see: writing Robocop tests); individual tests are tagged as either automatable or not in Moztrap
• Contribute towards improving the Robocop framework (see: Auto-tools); see open Robocop related bugs, source code

Continue reading

Over the last few weeks in-between Firefox for Android release cycle work, I have been bestowed on the interests of contributing to the Gaia UI automation efforts established from the Mozilla Web QA and Automation & Tools teams by assisting in writing, extending and building upon the available tests and APIs. Preliminary development of an extensive test suite of basic functional smoke-tests tests is underway.

Reviewing the available Gaia smoke-tests, the feasible test-cases can be written into Python based manifest-driven tests that are executed by Mozilla’s own automation driver, Marionette. Sharing much of the same API as Selenium and Web Driver, it makes writing tests rather easy. For example, a first test I have written recently is able to launch, interact and assert content behaviors of HTML elements in the bundled Gaia music application with relative ease — afterall, each application is merely HTML.

Each test that is built upon the Marionette foundation utilizes a Python package named Gaiatest (available on PyPi here) that is in-fact based on Marionette. It is used specifically for writing tests against Gaia and has a growing variety of atomic Gaia related APIs such as: unlocking the device home-screen, toggling WiFi/cellular data, setting volume and interactions with other device settings, etc. For example, here is the current data_layer.js file where we can write functions that interact with accessible public interfaces from the variety of available WebAPI. In turn we can we abstract and create APIs for use in our tests (e.g, preliminary tests such as test_dialer.py disables device volume and soon to be test_browser.py where one can toggle WiFi and or celluar data).

I would encourage all those who are currently writing Gaia applications to try out Marionette and Gaiatest and to contribute to our UI automation efforts by writing well-tested tests to expand our current test-suite. The [Github][] repository is where all activity is at with chatter taking place on #appsqa on Mozilla IRC.

Further information on running Marionette tests can be read here alongside a README for installing and setting up Gaiatest.

Continue reading

My last post on modifying an Android Stagefright H.264/AAC/MP3 decoder preference in Gecko, which toggles software and hardware decoding of those formats in Firefox for Android briefly talked about how that override can be usful for testing. As part o… Continue reading

Here at Mozilla, we’re always looking and investing in different ways
that our mobile related testing efforts and strategies can evolve and
expand into new avenues as Firefox for Android mature as a product.
We’re always engaged in seeking out different yet worthwhile services
and tools that can integrate into our regular QA process.

Whilst we continue to operate as a small team (Mobile QA for Firefox on
Android is about two-full time employees); we’re seeking all the help we
can get whether it be from community or from those who have created
services and utilities to assist mobile QA efforts.

We react in seeking solutions by continuing to identify and respond to
problems. In identifying our areas of contention, we still see Android
device fragmentation and access on-demand to Android devices as
major paint-points of Android testing. To administer any type of soluton
would involve boasting and utilizing the services and assistance from
others. Currently we use DeviceAnywhere for one-off direct device
testing; but we desired a service that targets automation.

A couple months ago we stumbled upon AppThwack, a service that
manages mobile application and fragmentation issues by deploying
uploaded Android Package (APK) files and deploying (installation) on
realAndroid based phones and tablets. The devices are hosted and
interfaced through web front-end that makes it easy to view and extract
results ran from automated tests (currently self-written and
provided MonkeyRunner tests are available, and on-track towards
deploying Robotium and possibly in the near future Mozilla based
Robocop tests). Alongside, screen-shots are provided from multiple
device orientations which make it usful for identifying any
user-interface issues. A test-run takes a couple minutes and any data
collected may be bundled and downloaded together to deploy towards any
new Bugzilla bug for track-keeping.

Currently, we are utilizing their service for testing Firefox for
Android builds during our rapid release Beta cycle and are looking
at different strategies of integration for the other channels within our
cycle.

If you would like to see what a test-run looks like click here.

Continue reading