Posts from October, 2014

Continuous Delivery – a generic plan

Matt Brandt

Many of Mozilla’s web projects have adopted concepts from continuous delivery.
Once a code change lands on a master branch it is shepherded to production with
little-to-no human intervention. Quickly getting new features out to users.

Definition of a participant

While the implementation details vary from project to project, the core concepts
tend to remain the same between projects. It is important for project stewards;
project managers, developers, testers, UX, community, etc to have a shared
vocabulary.

The purpose of this document is a starting point for this conversation.

Why we do this

We find that being able to move quicker allows us to create higher quality
software with limited resources. This fits our goal of creating features that
our users want, working in the open, and experimenting with our development
practices.

Continuous delivery has worked for us because of the culture of our community.
We accept shared responsibility for the quality of our work, are strong
collaborators, and have a culture that is supportive of learning from mistakes.

Over time we’ve become more confident in the quality of our work. Moving quickly
allows us to take a more proactive stance towards the quality of the products we
create as well as mitigating risk when creating new features.

What are QA’s responsibilities

  • Acting as a user advocate
  • Leading discussions that identify and assign risk to user stories
  • Driving exploratory testing efforts

What are the dev team’s responsibilities

  • Developing new features for the project and implementing fixes for issues
  • Developing unit tests
  • Maintaining code quality via code review and documentation of standards

Shared responsibilities

** project management, developers, qa, UX, l10n **

  • Deep knowledge of the application and the ability to identify areas of risk
  • Provide feedback on feature definition to the project manager and team
  • Identifying, creating, and maintaining the end-to-end test automation
  • Developing the appropriate mitigation strategies to lower risk of regressions landing in production
  • Staying current on best practices and pushing the team to explore the applied relevancy
  • Continually discussing and fine-tuning test coverage
  • Opening avenues for community collaboration on the project

How we do this

The basic concept is that any code that lands on the master branch should be in
a shippable state. New feature work should be categorized by the
amount of risk the feature or change to the code base poses.

Risk-based assessment

The team discusses and agrees upon a risk metric that they are comfortable with
assuming. Assumed risk isn’t a static conversation, but it is good to build a
skeleton that the team can agree on and enhance over time.

As a product should be broken into user stories so a team can apply a
hierarchy of risk. It is important to identify and engage as many stakeholders
as possible, each individual will have a unique and important point of view.

The list of stakeholders often include; project managers, developers, testers,
UX, IT, DBAs, the security team, etc. Sometimes the product has high enough
visibility in the organization that a proxy for the CEO is beneficial.

User stories and features are grouped into three categories

  • Features that users should never see broken in production
  • Features that can go to production broken but need to be discovered and fixed within a specific time period
  • Features that can regress on production and will be fixed once a user identifies the problem

An example from mozillians.org

  • The ability to create a new account should never go to production broken
  • The ability to edit the profile of an existing account can break but we want to discover and patch the problem within 1 hour of introducing the regression
  • CSS layout can regress on production and will be fixed when a user uses the in-site feedback form to notify us

Feature testing

Not a dogmatic rule, in general new features or defect fixes should be manually tested and verified. Exploratory testing is the rule. Depending on the risk assigned to a feature, projects have the option of using waffle flags. The use of feature flags allows us to hide features from the general public while allowing a team or a group of beta testers to test on production with real data.

An important note about feature verification

If a feature is deemed as low risk and it’s functionality is easy to verify, anyone on the team is
empowered to test and verify it. If it is a feature that requires deeper investigation and poses
high risk, the QA team is the group who verifies it. If feature set is big or and the use workflow
is changed it is important to engage the test team to flush out defects and usability concerns.

Automate tests

Feature work that falls into the category of “better never break” should see a heavy amount of exploratory testing before it is exposed to the public. Once the feature is baked and code churn has decreased, we seek an appropriate level of coverage using Selenium, Python’s Requests API, load testing, and fuzzers.

A component of the QA team automation strategy should also includes reviewing the amount of test coverage that follows a developers pull request as well as taking part in the code review. Often times the test team is able to influence and enhance the depth of coverage by simply asking for more developer level tests [Unit + Integration].

User feedback

Sites should always have an easily discoverable method for users to submit feedback and file bugs. The team must be religious about responding to requests for help in a timely manner and do so with empathy. .. do not respond to a user with “invalid” or “works as expected.”

Continue reading

Android MediaCodec use in Firefox for Android

Aaron Train

James ‘snorp’ Willcox has landed support for hardware decoding via the public MediaCodec Java class in Nightly for Android for devices running Android 4.1+ (Jellybean). This is replacing OMXCodec & the Stagefright library which was introduced as a replacement for OpenCore for media decoding. This relatively new public Java class is used for decoding H.264/AAC in MP4 for playback in the browser with the benefit of allowing for direct access to the media codecs on the device through a “raw” interface.

This should correct a number of playback issues which have been reported to us regarding problems on Android 4.1+ devices.

Victory!

How to Help

  • Install Nightly (available on 10/21/2014’s build)
  • Test video playback on your Android 4.1+ device (e.g, test page)
  • Talk to us on IRC about your experience or problems

Continue reading

LittleRP Build

Al Billings

LittleRP Frame

I’ve spent a good part of this weekend working on finishing my build of a LittleRP resin 3D printer. There was a kickstarter for kits and assembled printers from its creator but he’s also been on IRC in #dlp3dprinting on Freenode for quite a while. The design is CC-licensed right now, not true open source, but he’s been sharing his bill of materials and plans for the lasercut and 3d printed parts with anyone who wants to build one. I assembled parts mostly in August but then had to go to three security conferences and a week long trip away for work in the course of six weeks so I got derailed from assembly. In the meantime, he’s started documenting things a bit for assembly so the wait was probably a good thing. This is probably about $400 or $500 in parts plus an HD projector that costs $300-400 (I got it for the low end because a refurbished one was fine).

You can see a gallery of sample prints from the creator, which are of a much finer level of detail (and much smaller) than what you get from FDM printers like your printrbots or mendels. Another guy who has finished building his using the same process as me is Shane Graber and he’s got a pretty good gallery of his machine and sample prints on Flickr.

Necromancer?

My plan, other than just having an interest in playing with this technology, is to use the printer for tabletop RPG and other gaming pieces or parts. The level of detail is so high for the small sizes of the prints that games are a great application of this, assuming one is into game playing and design.

Continue reading