Release Testing Process

Wednesday, October 21st, 2009 @ 12:39 by tchungMozQA Badge

Welcome to the Release Testing page! Mozilla development cycle can range from 4 weeks till 2 months, depending on whether its a branch release or Trunk release. Branch releases are typically supporting the security point release builds (eg. Firefox 3.0.15 / Firefox 3.5.4), and trunk releases support upcoming releases in development (Firefox 3.1 beta 1). Both Branch and Trunk are very similiar in the test process, with one containing more test areas than the other. Lets break down what goes into each: Branch Release Testing

  • Products: Firefox 3.0.x, Firefox 3.5.x, Thunderbird 2.0.0.x
  • Timeframe: About 3-4 days
  • Areas covered: Smoketest, BFTs, L10n, Updates, Product-detail page

The process begins with bug fixes and checkins that land before a code freeze date. Once the last checkins are made, the build team will generate Mac, Linux, and Signed Windows bits on the nightly Ftp server. QA will then swoop in and run the covered areas for a expected period of 3-4 days, while verifying fixed bugs at the same time. If any blocker are found during the release cycle, bugs will be re-triaged immediately and options to respin the builds are considered. When testing has completed, builds are pushed onto the staging server for update testing. Web pages are staged and a second pass is made. When that's completed, the final push to the production environment is made, and a final check of release channel updates and download links is made. The process ends with a notice posted to http://developer.mozilla.org/devnews/ Trunk Release Testing

  • Products: Firefox 3.next
  • Timeframe: About 4-10 days
  • Areas covered: Smoketest, BFTs, FFTs, L10n, Updates, Plugin Regression, Extension Regression, Accessibility, Product-detail pages

Trunk testing is similar to branch testing, except there is a range of areas covered, depending on the timing of the release. The earlier in the cycle, the less complex the areas are covered. The later in the release cycle, the more features are landing, so the more complicated the coverage. Trunk testing focuses on pre-released software, and until it becomes an official released version, does it then move to branch testing. The order of the release process is no different than the branch process. For references, a release wiki tracks every branch and release cycle. Refer to https://wiki.mozilla.org/Releases for all you need to know for release testing and development progress.