Update Testing

Thursday, July 2nd, 2009 @ 14:43 by ashughesBugzilla BadgeCommunity LeadCrash Badge

This document is meant to explain how we test updates and how you can set up your test environment to test updates.

Table of Contents

1. Update Channels

2. Update Types

3. Forcing Update Failure

4. Complete Update

5. Debugging Software Update

 

Update Channels

Update Channels are used to tell Firefox where to check for updates.  We use four different channels at different stages of update testing and for different types of testers.

beta

This is the default channel used by "beta" versions of Firefox

nightly

This is the default channel used by "nightly" versions of Firefox

release

This is the default channel used by Firefox releases and release candidates

betatest

This is the channel used by QA to test updates before they are pushed live to the beta channel

releasetest

This is the channel used by QA to test updates before they are pushed live to the release channel

To change which channel you want Firefox to look for a file called channel-prefs.js and open it with a text editor.  You can find this file in one of the following locations (depending on your operating system):

Windows

{location of Firefox}\defaults\pref\channel-prefs.js

Mac

{location of Firefox}/Firefox.app/Contents/MacOS/defaults/pref/channel-prefs.js

Linux

{location of Firefox}/defaults/pref/channel-prefs.js

The file will contain the line pref("app.update.channel", "release"); Change "release" to whatever channel you want to test.

Update Types

There are three different types of updates which you will hear about while update testing.

partial

This type of update is from the second newest version of Firefox to the next version of Firefox.  For example, Firefox 3.0.10 to Firefox 3.0.11.  It is usually quite a small update.

complete

This type of update is from one version of Firefox to a version of Firefox more than one version newer.  For example, Firefox 3.5rc1 to Firefox 3.5rc3.  This update should be similar in size to the full download of Firefox.

fallback

This type of update is when a failure of an update is forced (see Forcing Update Failure below)

major

This type of update is from one major release of Firefox to the next major release of Firefox.  For example, Firefox 3.0.11 to Firefox 3.5.

Forcing Update Failure

There are times when we may want to test that update paths still work even in the face of a failure.  We typically check that all updates work with a failure.  For example, we will often test partial+fallback and complete+fallback for every release.  This ensures that even users who experience a failed update will still be offered an update.  In the case of a partial+fallback, a complete update should be offered upon failure.

To instigate a failure use the following steps:

1. Click Help menu > Check for Updates
2. Click "Update Firefox"
3. When you see the "Update Ready to Install" dialog, click "Restart Later"
4. Close Firefox
5. Open update.status in a text editor and change "pending" to "failed" (see Locations of update.status below)
6. Save the file and open Firefox
7. At this point you should be presented with an "Update Failed" dialog.  Click "Continue" to get the complete update and restart Firefox.
8. When Firefox restarts, type "about:" into the location bar and press ENTER.  Use this page to confirm the version was updated.

Locations of update.status

Windows

{location of Firefox}\updates\0\update.status

Mac

{location of Firefox}/Firefox.app/Contents/MacOS/updates/0/update.status

Linux

{location of Firefox}/updates/0/updates.status

Complete Update

The following is the steps required to do a complete and successful update.

1. Click Help menu > Check for Updates
2. Click "Update Firefox"
3. When you see the "Update Ready to Install" dialog, click "Restart Now"
4. When Firefox restarts, type "about:" into the location bar and press ENTER.  Use this page to confirm the version was updated.

Debugging Software Update

There are times when you may want to enable software update debugging.  For example, when you are checking for an update which should exist but Firefox cannot seem to find the update.  To enable debugging, do the following:

1. Type "about:config" in the location bar and press ENTER

2. Click "I'll be careful, I promise!"

3. Right click in the list of preferences and select New > Boolean

4. Type "app.update.log.all" as the preference name and give it a value of "true"

5. Type "app.update.log.all" in the Filter textbox to verify the pref has been added

Whenever you check for updates, an entry should appear in the Error Console (Tools menu > Error Console).  If the value is "<update></update>" it means no update was found.  If an update should have been found, file a bug.