Be a Beta Tester... try out the latest Firefox 3 Beta! Let us know if you find a bug or if you have any feedback by using Hendrix.  You can also join us for any of our upcoming Test Days if you want to work with the Mozilla QA team on specific issues or have any questions for us.  If you would like to help with future releases, join our betatesters mailing list.  Thanks!

Home

Talos

What is it?

Talos is a performance testing project. With a framework written in Python it runs Ts (startup test) and Tp (page load test) while monitoring memory and cpu usage. Ts is a simple Javascript web page that times the loading and then exits. Tp consists of a Javascript web page that cycles through a set of locally served web pages. The web page set is a collection of the top 500 web pages as listed by Alexa; they have been 'cleaned' so that all of their content is locally served.

The information collected by Talos is sent to a graph server so that it can be viewed and analyzed. Currently, the graph server is located at graphs.mozilla.org, divided between the continuous graphs (graphs of average results of a given test plotted over time) and discrete graphs (graphs of individual test runs). Check out Talos numbers explained for an explanation of the links displayed on the tinderbox waterfall page.

Talos can be installed and run on Windows, Linux and Mac.

Where does it live?

The code is checked in on trunk and is located in /mozilla/testing/performance/talos. It can be checked out with:

>cvs -d ":pserver:anonymous:anonymous@cvs-mirror.mozilla.org:/cvsroot" co mozilla/testing/performance/talos

Links of interest

  • continuous graph results
  • discrete graph results
  • Talos perf numbers explained
  • Test Reference Platform Pages
  • Talos Machine Names and Tasks
  • New Graph Server documentation
  • Tinderbox -> Talos Migration Plan

    • Configuration of Talos & The Test Boxes

  • Browser Configuration

    • The browser is set to run in position 0,0 with dimensions 1024x768.

    The following perferences are set:

    • browser.shell.checkDefaultBrowser : false
    • browser.warnOnQuit : false
    • dom.allow_scripts_to_close_windows : true
    • dom.disable_open_during_load: false
    • dom.max_script_run_time : 0
    • browser.dom.window.dump.enabled: true
    • network.proxy.type : 1
    • network.proxy.http : localhost
    • network.proxy.http_port : 80
    • dom.disable_window_flip : true
    • dom.disable_window_move_resize : true
    • security.enable_java : false
    • extensions.checkCompatibility : false
    • extensions.update.notifyUser: false

    Additionaly, it needs the following prefs.js file to be in the base_profile directory.

    user_pref("capability.principal.codebase.p0.granted", "UniversalPreferencesWrite UniversalXPConnect UniversalPreferencesRead");
    user_pref("capability.principal.codebase.p0.id", "file://");
    user_pref("capability.principal.codebase.p0.subjectName", "");
    user_pref("capability.principal.codebase.p1.granted", "UniversalPreferencesWrite UniversalXPConnect UniversalPreferencesRead");
    user_pref("capability.principal.codebase.p1.id", "http://localhost");
    user_pref("capability.principal.codebase.p1.subjectName", "");
    user_pref("signed.applets.codebase_principal_support", true);

  • Machine Configuration

    • Most of the machine configuration for the various platforms (Windows, Linux, Mac) can be found in the test reference platform pages.

    The screen size is set to 1280x1024.

    Performance Machines - Names and Tasks

    We are tracking the state of the various machines running performance testing here.

    Help!

    There's plenty to be done to make this framework more useful as a testing tool. If any of these sound interesting or you have another feature to suggest please make a forum post. Here are the highlights:

  • Migrate from tinderbox testing to Talos

    • In the long range, the Talos project is to replace the current test tinderbox automation system. For this to happen Talos will have to be able to run versions of all tests that are currently run on the test tinderboxes on all platforms (windows, mac, linux). Further details can be found here.

  • Graph server layout

    • The new graph server is an excellent tool for visualizing the huge amounts of data that Talos collects - and will eventually end up being sent data from a variety of other browser tests. But, it's layout can make it complicated if not completly baffling to use. We are currently putting together a new wire frame layout of what we'd like it to look like. When these drawings come in they'll be posted in the hopes of someone picking them up and applying them. The graph server documentation can be found here.

  • Distributable Page Set

    • Talos is currently using a page set gathered from the wilds of the internet. This includes all kinds of copyrighted material; it can't be distributed to developers or the general community without breaking fair use. Having a set of freely distributable web pages (be they donated or sourced from sites without breaking their copyrights/agreements) would be an excellent addition to our testing tools.

  • Build 'interesting' profiles to test with

    • Currently, before starting a test run Talos creates a new, clean profile for testing with. Blank profiles aren't something that many people browse with and doesn't reasonably test Firefox's behavior. Having a variety of profiles with different properties (long histories, large cache, etc) would give a better idea of how the browser really performs on a users system.