War on Mobile Crashes

nhirata

Bugzilla Search list:

1. Top crashes :

https://bugzilla.mozilla.org/buglist.cgi?keywords=topcrash&keywords_type=allwords&list_id=2131004&resolution=—&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=[native-crash]

– This is the list of top crashes that we are trying to push developers to look at.
– smooney will push core crash bugs to dev
– elancaster will push mobile crash bugs to dev
– We need to make sure all new top crashes are listed here (be vigilant when a severe crash bug appears)
– if it affects nightly / aurora  (use the fields affects nightly, affects aurora )

2. Reproducible crashes :

https://bugzilla.mozilla.org/buglist.cgi?keywords=reproducible&keywords_type=allwords&list_id=2131014&resolution=—&classification=Client%20Software&classification=Components&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=[native-crash]&product=Core&product=Fennec&product=Fennec%20Native

–  need to try to make sure that we can reproduce the bugs still
–  Need to hunt down regression ranges for at least the ones that are rated P1/P2
–  if it affects nightly / aurora  (use the fields affects nightly, affects aurora )

3. All the other crashes:

https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=notsubstring&list_id=2135266&field0-1-0=keywords&field0-0-0=status_whiteboard&classification=Client%20Software&classification=Components&query_format=advanced&value0-2-0=reproducible&value0-1-0=topcrash&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&field0-2-0=keywords&type0-0-0=substring&value0-0-0=native-crash&type0-2-0=notsubstring&product=Core&product=Fennec%20Native

– need to verify that these crashes are still shown in socorro.  if not, we will close them off
– need to check in socorro to see if the crash is seen in nightly and aurora (use the fields affects nightly, affects aurora )
– need to try to reproduce the bugs  [see Triaging Crash Bugs section]

4. Socorro
– need to check socorro for any new top crashes, repros, regression ranges, etc.
– other data are available :

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennecandroid.nightly.components.weekly.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.nightly.explosiveness.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.nightly.devices.weekly.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.nightly.startup.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.aurora.devices.weekly.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.aurora.explosiveness.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.aurora.startup.html

https://crash-analysis.mozilla.com/rkaiser/2012-01-22/2012-01-22.fennec.aurora.components.weekly.html

~~~~~~~~~~~~~

Triaging Crash Bugs:

1. go to a bug on the list
2. check the last comment to see when it was worked on last
– anything more than 2 weeks is a definite must for an update
3. go to soccoro : https://crash-stats.mozilla.com/products/FennecAndroid
4. select Advanced Search in the upper right hand corner
5. verify : Product = FennecAndroid , Version = All, period 30 days, Report Process Any, Report Type, any
6. place one line from the crash signature in the bug report in the Stack signature contains
7. filter the search ; java crashes are going to be a bit trickier… separate section.

A. if you don’t see the any results for any of the crash signatures, copy the url and paste it in the bug report and close the bug report off as WFM and stating that there has been no signature existing within the last 30 days.
<done>
Note: it can be reopened if we see more signatures later.

B. if you do see crash signatures with an exact match :
1. click the link
2. select Reports tab
3. sort by buildid; verify that the last crash happened in a recent build for both aurora and nightly.
4. if it happens to be affected in aurora/nightly mark it so in the bug under the tracking flags
5. place note that it happens in aurora/nightly
6. in the crash stats check the bugzilla tab to see how many bugs maybe duplicate.
– be careful as some may be desktop crashes
– place notes if you think they may be duplicate.  ie : the first 5 stacks should be the same in most cases if they are a duplicate.
7. make sure that the tags are set correctly :
a. if it’s in fennec native : place in [native-crash] in whiteboard
b. if it’s in fennec xul : place in [mobile-crash] in whiteboard
c. keywords : crash
d. verify correct Product, Component based on the crash stack (last frame that has a .cpp file is most likley the location where it should go.
e. verify correct platform (arm android)
f. verify severity critical
g. verify crash “signature” is placed in the crash signature field

[you can paste in the search to update]

8. if you think it’s a very important bug mark it with QA^ in the whiteboard
9. if you see that it’s a top crasher within socorro (crash-stats.mozilla.com), place topcrash in the keywords.
10. other things you might find : the earliest build that this crash occurs within soccoro
11. Graph might help in terms of visual representation of how much the crash is causing an issue in fennec.

Note: Correlation tab doesn’t really show anything for mobile.

– Java Signatures will be moving to it’s own field (bug 718907).
– If there is a Java signatures, it should go to the Java field.
– If not, double check the “App notes” field and make sure that there are no Java signatures in there.
– The Java crash signature is more likely to be the cause and not the crash stack

C. Try to reproduce:
Trickest part:
1. go to the crash signature link
2. go to a particular stack
3. click on the top link for the top frame and take a look.
<need to fill more>

Other things to note :
– bug 720622
– bug 720152
– BadTouchMemory is a purposeful crash…
– There’s a bug in regards to bad stacks from BadTouchMemory
– bug 711461
– There had been a bug in regards to wrong crash stack; I can’t seem to find it at this moment in time.

Filed under: Uncategorized