Releases/Firefox 29/Test Plan

From New wiki
Jump to navigation Jump to search

« Firefox 28 | Firefox 29 | Firefox 30 »

Firefox 29 Test Plan

QA Lead: Juan Becerra

Schedule

  • 2013.12.09: Nightly
  • 2014.02.03: Aurora
  • 2014.03.14: Pre-Beta feature sign-offs
  • 2014.03.20: Beta 1
  • 2014.03.25: Beta 2
  • 2014.03.27: Beta 3
  • 2014.04.01: Beta 4
  • 2014.04.03: Beta 5
  • 2014.04.08: Beta 6
  • 2014.04.10: Beta 7
  • 2014.04.15: Beta 8
  • 2014.04.17: Beta 9
  • 2014.04.22: Beta 10
  • 2014.04.23: Release Candidate
  • 2014.04.29: Final

Features

Disabled

The following features have been deemed not ready for release and will be delayed to Firefox 30 or later.

On Track

Signed Off

The following features have no blockers and have been deemed ready for release.


Nightly

Fixed Bugs

How to Help
  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Download and install the version of Firefox used to report the bug
  • Using the older build, test the claim made in the bug report to see if you can reproduce it
    • If you cannot reproduce the bug move on to another bug or ask the reporter to verify the fix
  • Using the latest Nightly, test to see if you can reproduce the bug
    • If you cannot reproduce the bug set the STATUS to VERIFIED FIXED and add a comment describing your testing
    • If you can still reproduce the bug set the STATUS to REOPENED and add a comment describing your testing
  • Add any bug reopened to the list below using the bug number template (verified fixes will be tracked elsewhere)
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field
Reopened Bugs

Reported Bugs

Triage unconfirmed bugs.

How to Help
  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Test the claim made in the bug report and see if you can reproduce it
  • If you can reproduce the bug, set the STATUS to NEW, add a comment describing your testing, and try to find a regression window
  • Any confirmed bugs should be added to the list below using the bug number template
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field
Confirmed Bugs

Stability

How to Help
  • Review the Firefox 29 Explosiveness Report
    • Any signatures rising significantly (>= 2.0 explosiveness rating) without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest numbers
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 29 Topcrash Report
    • Any signature in the top 10 without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest ranking information
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 29 tracked bugs that aren't tracked in 28 or 27.
    • Each bug listed needs to have its rank updated once per week
    • Use the topcrash reports from crash-stats to get the rank for each branch
    • Add this information to the bug report
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Conversely, remove the topcrash keyword if the crash is no longer in the top-10
  • For each of these tasks, ensure the etherpad is updated with your name and the date checked
Topcrash Bugs

QA Wanted

How to Help
  • Download and install the latest Firefox Nightly
  • Review this list of bug reports to find a bug you both understand and want to test (each should contain a comment detailing what's needed from QA)
    • If the bug needs steps to reproduce, try to find a reproducible set of steps
    • If the bug needs a regression window, try to find a regression window
    • If the bug is fixed and needs verification, test to confirm you can reproduce the bug on an older build but not the latest build
  • Any tested bugs should be added to the list below using the bug number template
  • Remove the qawanted keyword from any fulfilled request
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field
Tested Bugs

Testdays

Aurora

Branch Migration

Pre Sign-off Tasks

  • {{|}} Firefox branch migrated (Release Management)
  • {{|}} Mozmill Tests branch migrated (Automation Development)
  • {{|}} Updates throttled on aurora channel (Release Engineering)
  • {{|}} Updates verified throttled on aurora channel (QA)

Sign-off Tasks

  • {{|}} Mozmill Daily functional testruns pass with no failures ([report])
  • {{|}} Mozmill Daily auroratest update testruns pass with no failures ([report])
  • {{|}} Scope features tested and signed off for Aurora uplift
Features (feature tracking)
Feature Owner Documentation Aurora Merge Sign-off
Australis Cornel Ionce wiki page [DONE]
New First Run experience Catalin Varga tracking bug [DONE]
Firefox Accounts Tracy Walker wiki page [ON TRACK]
Disallow enumeration of navigator.plugins Catalin Varga tracking bug [DONE]
Fix common scheme typos Petruta Rasa tracking bug [DONE]

Post Sign-off Tasks

  • Aurora sign-off email sent (QA)
  • Updates unthrottled on aurora channel (Release Engineering)
  • Updates verified unthrottled on aurora channel (QA)

Beta

Regression Testing

  • Run these tests for whatever platform(s) you have available.
  • Report a bug if you encounter one.
  • If you report a bug check to see if it's a regression by testing a previous Firefox version (if it doesn't reproduce in earlier versions it's a regression).
  • Add any reported bugs to the Bugs Found section below.
  • All results will appear in this dashboard

Beta 1

Build Info

Testing

Issues

Beta 2

Build Info

Testing

Issues

Beta 3

Build Info

Testing

Issues

Beta 4

Build Info

Testing

Issues

Beta 5

Build Info

Testing

Issues

Beta 6

Build Info

Testing

  • [DONE] Automation: review the functional, endurance, remote, add-ons tests and report bugs on any failures
  • [DONE] On-going exploratory testing around Australis
  • [DONE] Fix Verifications, focusing on those most recently fixed during this cycle.
  • [DONE] Updates: run update tests on the betatest and releasetest channel and report any failures
  • [ON TRACK] Live Updates: verify live updates work on the beta channel via automation (report)

Issues

Beta 7

Build Info

Testing

  • [ON TRACK] Automation: review the functional, endurance, remote, add-ons tests and report bugs on any failures
  • [DONE] On-going exploratory testing around Australis
  • [DONE] Fix Verifications, focusing on those most recently fixed during this cycle.
  • [ON TRACK] Updates: run update tests on the betatest and releasetest channel and report any failures
  • [ON TRACK] Live Updates: verify live updates work on the beta channel via automation (report)

Issues

Beta 8

Build Info

Testing

  • [DONE] Automation: review the functional, endurance, remote, add-ons tests and report bugs on any failures
  • [DONE] On-going exploratory testing around Australis
  • [DONE] Fix Verifications, focusing on those most recently fixed during this cycle.
  • [DONE] Updates: run update tests on the betatest and releasetest channel and report any failures
  • [ON TRACK] Live Updates: verify live updates work on the beta channel via automation (report)

Issues

Beta 9

Build Info

Testing

  • [DONE] Automation: review the functional, endurance, remote, add-ons tests and report bugs on any failures
  • [DONE] Testing around video with Silverlight and Flash to cover bug 988156.
  • [DONE] On-going testing around Australis which had, again, several small fixes.
  • [DONE] Fix Verifications, focusing on those most recently fixed during this cycle.
  • [DONE] Updates: run update tests on the betatest and releasetest channel and report any failures
  • [ON TRACK] Live Updates: verify live updates work on the beta channel via automation (report)

Issues

RC Build 1

Build Info

Testing

Issues

Firefox 29.0.1

Build Info

Testing

Issues

Bugs by Status

Resolved

<bugzilla> { 

"f1":"cf_status_firefox29",
"o1":"anywordssubstr",
"v1":"fixed verified",
"include_fields":"id,summary"

} </bugzilla>

Unresolved

<bugzilla> { 

"f1":"cf_status_firefox29",
"o1":"nowordssubstr",
"v1":"--- unaffected verified fixed disabled wontfix",
"include_fields":"id,summary"

} </bugzilla>

Retrieved from "https://mozilla.mediawiki.site/w/index.php?title=Releases/Firefox_29/Test_Plan&oldid=974893"