B2G/QA/2.2

From New wiki
< B2G | QA
Jump to navigation Jump to search

Team Breakdown

Testing

  • V2.2 Contact window
    • Overall & MGSEI related: Eric Chang
    • Automation: Edward Chen
  • Smoketest
    • Daily by MGSEI starting from 2015/01/12(GMT-8) & V2.2 Branch is available.
    • Make sure not breaking anything on flame-kk.
    • Make sure working okay on L device.
    • Daily report: Eric Chang
  • Smoketest Automation
    • Daily by TPE starting from 2014/01/12(GMT-8) & V2.2 Branch is available.
    • Daily by MGSEI starting from 2014/03 (TBD).
    • Make sure not breaking anything on Flame-kk.
    • Make sure working okay on L device.
    • Daily report: TPE
      • gaia ui test - shift
      • hermes, alison, gerry, cynthia, shing, teri <end>
  • Automation new case creation & QA Ownership
    • Starting from FLR or FL.
    • Feature, bug, functional owners will help on this.
  • Full run
    • Starting from FL by MGSEI, targeting twice in 6 weeks
    • Trial run from 2014/12/22.
    • Also help to report invalid test cases.
    • TPE will fix invalid test cases after that.
  • Bugzilla related
    • Marigold & Eric will help with qawanted, verifyme & other tags.
  • MTBF
    • MTBF: Alison Shiue, Paul Yang, Gerry Chang
      • MTBF - shift
      • alison, gerry, eric, cynthia, shing, teri <end> edward, hermes
  • Performance
    • Will check with Geo.
  • Exploratory
    • Will limit the scope of testing.
    • By MGSEI
  • Usability testing

V2.2 QA Owners

Here is a table indicating QA contacts by major areas for Firefox OS V2.2.

2.2+ list

META bug for Test case review

Description Bugzilla QA Contacts
Better performance on Multicore (App startup time for core apps is comparable to Android) bug 1086963, bug 1082268, bug 1082262, bug 1074783 Edward Chen
L Porting bug 1094121 Askeing Yen, Hermes Cheng
256M RAM memory optimization for WFVGA, 5MP. bug 1082290 Teri Wen
BlueDriod (1005934 - Move bluedroid to a separate process) bug 1005934 Eric Chang, Teri Wen
CAF list of bugs bug 1031175, bug 1020757, bug 1068963, bug 1016807, bug 1043558, bug 1073252 Eric Chang
NFC payments, NFC API bug 979158, bug 1042851 Alison Shiue
Privacy bug 1073419, bug 1083953 Gerry Chang
Cut, Copy, and Paste bug 1023688 Gerry Chang, Edward Chen
Internal: Metric collection of MonthlyActiveUser bug 1109422 Shing Lyu
TCL list of bugs bug 1091544, bug 1061130, bug 1096815 Alison Shiue, Eric Chang
Support for multiple search suggestion providers bug 1099157, bug 1098494 Hermes Cheng

Feature by Components

Feature tracking by components

Feature Area Owners

Area QA Contacts
Management Brian Huang
Release Lead Brian Huang
Productivity Teri Wen, Edward Chen
Media Alison Shiue, Eric Chang
System Front End Hermes Cheng, Gerry Chang
Communications Eric Chang
Performance Edward Chen
Networking/Connectivity Alison Shiue, Eric Chang
System Platform Gerry Chang
Multimedia Platform Edward Chen
Devices Askeing Yen, Hermes Cheng
Bluetooth Eric Chang, Teri Wen
Metrics Shing Lyu, Gerry Chang
Stability Paul Yang, Walter Chan

Reports & Update

2014 Q4 2015 Q1 2015 Q2 2015 Q3

Code Complete

Release Signoff Status

Feature Complete

Release Signoff Status

Health:

Feature Complete Criteria

FC 2015/04/29

  • All feature landing exceptions must be resolved
  • Bugs with keyword late-l10n should be zero
  • Chipset vendor requirements or bugs needed to meet their FC date must be resolved
  • Completed one functional test run
    • Yes, 2 runs.
  • QA defines 2 areas of "functional areas" for exploratory testing and lead will send a "signoff" note when he feels its qualitatively ready -
    • Yes.
  • Smoketest Greenness - Latest length of time that we haven't had smoketest regressions needs to be greater than five days
    • No, OTA issue is not yet resolved.
  • MTBF Test Run lasts for 30+ hours:
    • Yes.
    • NEXUS-5-L 5.1 PVT build MTBF 38.5 hrs & counting.
    • FLAME-KK 4.4 PVT build MTBF 34.42 hrs.
  • blocking_b2g = 2.2+

<bugzilla> 

   {
       "cf_blocking_b2g": "2.2+",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

  • blocking_b2g = 2.2?

<bugzilla> 

   {
       "cf_blocking_b2g": "2.2?",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

Feature Landing

Release Signoff Status

Functional Signoff Status

Smoketest Status

Nominations

<bugzilla> 

   {
       "keywords": "smoketest",
       "cf_blocking_b2g": "2.2?",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

Blockers

<bugzilla> 

   {
       "keywords": "smoketest",
       "cf_blocking_b2g": "2.2+",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

Gaia UI Test Regressions Status

Nominations

<bugzilla> 

   {
       "keywords": "qablocker",
       "cf_blocking_b2g": "2.2?",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

Blockers

<bugzilla> 

   {
       "keywords": "qablocker",
       "cf_blocking_b2g": "2.2+",
       "resolution": "---",
       "include_fields": "id, assigned_to, status, resolution, component, summary"
   }

</bugzilla>

Test Coverage Completed

Everything with [2.2-feature-qa+] in QA Whiteboard

<bugzilla> 

   {
       "cf_qa_whiteboard": "[2.2-feature-qa+]",
       "flagtypes.name": "in-moztrap-",
       "include_fields": "id, qa_contact, status, resolution, component, assigned_to, summary"
   }

</bugzilla>

Verification Status

Overall Verification Summary

<bugzilla type="count" display="bar"> 

   {
       "cf_qa_whiteboard": "[2.2-feature-qa+]",
       "x_axis_field": "status"
   }

</bugzilla>

Verification Needed

<bugzilla> 

   {
       "cf_qa_whiteboard": "[2.2-feature-qa+]",
       "status": "RESOLVED",
       "include_fields": "id, qa_contact, status, resolution, component, assigned_to, summary"
   }

</bugzilla>

Verification Complete

<bugzilla> 

   {
       "cf_qa_whiteboard": "[2.2-feature-qa+]",
       "status": "VERIFIED",
       "include_fields": "id, qa_contact, status, resolution, component, assigned_to, summary"
   }

</bugzilla>

Definitions

Health Notations

  • [ON TRACK] QA thinks the health is trending towards green, but signoff isn't complete yet
  • [AT RISK] QA thinks the health is trending towards yellow, but signoff isn't complete yet
  • [DONE] QA has signed off to indicate the health is green
  • [MISSED] QA has chosen to not sign off to indicate the health is red

FLR Functional Team Criteria

FL Functional Team Criteria

The functional team FL health was determined against the following criteria:

  • MozTrap test coverage done?
  • All user stories verified to spec?
  • Smoketests green around your area?

The lowest health score out of each of the criteria above is used as the team's overall health status.

FC Functional Team Criteria

The functional team FC health is determined against the following criteria:

  • Have all features gone through a functional test run?
  • Did any risky feature areas get exploratory testing?
  • Were smoketests around your area green for 5+ days?
  • Did each outstanding FC signoff blocker get fixed?

The lowest health score out of each of the criteria above is used as the team's overall health status.

Retrieved from "https://mozilla.mediawiki.site/w/index.php?title=B2G/QA/2.2&oldid=1083097"