B2G/QA/2.2/2015-03-23Status
Jump to navigation
Jump to search
V2.2 Highlight & Risk
- Highlight
- Availability of Flame-L is unknown.
- QA tracks for the 2.2? & 2.2+ bugs
- FLAME-KK - Full run test
- NEXUS-5-L - Regression test
V2.2 Milestone
- 2015/05/18 V2.2 CC
- 2015/04/06 V2.2 FC
- 2015/02/23 V2.2 FL
- QA Shared calendar
- You can add it to your Google Calendar view by clicking on this link and then clicking on the small blue "+ Google Calendar" button at the bottom, right hand of the screen.
- If you are signed in with your @mozilla.com account, you can also access it this way:
- In Google Calendar, you should go to Other calendars > Browse Interesting Calendars > More > Resources for mozilla.com > Public (9) > Public - FxOS QA Schedule > Subscribe
V2.2 Testing Scope
- Focus on bugs in agreed list.
- B2G V2.2 Overview Page for more info.
- Current not yet resolved feature-b2g=2.2+ 8 bugs
- Current not yet resolved blocking-b2g=2.2+ 87 bugs
V2.2 Testing by Type
Smoke Testing
Flame-KK Yellow
- No bug found blocking manual smoketests but we have couple automation bugs need to investigate.
- New Bugs Breaking the Automation smoketests
- [Possibly a script error] - test_clock_create_new_alarm.py - Bug 1138278 - [Clock] After running test_clock_create_new_alarm.py, it should set the alarm time to 1 min more than the current time
- [Investigating] - test_IMAP_email_notification.py - Bug 1146273 - test_IMAP_email_notification.py failure. [NEW ticket]
- [Investigating] - test_ftu_skip_tour.py - Bug 1134445 - [FTU] About Firefox OS page has "Yes! Send data." checked primarily.
- [Investigating] - test_marketplace_packaged_app.py -Bug 1146275 - test_marketplace_packaged_app.py failure. [NEW ticket]
- Automation reports - are available on Github.IO & fxosqa-report-watchlist.
Nexus-5-L Yellow
- No bug found blocking manual smoketests but we have couple automation bugs need to investigate.
- Manual - are available on Google Drive
- Known issues - OTA not working, Video recording not working, no FM radio support, no DSDS support
Regression Testing
- RTL/V2.2 Flame-KK Done
- Moztrap result - 76 fail, 326 pass, check with Josh for detail.
- Full Regression/V2.2 Flame-KK Ongoing
- Moztrap result - 29 fail, 2373 pass, 63% complete
- Partial Regression/V2.2 Nexus-5-L Done
- Moztrap result - 30 fail, 1216 pass, 100% complete
- Bug list
- Bluetooth PTS/V2.2 Flame-KK Done
- Bug Meta dependency tree - 2 open bugs
Functional Testing
Yellow
- General (eric)
- Flame-L might be late for FC, roughly 6 weeks or more needed after SOW (SOW, legal review, porting, build time), PM will have an internal discussion to decide what to do.
- feature-b2g|blocking-b2g == 2.2?|+ [not resolved] http://mzl.la/1C4X1KT Need attention
- feature-b2g|blocking-b2g == 2.2?|+ [resolved] http://mzl.la/1E5mQqM Need verification
- RTL (Marigold)
- Health: Yellow
- Details: 18 blockers
- Privacy (Gerry)
- Cut, Copy, and Paste (Gerry, Edward, Teri, Eric)
- NFC (Alison)
- Health: Green
- Details:
- one blocker: https://bugzilla.mozilla.org/show_bug.cgi?id=1143528 (RESOLVED FIXED) .
- short_name support in manifest.webapp (Hermes)
- Health: Green
- Details: All 2.2 features were landed. Perform a test run for this feature this week.
- Search App (Hermes)
- Health: Green
- Details: 2 WIP blockers
- App-grouping (Hermes)
- Health: Green
- Details: No 2.2+ blockers for now
- Task Manager (Hermes)
- Health: Green
- Details: 2 WIP Blockers
- L-porting (Hermes)
- Health: Green
- Details: 1 WIP Blocker
- CAF bugs (Eric)
- Health: Green
- Details: 1 WIP (1020757)
- TCL bugs (Alison, Eric)
- Health: Green
- Details: Regression checking
- Bluetooth (Eric, Teri)
- Health: Green
- Details: Completed PTS testing - 3 bugs
- Completed basic testing on BT APIv2 - 2 bugs
- Metrics (Shing)
- Health: Yellow
- Details: 2 Blockers http://mzl.la/1F21hgq
- 256M RAM memory optimization for WFVGA, 5MP. (Teri)
- Health: Green
- Details: In progress, no blocker
- Multicore performance (Edward)
- Health:
- Details:
bisect result: http://bit.ly/1MGOa59
Exploratory Test
Flame-KK & NEXUS-5-L
- 262 bugs found - includes every bug in bugzilla affecting v2.2.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED - 132 bugs found.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 16 bugs found.
- RESOLVED, VERIFIED, CLOSED - 130 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 10 bugs found.
- Regression bugs
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 15 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 232 bugs found.
V2.2 Test Case Creation
On Track
- Moztrap link of V2.2 New Features suite.
Test Case Creation WIP
- [2.2-feature-qa+] & in-moztrap? (0 bugs) --> WORK DONE
- feature-b2g=2.2+ & in-moztrap? (0 bugs) --> WORK DONE
Test Case Creation DONE
- [2.2-feature-qa+] & in-moztrap+ (19 bugs) --> WORK DONE
- feature-b2g=2.2+ & in-moztrap+ (78 bugs) --> WORK DONE
V2.2 Testing Verification Status
Overall Verification Summary
Need Verification <bugzilla>
{
"cf_qa_whiteboard": "[2.2-feature-qa+]",
"status": "RESOLVED",
"include_fields": "id, qa_contact, status, resolution, component, assigned_to, summary"
}
</bugzilla>
Verification Done <bugzilla>
{
"cf_qa_whiteboard": "[2.2-feature-qa+]",
"status": "VERIFIED",
"include_fields": "id, qa_contact, status, resolution, component, assigned_to, summary"
}
</bugzilla>