QAForum Structure

来源：互联网发布：达尔罕软件下载编辑：程序博客网时间：2024/06/10 01:06

WebEx Ph 408.435.7000307 W Tasman Fax 408.435.7004San Jose, CA 95134 QAForum Test Cases Guideline Document Author: Vincyye Chan Last updated: 05/11/2004Revision: 2.1Revision History DateAuthorSection(s)Description of Change(s)04/29/04Vincyye ChanAllInitial draft05/03/04Vincyye ChanAllRevised base on feedbacks05/03/04Vincyye Chan3.1Add test Description template 5.2Add JavaClient and Agent to Table 5.2 3.2.1Add note for unused options in Priority05/07/04Vincyye ChanAllAdding Priority 1,2,3 to module05/11/04Vincyye ChanAllAdding Summary and TA information5/12/04Vincyye Chan3.3.24.23.2.1.1Adding Integration Suite OS/Browser selectionAdding Spec verificationUpdate Priority 1,2,3 definition and give examplesPREFACESummary: The QAForum Test Cases Guideline Document gives the overall guidance for creating test cases, test module, test suites, test plan and test run in the QAForum; and defines their corresponding correlation to the different test phases defined in QA Test Strategy document. All the definitions in this document will serve as the fundamental QAForum test cases guideline and should be well understand and followed when creating test cases in QAForum.Document Version Control: It is the reader's responsibility to ensure they have the latest version of this document. Questions should be directed to the owner of this document, or the project manager. PREFACE. 21. Introduction. 41.1 Purpose. 41.2 Overview.. 42. QAForum Structure Overview.. 53. QAForum Test Cases Guideline. 63.1 Creating Test Cases. 63.2 Creating Test Modules. 73.3 Creating Test Suites. 93.4 Creating Test Plan/Run. 104. QAForum Test Run and Test Cycle Phases Correlation. 124.1 Phase I Feature and Functional Interoperability. 124.2 Phase II System Benchmarking. 124.3 Phase III User Acceptance. 124.4 Bug Validation. 124.5 Test Automation. 134.6 Spec Verification. 134.7 Phase I, II, III Test Strategy Summary. 135. Appendix. 145.1 QAForum Structure Example – Phase I Run. 145.2 Naming Convention Reference Table. 155.3 Choosing the Objective. 166. Reference.. 177. Glossary.. 171. IntroductionQAForum Test Cases Guideline Document standardizes the QAForum test cases structure and clarifies the definitions and processes from test cases to test runs.1.1 PurposeThe main purpose of this document is to provide clear instructions to QA team on creating test cases, test modules, test suites, test plan and test run. It also gives the definition on all the options and provides examples to clearly guide the QA engineer through the process.Another important purpose of this document is to define the correlation between the Phase I, II, III testing cycle and the QAForum test runs. 1.2 Overview For each Phase of a release, QA needs to create a corresponding test run in the QAForum for tracking. Each of the run is based on a Plan; a Plan is consisted of test Suites; a Suite is the group of multiple test Modules and each test Module composed of tens of test cases. And for each test Run QA created, it should associate with a certain test Phase. 2. QAForum Structure Overview Test RunTest Plan Test Suite A [OS/Browser combinations] Test Module AA [Priority 1, Priority 2, Priority 3] Test Case 1 [Precondition, Steps, Expected Result], [Objective] Test Case 2 … Test Module BB Test Suite B Test Module XX Test Module YY Test Module AA Test Suite C Test Module DD Test Module EE …3. QAForum Test Cases Guideline3.1 Creating Test Cases3.1.1 Test Case ObjectivesFor each test cases entered, an objective must be selected to differentiate the purpose of each test cases. The possible options for objectives are as follows:§ FunctionalityTest Cases with this Objective are used to test the fundamental functionality of the feature. E.g. Send a Chat message; create a Poll § Functional InteroperabilityTest Cases with this Objective are used to test the behavior when combining two of more features together. Testing on the horizontal level of the product.E.g. Chat+Video § Product Interface (System/Product Integration)Test Cases with this Objective are used to test the integration from page to client and the flow of the product (end-to-end testing). This is considered the vertical level of testing for a product.E.g. Host Schedules a meeting -> Host Starts the meeting -> Attendees Join meeting -> Host Loads some document -> Host Starts Polling -> Attendee Chat with Host -> Host End meeting § Error HandlingTest Cases with this Objective are used to test any negative, exceptions and out of boundary casesE.g. Input a Character for a numeric field § Backward CompatibilityTest Cases with this Objective are used to test the files saved with an older version can still work in the newer versionE.g. Polling files created from older version can still be loaded into the newer version § Upgrade Test Cases with this Objective are used to test if site can still function normally after Super Admin and site upgraded to a later versionE.g. T19 -> T20 upgrade § MigrationTest Cases with this Objective are the specific test cases for Migration other than the general site testing.E.g. SA default value matches site behavior § Use CasesTest Cases with this Objective are used to test user scenarios provided by PM, tech support, etc. China team creates the general use cases and US leads will provide customer use cases. § PerformanceTest Cases with this Objective are used to establish the performance Benchmark for our product. E.g. Performance date on Load test and Stress test § SecurityTest Cases with this Objective are used to test the Product Security.E.g. JMA test cases, login, change meeting ID to join meeting3.1.2 Test Case StructureIn order to construct a test case in a better and clearer way, we must specify the following in the description:1. [Precondition] – defines the entry criteria for the test case (e.g. super-admin option, lab computer setup)2. [Steps] – clearly list each operation needed for this test case and enter in the correct order3. [Expected Result] – defines what is the output should be based on spec3.1.3 GuidelineWhen creating test cases, the followings apply:q Select the correct Objective for each test caseq Choose the right test Category (see next section for the different Category available)q Mark if the test cases can be automated or notq Description of test case should be precise and clearNote: In all descriptions, please avoid any ambiguous wordings like some, may be, etc, and make sure the meaning of the wordings being used.3.2 Creating Test ModulesA test Module consists of many individual test cases for either a particular feature (Chat) or have a common test objective (Performance). 3.2.1 Module Priority3.2.1.1 Priority DefinitionThere are three categories to choose from when creating a test case for a module:1. Priority 1Test cases include core cases for the feature module (the main functionality of the feature). These test cases will be executed for Regression test cycle or for Certification test. And they will also be the considered as the priority test cases for automation. E.g. In Polling, create a new Poll with Single/Multiple/Short answer and start/end Poll are the Priority 1 cases(You have to think what Polling does, the main functionality of Polling) E.g. File Browse: Transfer a file from remote to local and from local to remote, abort a file transfer2. Priority 2Majority of the test cases for the module should belong to this priority. Test cases in this priority are created to test the secondary functionality of the features (secondary functionality means even without using these functionality, the main functionality of the feature can still work. E.g. Polling: Change type, Options, Save Poll etc, will be Priority 2 cases(Priority 2 cases are secondary functionality of Polling, without these functionality working, Polling can still run – like even Save Poll not working, we can still create/start/end a Poll) E.g. File Browse: Transfer multiple files, abort multiple transfers, transfer a large file 3. Priority 3Test cases in this priority include low usage functionality, corner cases and exception test cases. Besides, most of the error handling cases should belong to this priority. E.g. Polling: Disable Timer, Poll question contains more than 20 multiple choices, go up and down between choices, multiple choice Poll questions does not have any choice etc. are all Priority 3 cases (Priority 3 cases are corner test cases, or seldom used functionality as well as exceptions and error handling cases) E.g. File Browse: Transfer different type of files, Transfer a file already exist, Close transfer window when transfer going on 3.2.1.2 Test Cases RatiosPriority 130% of all casesPriority 240% of all casesPriority 330% of all cases 3.2.2 Naming ConventionAll test Modules must follow the same naming convention in order to distinguish between Page, Client, and their sublevel features.Definition: [Product Name]-[Product Layer]-[Feature Name]-{Feature Subset}Example:SC-Page-SiteAdmin-EditUserSC-Client-ChatSC-Integration-Performance-StressSC-Page-FISC-Client-FISC-Integration-PI Note: Refer to Appendix table 5.2 for Product/Product Layer/Feature names3.2.3 GuidelineWhen creating test Modules, the following applies:q Add all Test Case in Module into the correct Branch (e.g. T20)q Uses the correct Naming Convention when creating Module name 3.3 Creating Test SuitesA Suite is used to organize the test Modules based on different intensions. It also defines on what platform(s) these suites will be executed.3.3.1 Suites CategoriesA test Suite can be categorized in one of these categories: 1. Feature Level The Feature Level test Suites are organized into Level I Suite, Level II Suite and Level III Suite. The Level I, II and III test Suites are run against Level 1, 2 and 3 OS/Browser combinations respectively.2. Integration Level (FI, PI, MT, Performance, etc.)Focus on the integration type of testing on modules and the product as well as the performance testing.3. TA Test Automation cases for detecting high-level issues for packages and regression checking during bug fixes. 3.3.2 Basic Test Suites for Creating RunFor each release, QA leads need to create 9 test suites including 3 for feature level, 3 for integration level and 3 for TA.Level I Feature Suite§ Includes ALL Feature modules* and selects Level 1 Browser/OS combinations to run on§ Priority 1&2&3 test cases for New Features § Priority 1&2 test cases in Old FeaturesLevel II Feature Suite § Includes ALL Feature modules* and selects Level 2 Browser/OS combinations § Priority 1&2 test cases for both New and Old FeaturesLevel III Feature Suite § Includes minimum function Feature modules* and selects Level 3 Browser/OS combinations § Only cover basic test cases like Join, leave meeting and major component can start (All cases covered by SVT suite)Phase I Integration Suite§ Select Level 1 Browser/OS combinations§ Includes Priority 1&2 test cases in Functional Interoperability module§ Includes Priority 1&2 test cases in Product Interface module Phase II Integration Suite§ Select Level 1 Browser/OS combinations§ Select all Priority 3 test cases in Functional Interoperability module§ Select all Priority 3 test cases in Product Interface module § Include all Use Cases in all modules including the CustomerUseCases module§ Include all Backward Compatibility test cases in each module§ Include all test cases in Upgrade module§ Include all test cases in Performance and Stress test module§ Include all test cases in Product Installation module Phase III Integration Suite§ Select Level 1 Browser/OS combinations§ Select all Priority 1&2 test cases in Functional Interoperability module§ Select all Priority 1&2 test cases in Product Interface module § Select all test cases in MT and Priority 1 test cases in all ModulesSVT Suite§ The very basic functionality test cases being automated by TABVT Suite§ Larger scoop of basic functionality test cases being automated by TAURL API Suite§ All automated test cases in URL API module* Selection should base on Level 1,2,3 features in the Functional Priority Document3.4 Creating Test Plan/Run3.4.1 Test PlanA test Plan is defines which test Suites to be executed in a test Run. QA should create one test plan for each testing cycle.Phase I Run – 100% Product walkthrough, find 70% bugs**Includes§ Level I Feature Suite§ Level II Feature Suite§ Level III Feature Suite§ Phase I Integration SuitePhase II Run – Focus on the System Benchmarking and use cases testing**Includes§ Phase II Integration SuitePhase III Run – Focus on finding any regression issue caused by bug fix**Includes§ Phase III Integration SuiteTA Run Includes§ SVT Suite § BVT Suite § URL API Suite ** For detail definition for each Phase, please refer to QA Test Strategy document. 3.4.2 Test RunA test Run is created based on a Plan. A Plan defines which Suites to run. QA needs to create one test Run for each of the test cycles (Phase I, II, III) as well as the TA Run. The next session will explain the relationship between test Run and the three test Phases. 4. QAForum Test Run and Test Cycle Phases Correlation4.1 Phase I Feature and Functional InteroperabilityPhase I will execute test cases in each of the following area:1. Feature testNew Features: Priority 1&2&3 test cases (Level I, II, III Feature Suite)Old Features: All Priority 1&2 test cases (Level I, II, III Feature Suite)2. Functional Interoperability testPhase I Integration Suite3. Product Interface testPhase I Integration Suite4.2 Phase II System BenchmarkingPhase II will execute test cases in each of the following area:1. Package InstallationPhase II Integration Suite2. Use CasePhase II Integration Suite3. Performance Benchmark and Stress TestPhase II Integration Suite4. Backward Compatibility and Upgrade TestPhase II Integration Suite5. Functional Interoperability and Product Interface SubsetPhase II Integration Suite4.3 Phase III User Acceptance1. Regression TestPhase III Integration Suite; Feature suites (Including any feature module(s). Each team can decide if this is needed based on the bug fix quality and scoop when CF)2. Migration Test (MT)Phase III Integration Suite4.4 Bug ValidationDuring each execution Phase, by default QA should validate all fixed bugs and run regression test on effected area to detect regression. 4.5 Test AutomationTA will be part of every test Phase. All TA scripts will be run multiple times depend on their purpose.SVT Suite – Run against every All-In-One package to ensure the package qualityBVT Suite – Run during every build to detect regressionURL API Suite – Run periodically or during each milestone4.6 Spec VerificationThe spec verification will not have a separate suite but should be included in the test phases.UI and Overall Layout Test – should be included in Phase I testing. It can either have a separate module or verify with spec in hand.Wordings and Text Layout – should be included in Phase II testing by going through word-by-word, paragraph-by-paragraph for all the messages and text base on the spec. 4.7 Phase I, II, III Test Strategy SummaryTest Plan/RunTest TypeTest SuiteTest ScopeTest Case PriorityOS/Browsers12312330%40%30%Defined in WBX005726Phase IFeature Suite (FS)Level IAll feature modulesNEWNEWNEWÖ OLDOLD Level IIAll feature modulesNEWNEW Ö OLDOLD Level IIIMinimum modulesSVT Cases ÖSVT Cases Integration Suite(IS) PhasePhase I IntegrationFI, PIÖÖ TASVT, BVT and URL API Phase IIPhase II IntegrationFI, PI Ö Use Cases, BC, Upgrade, Performance, Stress, Installation TASVT, BVT and URL API CFCFPhase IIIPhase III IntegrationFI, PIÖÖ MT Selected Feature ModulesNEW/OLD TASVT, BVT and URL API ERER 5. Appendix5.1 QAForum Structure Example – Phase I RunRunPlanSuiteModuleTest Case Phase I Test Run Phase I Test Plan Level I SuiteTC-Page-CalendarCase 1: Session Information of a single training sessionCase 2: Listing of a multiple training sessionCase 3: …TC-Page-SiteAdmin-Add/EditUser Case 1: Add a new userCase 2: Modify user informationCase 3: …TC-Client-BOCase 1: Host Allow BO sessionCase 2: Host Invite attendee to a BO sessionCase 3: …… Level II SuiteTC-Page-CalendarCase 1: Session Information of a single training sessionCase 2: …TC-Page-SiteAdmin-Add/EditUserCase 1: Add a new userCase 2: …TC-Client-BOCase 1: Host Allow BO sessionCase 2: …… Level III SuiteTC-Page-InstantSessionCase 1: Starts a instant sessionTC-Page-Start/Join MeetingCase 1: Join in a session from Live SessionsTC-Client-ASCase 1: Present share MS Word application… Phase I Integration SuiteTC-Client-FICase 1: Starts Chat while doing FT with large file (>100K)Case 2: In BO session but continue main session telephonyCase 3: …TC-Integration-PICase 1: Set the site to allow only 5 attendees in each session -> Host starts a session contains Test -> Host starts telephony -> Host launches TestCase 2: Site uses custom session type and attendee e-Commerce -> Host starts a recorded session and registered attendee join in -> Host do App Share and Doc Share -> Host end meeting -> Attendee access the recorded sessionCase 3: …… 5.2 Naming Convention Reference TableLevelsName Can Be SpecifiedRemarksProduct MC TC EC SC SMT SMARTtech is a standalone product$C CC EE Including MyWebex, login, etc.WCC DB, etc.Product LayerPage Client JavaClient Agent Used by SC onlyPE Used by MC onlyIntegration Feature NamePerformanceOnly used by Integration Product LayerCustomerUseCasesOnly used by Integration Product LayerFunctional Interoperability (FI)Only used by Integration Product LayerProduct Interface (PI)Only used by Integration Product LayerMigration Test (MT)Only used by Integration Product LayerInstallationOnly used by Integration Product LayerUpgradeOnly used by Integration Product LayerI18n L10N … Examples:MC-Client-i18nMC-Page-L10NMC-PE-EmailMC-Integration-Performance-LoadMC-Integration-FIMC-Integration-FI-CrossPlatformMC-Integration-PIMC-Integration-PI-CrossPlatformMC-JavaClient-AppShareMC-Client-ClientDownload 5.3 Choosing the ObjectiveModule TypeObjectives can be usedFeature Module (e.g. MC-Client-Chat)FunctionalityUse CaseError HandlingBackward CompatibilitySecurityFunctional Interoperability Module (e.g. MC-Integration-FI)Functional InteroperabilityError HandlingUse CasesSecurityProduct Interface(e.g. MC-Integration-PI)Product InterfaceError HandlingUse CasesSecurityPerformance(e.g. MC-Integration-Performance-Stress)PerformanceUpgrade(e.g. MC-Integration-Upgrade)UpgradeError HandlingUse CasesSecurityInstallation(e.g. MC-Integration-Installation)FunctionalityError HandlingUse CasesSecurityMigration(e.g. MC-Integration-Migration)MigrationError Handling6. Referenceq QA Test Strategy – defines QA test life cycle and detail meaning on each cycle during the whole release.q Functional Priority Document – details what are the features belongs to each Level of testing (Level 1, 2 and 3) and list out the OS/Browser combination in each Level. Each Service should have its own Functional Priority Document.q Systems and Browsers (WBX005726) – Defines OS/Browser combinations need to be tested for each release.7. GlossaryFI Functional InteroperabilityPI Product InterfaceMT Migration ToolBC Backward Compatibility