Requirement Validation Scenarios
The scenarios are in the context of a RM consumer , such as a Quality Management Tool, who has the interest to cover requirement(s) that are planned to be included in a software release. The requirement(s) need to be implemented as well as tested in the Software Engineering Lifecycle.
Persona
Product Owner (Business Analyst) - Ursula is the owner of the JKE Banking solution, providing the business requirements for the solution. She coordinates a team of product owners in charge of various products and feature teams that are part of the JKE Banking solution.
Product Owner (Requirements Analyst) - Bob is the owner for delivering the features in the JKE Bank product in support of the Business Recovery Matters project. He works for Ursula. The entire JKE Banking solution is so large it requires several Product Owners representing various products that comprise the solution. He is responsible for representing the stakeholders needs and prioritizing the Product backlog for his feature.
Test manager - Tammy is leading the quality verification and is practicing independent system testing in support of the team’s “whole team” approach to testing. She creates and oversees the overall test plan for the project.
Tester (team member) - Tanuj is a tester embedded with the development team practicing concurrent testing. In this role, Tanuj is responsible for contributing to and executing against the Test Plan.
Team Lead/Development Lead/Scrum Master - Marco is a developer and team lead. His team is practicing agile development. He is also the Scrum Master responsible for defining and coaching the project teams on Agile practices, and for rolling out and enacting these practices for his feature team.
Tester (team member) - Tanuj is a tester embedded with the development team practicing concurrent testing. In this role, Tanuj is responsible for contributing to and executing against the Test Plan.
Scenario A: Product Manager Bob groups requirements for the release into collection and creates “Test coverage” request for Test Manager
- Bob has created a set of requirements for the release. The requirements are not necessarily fully elaborated at this moment;
- Bob creates one or more Requirement Collections for the release, and adds requirements into the collection(s);
- Bob creates a “Test Coverage” Request, which can be a “Quality Task” type of Change Request in a CM provider and assigns it to the Test Manager/Lead, indicating these Requirement Collections need to be fulfilled by the release;
Scenario B: Test Manager Tammy associates test plan and test cases to cover the requirements
- Tammy associates the requirement collection(s) with test plan(s), the test plan can be new ones or existing ones.
- User can create the association for the Requirement Collection from an RM application using the ValidatedBy link;
- User can create the association for the Test Plan from a QM application using the validatesRequirementCollection link;
- In the context of Test Plan and Requirement Collection association, Tammy wants to make sure all the requirements in the collection are covered by Test Cases in the Test Plan.
- If there is more than one Test Plan then the Test Manager may create a Master Test Plan[1] that contains the Level Test Plans, having at most one level of hierarchy. The Requirement Collections may be linked to the Master Test Plan, its child Test Plans, or a combination of both. The main idea is that each Requirement in the Requirement Collections needs to be covered by a Test Case in at least one of these Test Plans.
- Tammy starts to analyze the test coverage gap. The objective can be achieved by a traceability view, a report or a reconciliation tool. In the following sub-items, we will use reconciliation as an example to achieve the goal.
- In an RM application, user can reconcile the Requirement Collection to discover:
- which requirements in the collection do not have test coverage by the Test Plan(s);
- Whether there are test plans which have redundant coverage; (If you want to have a “just enough” coverage)
- In a QM application, user can reconcile the Test Plan to discover:
- which requirements in the requirement collection do not have test case coverage yet;
- Which test cases under the test plan do cover any requirements that are not in the current release; (again, If you want to have a “just enough” coverage)
- Given the test coverage gap analyze results in #4, Tammy can generate or reuse Test Cases for each requirements that do not have test coverage yet. Unnecessary Test Plans or Test Cases can be removed from the traceability graph.
Note: As long as the requirement collection has not been finalized, these test case coverage activities may need to be revisited (see next scenario).
Scenario C: Test Manager Tammy is aware of requirement changes and reflect the changes for corresponding test cases
This scenario only applies for the process where requirement artifacts are not finalized before associate with test artifacts. [unlike scenario described in [Scenario A: Requirements are implemented, validated and delivered][2]In a scenario where requirements and tests are developed in parallel, Test Manager and Testers need to keep track of the requirement changes, to make sure the test are always reflect the latest requirement design. 2 direct approaches that can achieve this goal will be elaborated below:
Suspect Scenario [Hypothetical]
- Project Admin defines a Suspect Profile for a set of RM resources by applying “Watching” Validated By. The profile limits the interests to certain requirement types and certain attributes;
- Bob associates the Suspect Profile with the Test Plans and Test Cases;
- Bob changes a requirement or requirement collection in RM;
- The change causes a Test Plan or Test Case to become suspect. The suspicion could be shown in a Viewlet, Table Views, or Test Artifact Editors;
- In a viewlet, Test Plan whose associated requirement collection changed will be listed there before the suspicion been cleared. The changes might includes:
- Requirements newly added to the collection;
- Requirements removed from the collection;
- Requirements deletion;
- Collection attribute changes;
- Changes to the attributes of requirements in the collection;
- Collection been deleted;
- In a viewlet, Test Case whose associated requirement changed will be listed there before the suspicion been cleared. The changes might includes:
- Requirement attribute changes;
- Requirement deletion;
- [Optional] Email for suspicion are sent out to interest parties; i.e. Owner, Reviewers and Approvers of the Requirement, Tests; Project Manager, Test Lead, etc.
- Tanuj who own the Test Plans or Test Cases review the suspect information by visit the “delta” history. And decide the follow up action:
- Clear the suspicion;
- Directly update the test artifact, and then clear the suspicion;
- Create a quality task Change Request (in CCM) to postpone the decision until later;
Reconcile Scenario
- Marco reconciles a test plan to compare changes made after last reconciliation; The changes may include:
- Requirements newly added to the collection;
- Requirements removed from the collection;
- Requirements deletion;
- Collection attribute changes;
- Changes to the attributes of requirements in the collection
- Collection deletion
- Tanuj reconciles a test case to compare changes made since last reconciliation; The changes may include:
- Requirement attribute changes;
- Requirement deletion;
- Marco/Tanuj compares the “delta” history of the RM resource;
- Based on an analysis of the history, Test Lead/Tester makes decisions around the follow up action. The follow up action might include:
- Mark the Test Case or Test Plan as suspect;
- Ignore the change;
- Set the Test Artifact as “up to date” with respect to the requirement;
- Create a quality task Change Request to postpone the decision until later
Scenario D (optional): Tester and Requirement Analyst collaborate for test construction
- Tanuj reviews the associated requirements to construct the test case;
- Tanuj finds there are unclear or lack of information in the requirements;
- Tanuj creates a change request to the requirement analyst to provide more information or correct the current requirement content;
- Bob discusses with Tanuj in the change request, reject or make changes to the requirement;
- Tanuj completes the test case construction and asks the requirement analyst for review;
- Bob approve or reject the test design and provides more comments;
Scenario E: Project Manager/Test Manager Marco traces the requirement coverage
- For all the requirement collections in a release, Marco reviews whether there are test plans - in the same release - associated with the requirement collections; In addition, (s)he might also check, whether the Development Plan Items related to the requirement collections are associated with the corresponding Test Plans;
- For all the requirements in a release, Marco reviews whether there are test cases - in the same release - associated with the requirements; In addition, (s)he might also check, whether the Development Tasks related to the requirements are associated with the corresponding Test Cases;
- If there are omit coverage:
- Marco submit a Quality Task Change Request to follow up with;
- Marco generate or reuse the Test Plan/Case;
Scenario F: Project Manager/Test Manager Marco traces the requirement execution status
- For all the requirement collection in a release, Marco reviews:
- how many of them had been executed
- what’s the latest execution results
- Are there any defects been created during the execution? What’s the current status for the defects?
- For the requirements that has high priority, Marco might submit request to Tanuj and Deb (Defect Owners) to accelerate the execution and fixing;
- Marco generates report to stackholders for the project progress;
(Optional) Given the risk and status, Marco re-evaluate whether some of the requirements need to move out of the current release;
References
- [1] IEEE std. 829-1998 Standard for Software Test (it is suitable fro any kind of test plan including Master, Acceptance, System, Integration, Unit): http://www.computing.dcu.ie/~davids/courses/CA267/ieee829mtp.pdf
- [2] http://open-services.net/bin/view/Main/RmRequirementCompletedScenario
