Automation Plan Execution Environments

Sometimes an AutomationPlan needs to be executed in a certain type of environment. For example, an AutomationPlan could be created for testing a web application using Firefox, DB2, and Tomcat on Linux 64bit architecture. Another AutomationPlan could be created for testing the same web application using Safari, Oracle, and Websphere on Windows 32bit architecture. In terms of test coverage you might imagine that AutomationPlans could be created for executing this same test in many other combinations or permutations of browser/database/application server/OS. In this scenario, each such grouping is called an execution environment.

The set of AutomationPlans for executing a single test case are often identical with the exception of their execution environment. In fact it is typical for a test architect to define their test cases in a generic way that abstracts out the environmental details and focuses instead on the inputs/outputs for the test cases and the expected precondition/postcondition of the system under test. Then at a later time the test architect may define or automatically generate a multitude of execution environments and create an AutomationPlan for executing each test case on each environment. The execution environments might also be defined in a separate system such as cloud provisioning or in a virtualization server, and then bound to the AutomationPlans during an execution setup/preparation phase or just in time before execution.

In this type of scenario it is convenient to conceptualize an execution environment as a resource having a URI and having its own lifecycle that is independent from any AutomationPlans that use it. This allows the Automation Provider to delegate the responsibilities of defining execution environments to an external provider, and allows a single execution environment to be bound to multiple AutomationPlans or vice-versa. It may also provide an opportunity to handle execution environments as reconcilable resources.

Contents

• Automation Plan Execution Environments
  • Scenarios
    • Test architect defines execution environments and generates Automation Plans for Test Cases
    • Test architect generates Automation Plans for Test Cases using execution environments managed by an external provider (variation of previous scenario)

Scenarios

Test architect defines execution environments and generates Automation Plans for Test Cases

1. Test architect creates a catalog that defines various types of IT resources used for testing. For example the catalog might contain various types of web browsers, databases, application servers, and operating systems.
2. Test architect generates a list of execution environments from a catalog of IT resources. This list of execution environments includes several different combinations of the types of resources in the catalog. For example, one such combination might be “Firefox/DB2/Tomcat/Linux”.
3. Test architect selects a set of execution environments from the list that will be covered by the overall testing effort.
4. Test architect creates a test case. The test case defines a precondition/postcondition and the expected input/output parameters for a particular test scenario.
5. Test architect generates a set of AutomationPlans for the Test Case, one for each of the execution environments.

Test architect generates Automation Plans for Test Cases using execution environments managed by an external provider (variation of previous scenario)

1. Test architect creates a test case. The test case defines a precondition/postcondition and the expected input/output parameters for a particular test scenario.