Automation Plan Execution Environments

Sometimes an AutomationPlan needs to be executed in a certain type of environment. For example, an AutomationPlan could be created for testing a web application using Firefox, DB2, and Tomcat on Linux 64bit architecture. Another AutomationPlan could be created for testing the same web application using Safari, Oracle, and Websphere on Windows 32bit architecture. In terms of test coverage you might imagine that AutomationPlans could be created for executing this same test in many other combinations or permutations of browser/database/application server/OS. In the scenarios below, each such grouping is referred to as an execution environment.

The set of AutomationPlans for executing a single test case are often identical with the exception of their execution environment. In fact it is typical for a test architect to define their test cases in a generic way that abstracts out the environmental details and focuses instead on the inputs/outputs for the test cases and the expected precondition/postcondition of the system under test. Then at a later time the test architect may define or automatically generate a multitude of execution environments and create an AutomationPlan for executing each test case on each environment. The execution environments might also be defined in a separate system such as cloud provisioning or in a virtualization server, and then bound to the AutomationPlans during an execution setup/preparation phase or just in time before execution.

In this type of scenario it is convenient to conceptualize an execution environment as a resource having a URI and having its own lifecycle that is independent from any AutomationPlans that use it. This allows the Automation Provider to delegate the responsibilities of defining execution environments to an external provider, and allows a single execution environment to be bound to multiple AutomationPlans (or vice-versa). It may also provide an opportunity to treat execution environments as reconcilable resources.

Contents

• Automation Plan Execution Environments
  • Scenarios
    • Test architect defines execution environments and generates Automation Plans for Test Cases
    • Test architect generates Automation Plans for Test Cases using execution environments managed by an external provider (variation of previous scenario)
    • Automated Execution of an AutomationPlan using an execution environment
  • Example
  • AutomationPlan RDF/XML Sample

Scenarios

Test architect defines execution environments and generates Automation Plans for Test Cases

1. Test architect creates a catalog that defines various types of IT resources used for testing. For example, the catalog might contain various types of web browsers, databases, application servers, and operating systems.
2. Test architect generates a list of execution environments from a catalog of IT resources. This list of execution environments includes several different combinations of the types of resources in the catalog. For example, one such combination might be “Firefox/DB2/Tomcat/Linux”.
3. Test architect selects a set of execution environments from the list that will be covered by the overall testing effort.
4. Test architect creates a test case. The test case defines a precondition/postcondition and the expected input/output parameters for a particular test scenario.
5. Test architect generates a set of AutomationPlans for the Test Case, one for each of the execution environments
6. Test architect adds a reference (e.g. URI) from each AutomationPlan to its execution environment

Test architect generates Automation Plans for Test Cases using execution environments managed by an external provider (variation of previous scenario)

1. Test architect creates a test case. The test case defines a precondition/postcondition and the expected input/output parameters for a particular test scenario.
2. Test architect creates an AutomationPlan for the test case
3. Test architect consults an external provider for a list of available execution environments
4. Test architect selects an execution environment for the AutomationPlan
5. Test architect adds a reference (e.g. URI) from the AutomationPlan to the execution environment

Automated Execution of an AutomationPlan using an execution environment

1. Test engineer posts an AutomationRequest for an AutomationPlan that has a related execution environment.
2. Test agent (aka “adapter”) examines the execution environment for the AutomationPlan.
3. Test agent sets up or obtains access to the IT resources for the execution environment
4. Test agent executes the test case related to the AutomationPlan using the execution environment
5. Test agent posts an AutomationResult
6. (Optional) Test agent tears down the execution environment.

Example

AutomationPlan RDF/XML Sample

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:dcterms="http://purl.org/dc/terms/"
  xmlns:oslc="http://open-services.net/ns/core#"
  xmlns:oslc_auto="http://open-services.net/ns/auto#">

    <oslc_auto:AutomationPlan
      rdf:about="http://example.org/autoplans/1234">
     ...
      <oslc_auto:hasExecutionEnvironment rdf:resource="[environment URI]" />
     ...
    </oslc_auto:AutomationPlan>
</rdf:RDF>