sun.com docs.sun.com My Sun Worldwide Sites

Previous Previous     Contents     Next Next

1.2.3 JavaTest Harness

The JavaTest™ harness version 3.2.1 is a set of tools designed to run and manage test suites on different Java platforms. The JavaTest harness can be described as both a Java application and a set of compatibility testing tools. It can run tests on different kinds of Java platforms and it allows the results to be browsed online within the JavaTest GUI, or offline in the HTML reports that the JavaTest harness generates.

The JavaTest harness includes the applications and tools that are used for test execution and test suite management. It supports the following features:

  • Sequencing of tests, allowing them to be loaded and executed automatically

  • Graphic user interface (GUI) for ease of use

  • Automated reporting capability to minimize manual errors

  • Failure analysis

  • Test result auditing and auditable test specification framework

  • Distributed testing environment support

To run tests using the JavaTest harness, you specify which tests in the test suite to run, how to run them, and where to put the results as described in Chapter 4.

1.2.4 TCK Compatibility Test Suite

The test suite is the collection of tests used by the JavaTest harness to test a particular technology implementation. In this case, it is the collection of tests used by the SAAJ TCK 1.3 to test a SAAJ 1.3 implementation. The tests are designed to verify that a licensee's runtime implementation of the technology complies with the appropriate specification. The individual tests correspond to assertions of the specification.

The tests that make up the TCK compatibility test suite are precompiled and indexed within the TCK test directory structure. When a test run is started, the JavaTest harness scans through the set of tests that are located under the directories that have been selected. While scanning, the JavaTest harness selects the appropriate tests according to any matches with the filters you are using and queues them up for execution.

1.2.5 Exclude Lists

Each version of a TCK includes an Exclude List contained in a .jtx file. This is a list of test file URLs that identify tests which do not have to be run for the specific version of the TCK being used. Whenever tests are run, the JavaTest harness automatically excludes any test on the Exclude List from being executed.

A licensee is not required to pass any test--or even run any test--on the Exclude List. The Exclude List file, <TS_HOME>/bin/ts.jtx, is included in the SAAJ TCK.


Note - From time to time, updates to the Exclude List are made available on the Java Partner Engineering Web site. You should always make sure you are using an up-to-date copy of the Exclude List before running the SAAJ TCK to verify your implementation.


A test might be in the Exclude List for reasons such as:

  • An error in an underlying implementation API has been discovered which does not allow the test to execute properly.

  • An error in the specification that was used as the basis of the test has been discovered.

  • An error in the test itself has been discovered.

  • The test fails due to a bug in the tools (such as the JavaTest harness, for example).

In addition, all tests are run against the Sun Microsystems reference implementations. Any tests that fail when run on a reference Java platform are put on the Exclude List. Any test that is not specification-based, or for which the specification is vague, may be excluded. Any test that is found to be implementation dependent (based on a particular thread scheduling model, based on a particular file system behavior, and so on) may be excluded.


Note - Licensees are not permitted to alter or modify Exclude Lists. Changes to an Exclude List can only be made by using the procedure described in SOAP with Attachments API for Java 1.3 Test Appeals Process.


1.2.6 SAAJ TCK Configuration

You need to set several variables in your test environment, modify properties in the <TS_HOME>/bin/ts.jte file, and then use the JavaTest harness to configure and run the SAAJ tests, as described in Chapter 4.

1.3 Getting Started With the SAAJ TCK

This section provides an general overview of what needs to be done to install, set up, test, and use the SAAJ TCK. These steps are explained in more detail in subsequent chapters of this guide.

ProcedureTo get started with the SAAJ TCK

  1. Make sure that the following software has been correctly installed on the system hosting the JavaTest harness:

    • Java SE 5

    • Apache Tomcat 5.5.x

    • SAAJ TCK version 1.3

    • An implementation of the SAAJ 1.3 specification

    See the documentation for each of these software applications for installation instructions.

  2. Install the SAAJ TCK 1.3 software.

    See Chapter 3 for additional information.

  3. Set up the SAAJ TCK software.

    See Chapter 4 for details about the following steps.

    1. Set up your shell environment.

    2. Modify the required properties in the <TS_HOME>/bin/ts.jte file.

    3. Configure the JavaTest harness.

  4. Test the SAAJ 1.3 implementation.

    Test the SAAJ implementation installation by running the test suite. See Chapter 5.

Previous Previous     Contents     Next Next
Company Info Contact Terms of Use Privacy Copyright 1994-2006 Sun Microsystems, Inc.