cd <TS_HOME>/bin
ant gui
Executing Tests |
Previous | Next | Contents |
The XML Web Services TCK uses the JavaTest harness to execute the tests in the test suite. For detailed instructions that explain how to run and use JavaTest, see the JavaTest User’s Guide and Reference in the documentation bundle.
This chapter includes the following topics:
Note
|
The instructions in this chapter assume that you have installed and configured your test environment as described in Chapter 3, "Installation," and Chapter 4, "Setup and Configuration,", respectively. |
As explained in Appendix B, "Packaging the Test Applications in Servlet-Compliant WAR Files With VI-Specific Information," the XML Web Services TCK introduces the concept of repackaging the TCK tests.
There are two general ways to run the XML Web Services TCK using the JavaTest harness software:
Through the JavaTest GUI
From the command line in your shell environment
Note
|
The |
As explained in Appendix B,
the JAX-WS TCK introduces the
concept of rebuilding the TCK tests. To provide the user an ability to
run the TCK against the Vendor Implementation and the Reference
Implementation, the use of the keywords
feature (in GUI mode) or
option `-Dkeywords=`value (in command line mode) is provided.
By default, the TCK is configured to run all tests in both directions
for all the tests except the signature tests. Setting keywords
allows
the user to change which tests will be run.
Setting the keywords
property to all
(the default) does not filter
out any tests, and results in the prebuilt tests to be run in the
forward direction, and the Vendor rebuilt tests to be run in the reverse
direction.
Setting keywords
to forward
causes the prebuilt tests to be run in
the forward direction only.
Setting keywords
to reverse
causes the Vendor rebuilt tests to be
run in the reverse direction only.
Refer to the JavaTest User’s Guide and Reference in the documentation
bundle for information regarding how to configure the keywords
feature
in GUI mode. For command line mode, add the following to your command
line -Dkeywords=``value
, where value is either forward
, reverse
,
or all
.
Change to any subdirectory under <TS_HOME>/src/com/sun/ts/tests
.
Start JavaTest using the following command:
<TS_HOME>/tools/ant/bin/ant [-Dkeywords=forward|reverse|all] runclient
Note
|
The |
Example 5-1 Running JAX-WS TCK Signature Tests
To run the JAX-WS TCK signature tests, enter the following commands:
cd <TS_HOME>/src/com/sun/ts/tests/signaturetest/jaxws
<TS_HOME>/tools/ant/bin/ant [-Dkeywords=forward|reverse|all] runclient
Example 5-2 Running a Single Test Directory
To run a single test directory in the forward
direction, enter the
following commands:
cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api/javax_xml_ws/Dispatch
<TS_HOME>/tools/ant/bin/ant -Dkeywords=forward runclient
Example 5-3 Running a Subset of Test Directories
To run a subset of test directories in the reverse
direction, enter
the following commands:
cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api
<TS_HOME>/tools/ant/bin/ant -Dkeywords=reverse runclient
Use the following modes to run a subset of the tests:
From the JavaTest main menu, click Configure, then click Change
Configuration, and then click Tests to Run.
The tabbed Configuration Editor dialog box is displayed.
Click Specify from the option list on the left.
Select the tests you want to run from the displayed test tree, and
then click Done.
You can select entire branches of the test tree, or use Ctrl+Click
or
Shift+Click
to select multiple tests or ranges of tests, respectively,
or select just a single test.
Click Save File.
Click Run Tests, and then click Start to run the tests you selected.
Alternatively, you can right-click
the test you want from the test tree
in the left section of the JavaTest main window, and choose Execute
These Tests from the menu.
Click Report, and then click Create Report.
Specify the directory in which the JavaTest test harness will write
the report, and then click OK
A report is created, and you are asked whether you want to view it.
Click Yes to view the report.
Change to the directory containing the tests you want to run.
Start the test run by executing the following command:
ant runclient
The tests in the directory and its subdirectories are run.
You can run certain tests in batch mode based on the test’s prior run
status by specifying the priorStatus
system property when invoking
ant
Invoke ant
with the priorStatus
property.
The accepted values for the priorStatus
property are any combination
of the following:
fail
pass
error
notRun
For example, you could run all the XML Web Services tests with a status of failed and error by invoking the following commands:
ant -DpriorStatus="fail,error" runclient
Note that multiple priorStatus
values must be separated by commas.
Some test scenarios are designed to ensure that the configuration and deployment of all the prebuilt XML Web Services TCK tests against one Compatible Implementation are successful operating with other compatible implementations, and that the TCK is ready for compatibility testing against the Vendor and Compatible Implementations.
Verify that you have followed the configuration instructions in Section 4.1, "Configuring Your Environment to Run the TCK Against the Compatible Implementation."
If required, verify that you have completed the steps in Section 4.3.2, "Deploying the Prebuilt Archives."
Run the tests, as described in Section 5.1, "Starting JavaTest," and, if desired, Section 5.2, "Running a Subset of the Tests."
This test scenario is one of the compatibility test phases that all Vendors must pass.
Verify that you have followed the configuration instructions in Section 4.2, "Configuring Your Environment to Repackage and Run the TCK Against the Vendor Implementation."
If required, verify that you have completed the steps in Section 4.3.3, "Deploying the Test Applications Against the Vendor Implementation."
Run the tests, as described in Section 5.1, "Starting JavaTest," and, if desired, Section 5.2, "Running a Subset of the Tests."
A set of report files is created for every test run. These report files can be found in the report directory you specify. After a test run is completed, the JavaTest harness writes HTML reports for the test run. You can view these files in the JavaTest ReportBrowser when running in GUI mode, or in the web browser of your choice outside the JavaTest interface.
To see all of the HTML report files, enter the URL of the report.html
file. This file is the root file that links to all of the other HTML
reports.
The JavaTest harness also creates a summary.txt
file in the report
directory that you can open in any text editor. The summary.txt
file
contains a list of all tests that were run, their test results, and
their status messages.
Use the following modes to create test reports:
From the JavaTest main menu, click Report, then click Create Report.
You are prompted to specify a directory to use for your test reports.
Specify the directory you want to use for your reports, and then
click OK.
Use the Filter list to specify whether you want to generate reports for
the current configuration, all tests, or a custom set of tests.
You are asked whether you want to view report now.
Click Yes to display the new report in the JavaTest ReportBrowser.
Specify where you want to create the test report.
To specify the report directory from the command line at runtime, use:
ant -Dreport.dir="report_dir"
Reports are written for the last test run to the directory you specify.
To specify the default report directory, set the report.dir
property in <TS_HOME>/bin/ts.jte
.
For example:
report.dir="/home/josephine/reports"
To disable reporting, set the report.dir
property to "none"
,
either on the command line or in <TS_HOME>/bin/ts.jte
.
For example:
ant -Dreport.dir="none"
Use the following modes to view an existing test report:
From the JavaTest main menu, click Report, then click Open Report.
You are prompted to specify the directory containing the report you want
to open.
Select the report directory you want to open, and then click Open.
The selected report set is opened in the JavaTest ReportBrowser.
This test scenario is ensures that the configuration and deployment of all the prebuilt JAX-WS TCK tests against the JAX-WS Reference Implementation are successful, and that the TCK is ready for compatibility testing against the Vendor and Reference Implementations.
Verify that you have followed the configuration instructions in Configuring Your Environment to Run the JAX-WS TCK Against the JAX-WS Reference Implementation.
Specify forward
for the keywords
option.
Verify that you have completed the steps in Deploying the JAX-WS TCK Prebuilt Archives.
Run the tests, as described in Starting JavaTest and, if desired, Running a Subset of the Tests.
This test scenario is one of the compatibility test phases that all Vendors must pass. This ensures that the prebuilt JAX-WS TCK tests built against the JAX-WS RI can be successfully run against the Vendor Implementation (VI).
Verify that you have followed the configuration instructions in Configuring Your Environment to Run the JAX-WS TCK Against the Vendor Implementation.
Specify forward
for the keywords
option.
Verify that you have completed the steps in Deploying the JAX-WS TCK Prebuilt Archives
Run the tests, as described in Starting JavaTest and, if desired, Running a Subset of the Tests.
This test scenario is one of the compatibility test phases that all Vendors must pass. This ensures that the JAX-WS TCK tests that are rebuilt using the Vendor’s toolset can be successfully run against the Reference Implementation.
Verify that you have followed the configuration instructions in Configuring Your Environment to Rebuild and Run the JAX-WS TCK Against the JAX-WS RI.
Refer to Appendix B, to learn about rebuilding the JAX-WS TCK tests.
Specify reverse
for the keywords
option.
Verify that you have completed the steps in Deploying the Rebuilt JAX-WS TCK Tests Against the JAX-WS Reference Implementation.
Run the tests, as described in Starting JavaTest and, if desired, Running a Subset of the Tests.
Specify all
for the keywords
option.
Verify that you have completed the steps in Deploying the JAX-WS TCK Prebuilt Archives.
Verify that you have completed the steps in Deploying the Rebuilt JAX-WS TCK Tests Against the JAX-WS Reference Implementation
Run the tests, as described in Starting JavaTest and, if desired, Running a Subset of the Tests.
Previous | Next | Contents |