Task 4.4 is responsible for the test and validation of middleware to be installed in the CrossGrid production testbed. The verification of the middleware by task 4.4 is the last step of the quality assurance process before the actual deployment of the software in the production testbed. 

Aim of the validation and requirements
In this context the validation aim is to verify integrated and packaged middleware ready to be installed in the testbed (preferably in rpm format). These products must be delivered for testing with proper documentation including:

• User manuals
• Installation manuals
• Development manuals
• Software requirement documents
• Software design documents

The middleware must also be accompanied by the corresponding unit tests.

The validation covers all middleware to be installed in the CrossGrid production testbed independently from being developed inside of the project or having an external origin (eg. EDG, Globus). However the middleware must have passed by the WP4 integration.

The validation and deployment processes
The validation  process covers the documentation, the installation and the software itself. Basically the documentation is followed to install the software in systems dedicated or allocated for use by the "Test and validation" testbed. The software is then tested using unit tests provided the the developers that are modified or used jointly with task 4.4 tests to verify the correct behavior of the software. System tests with the software integrated with the remaining testbed components are then performed. These tests should be performed first at local level and then between sites. Finally stress tests are performed exercising the component. The stress tests are basically system tests used in an intensive way. The system and stress tests are performed with the software integrated with the remaining middleware components, they try to mimic real usage situations and verify the software behavior under "stress" conditions. During the tests process including the installation, configuration and usage the manuals provided are followed and verified. 

The software validation process followed by task 4.4 is shown in the following diagram.  

As it can be observed the WP4 integration team is responsible for the preparation and integration of testbed software distributions. Once a distribution of a new software package is considered ready by the WP4 integration team a  test request is sent to the task 4.4. For that purpose a test request form is available at the task 4.4 web site. Test requests for individual packages can be also accepted, this is in accordance with the testbed incremental evolution approach followed by WP4. 

Test requests will only be accepted for components that have passed by the WP4 integration and are accompanied by the proper documentation. 

Knowledge about the CrossGrid release and deployment process is essential to understand when to request a validation. This process is described in the next diagrams. There are two possible scenarios.

Normal deployment - is the most frequent deployment track. It applies to new packages that have never been validated before or to new package releases.

Fast deployment - is a deployment track where the package does not pass through the "test and validation" procedure. It is applicable only to minor bug corrections and security corrections. Under this procedure the developer ask the Iteam to deploy the package in the production testbed with being validated. The developer must provide the following information:

• A list of the corrections performed.
• The bugtraker numbers corresponding to the corrected bugs.
• The list of differences obtained through the command.
       cvs  -diff  old-version  new-version

Based on the above information the Iteam decides if the package can be deployed directly or if a validation is required. The validation team can provide recommendations on whether certain issues when corrected should be validated.

The validation
Once a test request is received it is scheduled according with its priority and the available personnel and computational ("test and validation testbed") resources. The testers are then appointed.

The validation itself starts with the software installation in a separated "Test and Validation" testbed following the documentation provided with the software. Issues detected on the installation, actual software and corresponding documentation are reported back to the integration team and to the developers. During the test process interaction with the developers and integrators will occur whenever necessary.

The software validation will mimic as close as possible the real deployment in the production testbed and the software usage integrated with the other existing components

If the software is considered stable it can be deployed in the production testbed. Finally after deployment the behavior of the software services is closely monitored to verify its stability. 

The actual test and validation
The information in this section is mostly important for the middleware testers.
The actual validation will cover the following points:

• DOCUMENTATION
  • Whether it is technically correct.
  • Whether there is something missing.
  • Whether its is clear and easy to understand
  • Typos and language errors.
• USABILITY
  • Whether the software is easy to deploy.
  • Whether the software usage is easy to understand.
  • Whether the software is easy to use.
  • Whether the user interfaces produce correct output.
  • Whether the interactive response speed is acceptable.
• INSTALLATION
  • Whether the software installs correctly.
  • Whether the installation procedure is compliant with the CrossGrid WP4 development guide.
• FUNCTIONALITY
  • Whether the announced functionalities are present.
  • Whether the software works without flaws.
    • Unit tests
    • System tests (local and between sites)
    • Stress tests (local and between sites)
  • Whether the software is compliant with the design and user requirements.
• COMPATIBILITY
  • Whether the software is compliant with the other components of the distribution.
  • Whether the software is compatible with the EDG middleware.
  • Whether the software is backwards compatible.
• SECURITY
  • Verify and register security related information such as:
    • Used  TCP/UDP port numbers and possible firewall issues.
    • Used libraries/packages and related problems.
    • Used data access and authentication/authorization methods.
    • Certificate usage issues.
    • Installed file attributes and ownership.
• PREVIOUSLY REPORTED ISSUES
  • Verify that previously reported issues that have been marked as corrected are not present.

Due to the limited time available for performing the validation requests the procedure may have to focus mostly on the documentation, installation and functionality.

Problems classification
Problems found while performing the validation tests will be classified regarding their severity and priority, these two attributes define the urgency and impact of the problem and help to determine which issues must be addressed first.

For each severity level described below a guideline action is recommended. The severity classification is as follows:

• Critical – When problem causing a failure of the complete software system, subsystem or program within the system is found.

ACTION: The component should not be released. Further tests on the component and components depending on it will not be performed until the problem is corrected.

• High – When a problem that does not cause a failure, but causes the system to produce incorrect, incomplete, inconsistent results or impairs the system usability is found.

ACTION: Developers MUST perform the required corrections but the component can pass to the next phase. A deadline will be given to the developers to make the corrections;

• Medium – When a problem that does not cause a failure, does not impair usability, and does not interfere in the fluent work of the system and programs has been found.

ACTION: Corrections should be performed but since the component critical functionalities have been successfully tested and the component can be deployed. However the problem must be mentioned in the documentation.

• Low – When the problem found is aesthetic is an enhancement or is a result of non-conformance to a standard.

ACTION: The problem does not interfere with the functionality or operation therefore the component can be deployed. However the problem must be mentioned in the documentation.

 The priority classification is as follows:

• Immediate –  When the problem should be resolved immediately.
• High – When the problem should be resolved as soon as possible in the normal course of development activity, before the software is released.
• Medium – When the problem should be repaired after all other more serious problems have been fixed.
• Low – When the problem can be resolved in a future major system revision or not resolved at all.