2.4 JBoss Portal Test Plan

QA Portal test plan utilizes multiple testing methods and software.  The testing methods use of software as well as matrix numbers are displayed.  The test plan comprises of five main methods of testing which are described in greater detail below.

Pre-Conditions for testing:

• check out JBoss App Server from head in CVS (4.0.4)

• check out JBoss Portal from head in CVS (2.4)

• verify that 1.5 JDK is installed on the build machine

• build JBoss App Server and JBoss Portal

• download the App Server with appropriate Portal files to QA Labs

• start the App Server

• point a web browser to the App Server's location and verify the Portal appears in the browser

• click here for detailed install information

-

1. Performance Testing

Performance testing consists of simulating a single user performing multiple tasks on the Portal.  The results should set a standard for Scalability testing as well as be compared against competitors numbers and/or industry standards.

Pre-Conditions:

• Create and add multiple pages to the portal

• Create and add sub-pages to an existing page

• Create multiple portlets on each page

• Create a user from the admin account

• Create a grinder script which makes HTTP/HTTPS requests to the app server

Tests include:

• run the grinder script with 1 user for 5 runs

The script will simulate the following tasks with both a user and admin:

• log into Portal

• navigate through all the listed themes

• navigate to each page of the portal

• navigate to sub-pages of pages

• traverse through the portlets

• navigate to external sites (portletswap.com)

• log out of Portal

Results include:

• Mean Time - avg time to recieve full response from server

• Mean Time Standard Deviation - avg standard deviation time to recieve full response from server

• TPS - Avg number of transactions in a second time length

• Mean Response Length - Avg size of HTTP response from server (bytes)

• Response Bytes per Second - Avg number of bytes per second recieved from server (measures comsumption bandwidth)

• Response Errors - Number of response error codes (404, 500, etc.)

-

2. Scalability Testing

Scalability Testing is performance testing with a high concurrency of users hitting the server while executing a low number of runs.

Pre-Conditions:

• Verify a grinder script from Performance test exists whose properties file can be slightly modified to allow for the appropriate number of threads(users) and runs (Grinder script is created during Performance pre-conditions)

• Modify the grinder script properties file to run with 1000 threads

Tests include:

• run the modified grinder script with 1000 users hitting the server simultaneously for 5 runs

Results include:

• Identical type of results from Performance tests

• scalability testing results should show little or no change when compared to performance testing results

-

3. Soak Testing

Soak testing is performance testing with a high concurrency of users over a large period of time.

Pre-Conditions:

• Verify a grinder script from Performance test exists whose properties file can be slightly modified to allow for the appropriate number of threads(users) and runs (Grinder script is created during Performance pre-conditions)

• Determine the amount of time in minutes(X) it takes to execute a complete run of 1000 users

• Modify the grinder script properties file to run with 1000 threads for Z runs where Z=(60/X)24

Tests include:

• run the modified grinder script with 1000 users hitting the server simultaneously for 24 hours

Results include:

Review any information pertaining to the following:

• memory leaks

• garbage collection issues

• crashes

-

4. Integration Testing

Integration testing is the testing of two or more integrated components.  Scenerios test the integration points of JEMS components, many of which are executed when testing the application server.  An application could be written which would automate integration testing of the components.

Portal Integration testing consists of functional testing as well.  Functional testing is created from a user's perspective and verifies system behavior.  This will validate the integration between Portal, the app server and all other JEMS components which comprise of the Portal package.  Pre-Conditions, Tests, and Results vary for each feature.  Below are a few new features in Portal 2.4 which can be functionally tested.

New Features

JBPORTAL-467

Pre-Conditions:

• App Server is running

Tests:

• login to the Portal

Results:

• verify the login theme is identical to the Portal theme

JBPORTAL-554

Pre-Conditions:

• App Server is running

• upload a file to CMSAdmin

Tests:

• (1)navigate to management page in the Portal as Admin and click on the eyeball

• (2)browse to the specific location of the document

Results:

• (1)user should be able to view the document

• (2)user should be able to download the document

JBPORTAL-535

Pre-Conditions:

• App Server is running

• Create a page

• Edit core\src\resources\portal-core-sar\conf\data.xml by modifying the value of the property "portal.defaultObjectName" to the name of the page created

Tests:

• restart the Portal

Results:

• default page should now be the created page

JBPORTAL-405

Pre-Conditions:

• App Server is running

• User is denied access to a portlet

Tests:

• User navigates to the page with the access denied portlet (access denied message should be displayed on the portlet)

• Admin modifies the setting to "hide portlet"

• User navigates to the page with the access denied portlet

Results:

• Portlet should not be visible to the user

• due to time constraints, currently there are no plans to perform integration testing on Portal 2.4

The following tests cases test the functionality of the admin management portlet.

Management Portlet

Positive Test:

• build jboss-portal.sar and deploy to the server's deploy directory

• jboss-portal.sar should contain

- portal class file

- jsp (if necessary)

- updated portal.xml

• create a portal instance from management portlet

• add the portlet instance to a page

• portlet should now reside on the page to which it was added

Transition Portlet from 2.2 to 2.4:

• perform the positive test

• deploy jboss-portal.sar to the 2.4 server

Update at Runtime:

• perform steps 1 and 2 from positive test while the server is running

• portlet should be available to create an instance

Security Tests:

• modify the security level of a portlet instance

- default view

- view only when logged in with admin

- view only when logged in with user

• modify the security level of a portlet

• verify portlet instance security overrides portlet security

• modify the security level of a page

- default view

- view only when logged in with admin

- view only when logged in with user

• modify the security level of a portal

CMS Manager Tests

Create a Folder:

• verify folder can be added

Create a File:

• with .html ext

• with .txt ext

• with .doc ext

Edit a File:

• verify html and txt files can be edited

- perform the following for both html and txt files

- after selecting an html or txt file, there should be an edit icon

- select the edit icon and modify the file

- save the modified file without changing the file name

- a different version should be created

- select the modified version and verify the it contains the information added

• verify doc files cannot be edited

- select a doc file

- there should not be an edit column

Upload a File:

- click on the upload icon

- select an available destination and fill in info

- verify the file was uploaded to the appropriate folder

-

5. Availability Testing

Availability testing is testing the system's ability to continue normal operations while failures occur during high concurrency usage.  The scalability tests will be used to provide high thread volume while failure scenerios are introduced to the system.  The main goal is to reduce and/or eliminate downtime by minimizing repair time.

Pre-Conditions:

• Verify the scalability script is available to run for a specified amount of time (dependent on which test is run) to execute the error conditions

Test include:

• unplugging network cable

• generate exceptions

• database drive fills up

• clustering functionality error

-

Test Time Frame (Time in days)

Test Matrix (Time in hours)

Test Software