Archive

Archive for June, 2008

Tools for Software Test Automation

The following tools are provide automated functional testing and performance testing for environments including Dot Net, Java, CORBA, WWW, WAP, client/server, UNIX, and Windows & Etc …

1 .Segue Software delivers the industry’s premier e-business reliability products and solutions to help you address the risks and complexities that can affect the performance and quality of your e-business system.

  • SilkTest – Automated functional and regression testing
  • SilkPerformer – Automated load and performance testing
  • SilkMonitor – 24×7 monitoring and reporting of Web, application and database servers
  • SilkPilot – Unit testing of CORBA objects
  • SilkObserver – End-to-end transaction management and monitoring for CORBA applications
  • SilkMeter – Access control and usage metering
  • SilkRealizer – Scenario testing and system monitoring
  • SilkRadar – Automated defect tracking

2. Mercury offers a suite of solutions that automate testing and quality assurance for client/server software and systems, e-business applications, and enterprise resource planning applications.

  • QuickTest ProfessionalTM – E-business functional testing
  • LoadRunner® – Enterprise load testing
  • TestDirectorTM – Integrated test management
  • WinRunner® – Test automation for the enterprise

Recover Scenarios in QTP

What are Recover Scenarios?

While executing your scripts you may get some UNEXPECTED/UNPREDICTABLE errors. (like printer out of paper). To “recover” the test (and continue running) from these unexpected errors you use Recovery Scenarios.

Next question arises,

When to use “on error resume next” or programmatic handling of errors VS Recovery Scenarios ?

If you can predict that a certain event may happen at a specific point in your test or component, it is recommended to handle that event directly within your test or component by adding steps such as If statements or optional steps or “on error resume next”, rather than depending on a recovery scenario. Using Recovery Scenarios may result in unusually slow performance of your tests.They are designed to handle a more generic set of unpredictable events which CANNOT be handled programmatically.

For Example:

A recovery scenario can handle a printer error by clicking the default button in the Printer Error message box.

You cannot handle this error directly in your test or component, since you cannot know at what point the network will return the printer error. You could try to handle this event in your test or component by adding an If statement immediately after the step that sent a file to the printer, but if the network takes time to return the printer error, your test or component may have progressed several steps before the error is displayed. Therefore, for this type of event, only a recovery scenario can handle it.

I would not go into details of how to create files and how to define them since they are fully covered in QTP Documentation. Mercury QuickTest Professional User’s Guide > Working with Advanced Testing Features > Defining and Using Recovery Scenarios >

What constitute Recovery Scenarios?

A recovery scenario consists of the following:

  • Trigger Event. The event that interrupts your run session. For example, a window that may pop up on screen, or a QTP run error.
  • Recovery Operations. The operations to perform to enable QTP to continue running the test after the trigger event interrupts the run session. For example, clicking an OK button in a pop-up window, or restarting Microsoft Windows.
  • Post-Recovery Test Run Option. The instructions on how QTP should proceed after the recovery operations have been performed, and from which point in the test QTP should continue, if at all. For example, you may want to restart a test from the beginning, or skip a step entirely and continue with the next step in the test.

Recovery scenarios are saved in recovery scenario files having the extension .rs. A recovery scenario file is a logical collection of recovery scenarios, grouped according to your own specific requirements.

Is there a method to programmatically call them?

By default, QTP checks for recovery triggers when an error is returned during the run session. You can use the Recovery object’s method to force QTP to check for triggers after a specific step in the run session.

For a complete list go to QTP Documentation > Quick Test Advanced References > Quick Test Automation > Recovery Object

Limitations of DataTable QTP

The limitations listed below are specifically for QTP 8.2:

Maximum worksheet size—65,536 rows by 256 columns
Column width—0 to 255 characters
Text length—16,383 characters
Formula length—1024 characters
Number precision—15 digits
Largest positive number—9.99999999999999E307
Largest negative number— -9.99999999999999E307
Smallest positive number—1E-307
Smallest negative number— -1E-307
Maximum number of names per workbook—Limited by available memory
Maximum length of name—255
Maximum length of format string—255
Maximum number of tables (workbooks)—Limited by system resources (windows and memory)

Some Useful Tips with QTP

We Sharing some of the useful tips on quick test professional. I used while working and found from gds (Project). I urge the readers to share their experiences/tips they have used while working on QTP. You can use the comments section to do the same.

1. How to add a constant number in a datatable?

This is more to do with MS excel then QTP!! but useful to know because at times it becomes frustrating to the novices.
Just append ‘ to the number
Ex: if you wish to enter 1234567 in datatable then write it as ‘1234567

2. How can i check if a parameter exists in DataTable or not?

The best way would be to use the below code:
on error resume next
val=DataTable(“ParamName”,dtGlobalSheet)
if err.number<> 0 then
‘Parameter does not exist
else
‘Parameter exists
end if

3. How can i check if a checkpoint passes or not?
chk_PassFail = Browser(…).Page(…).WebEdit(…).Check (Checkpoint(“Check1”))
if chk_PassFail then
MsgBox “Check Point passed”
else MsgBox “Check Point failed”
end if

4. My test fails due to checkpoint failing, Can i validate a checkpoint without my test failing due to checpoint failure?
Reporter.Filter = rfDisableAll ‘Disables all the reporting stuff
chk_PassFail = Browser(…).Page(…).WebEdit(…).Check (Checkpoint(“Check1”))
Reporter.Filter = rfEnableAll ‘Enable all the reporting stuff
if chk_PassFail then
MsgBox “Check Point passed”
else
MsgBox “Check Point failed”
end if

5. What is the difference between an Action and a function?

Action is a thing specific to QTP while functions are a generic thing which is a feature of VB Scripting. Action can have a object repository associated with it while a function can’t. A function is just lines of code with some/none parameters and a single return value while an action can have more than one output parameters.

6) Where to use function or action?
Well answer depends on the scenario. If you want to use the OR feature then you have to go for Action only. If the functionality is not about any automation script i.e. a function like getting a string between to specific characters, now this is something not specific to QTP and can be done on pure VB Script, so this should be done in a function and not an action. Code specific to QTP can also be put into an function using DP. Decision of using function/action depends on what any one would be comfortable using in a given situation.

7) When to use a Recovery Scenario and when to us on error resume next?
Recovery scenarios are used when you cannot predict at what step the error can occur or when you know that error won’t occur in your QTP script but could occur in the world outside QTP, again the example would be “out of paper”, as this error is caused by printer device driver. “On error resume next” should be used when you know if an error is expected and dont want to raise it, you may want to have different actions depending upon the error that occurred. Use err.number & err.description to get more details about the error.

8) How to use environment variable?
A simple defintion could be… it is a variable which can be used across the reusable actions and is not limited to one reusable action.
There are two types of environment variables:
1. User-defined
2. Built-in
We can retrieve the value of any environment variable. But we can set the value of only user-defined environment variables.

To set the value of a user-defined environment variable:
Environment (VariableName) = NewValue

To retrieve the value of a loaded environment variable:
CurrValue = Environment (VariableName)

Example
The following example creates a new internal user-defined variable named MyVariable with a value of 10, and then retrieves the variable value and stores it in the MyValue variable.

Environment.Value(“MyVariable“)=10
MyValue=Environment.Value(“MyVariable“)

9) What are the files and subfolders of a QuickTest Professional test?
The files and folders hold binary and text data that are required for the test to run successfully.
The following table provides a description, the type, and comments regarding the files that make up a QuickTest Professional test.

File Name

Description

Type

Comments Regarding File

Test.tsp

Test settings

Binary

Do not edit

Default.xls

Data table parameters

Excel similar

Can be edited using Excel

Parameters.mtr

Parameterization information

Binary

Do not edit

Action

Action folder (See other table)

<!–[if !supportEmptyParas]–> <!–[endif]–>

<!–[if !supportEmptyParas]–> <!–[endif]–>

Default.cfg

Load test configuration file

Text

Do not edit

Default.prm

Load test configuration file

Text

Do not edit

Default.usp

Load test configuration file

Text

Do not edit

.usr

Load test configuration file

Text

Do not edit

Thick_usr.dat

Load test configuration file

Text

Do not edit

Thin_usr.dat

Load test configuration file

Text

Do not edit

Files within Action folder:

File Name

Description

Type

Comments Regarding File

Script.mts

Action script

Text

Edit text preceding the @@ sign only

Resource.mtr

Object Repository

Binary

Do not edit

Snapshots

Active screen files

Folder

Do not edit

There are few more files extensions like

.MTB Batch File
.LCK Locked

10) How to rename a checkpoint (QTP 9.0)?
Example:
Window(“Notepad”).WinEditor(“Edit”).Check CheckPoint(“Edit”)
In the above example, the user would like to change the name of the CheckPoint object from “Edit” to something more meaningful.
Note:
This functionality is new to QuickTest Professional 9.0.This is not available for QTP 8.2 and below.
1. Right-click on the Checkpoint step in the Keyword View or on the Checkpoint object in Expert View.
2. Select “Checkpoint Properties” from the pop-up menu.
3. In the Name field, enter the new checkpoint name.
4. Click . The name of the checkpoint object will be updated within the script.
Example:
Window(“Notepad”).WinEditor(“Edit”).Check CheckPoint(“NewCheckPointName”)
Note:
You must use the QuickTest Professional user interface to change the name of the checkpoint. If you manually change the name of the checkpoint in the script, QuickTest Professional will generate an error during replay. The error message will be similar to the following:

“The “” CheckPoint object was not found in the Object Repository. Check the Object Repository to confirm that the object exists or to find the correct name for the object.”

The CheckPoint object is not a visible object within the object repository, so if you manually modify the name, you may need to recreate the checkpoint to resolve the error.

11) Does QuickTest Professional support Internet Explorer 7.0?
QuickTest Professional 9.1
QuickTest Professional 9.1 supports Microsoft Internet Explorer 7.0 Beta 3. Internet Explorer version 7.0 is now certified to work and to be tested with QuickTest Professional version 9.1.
QuickTest Professional 9.0
QuickTest Professional 9.0 supports Internet Explorer 7.0 Beta 2.
QuickTest Professional 8.2 and below
QuickTest Professional 8.2 and below do not include support for Internet Explorer 7.0.

Does QuickTest Professional support Firefox?
QuickTest Professional 9.1 and 9.2
QuickTest Professional 9.1 provides replay support for Mozilla Firefox 1.5 and Mozilla Firefox 2.0 Alpha 3 (Alpha-level support for Bon Echo 2.0a3).
Notes:

QuickTest Professional 9.1 will not record on FireFox. You can record a test on Microsoft Internet Explorer and run it on any other supported browser, such as FireFox.

The .Object property for web objects is not supported in FireFox.

QuickTest Professional 9.0
QuickTest Professional 9.0 provides replay support for Mozilla FireFox 1.5.
Notes:

QuickTest Professional 9.0 will not record on FireFox. You can record a test on Microsoft Internet Explorer and run it on any other supported browser, such as FireFox.

The .Object property for web objects is not supported in FireFox.

QuickTest Professional 8.2 and below

QuickTest Professional 8.2 and below do not have support for Firefox.

12. Problem
After Quick Test Professional is started, Windows Media will not start. It returns the error message “wmplayer.exe has generated errors and will be closed by Windows. You will need to restart the program. An error log is being created.”
If you start Window’s Media Player first, it will continue to work normally after starting QuickTest Professional.
Solution:
Include the Windows Media Player’s executable in the NoBBTApps section of the mic.ini file.
1. Close QuickTest Professional.
2. Go to \bin\mic.ini.
3. Include wmplayer.exe in the NoBBTApps section of mic.ini file.

Example:
[NoBBTApps]
wmplayer.exe=rek
4. Save the mic.ini file and restart QuickTest Professional.

13. What is the lservrc file in QTP?
The lservrc file contains the license codes that have been installed
The lservrc file contains the license codes that have been installed. Whenever a new license is created, the license code is automatically added to this file. The lservrc file is a text file, with no extension.

File Location:

1. For a Concurrent (Floating) license installation:
#server installation directory#\#language#”

Example:
C:\Program Files\XYZ Technologies\ABC Server\English\lservrc
2. For a Seat (Stand-alone) license installation:
#AQT/QTP installation directory#\bin”

Example:
C:\Program Files\Mercury Interactive\QuickTest Professional\Bin\lservrc

14. What to do if you are not able to run QTP from quality center?
This is for especially for newbies with QTP.
Check that you have selected Allow other mercury products to run tests and components from Tools–> Options–> Run Tab.

15. Does QuickTest Professional support Macintosh operating systems?
No, QTP is not expected to run on this OS. Only on Windows.

Categories: QTP Tags: ,

Regression Testing

Regression Testing

 

As some one has said, changes are the only thing constant in this world. It holds true for software also, existing software are either being changed or removed from the shelves. These changes could be due to any reason. There could be some critical defects which need to be fixed; there could be some enhancements which have to happen in order to remain in the competition. 

Regression testing is done to ensure that enhancement, defect fixes or any other changes made to the software have not broken any existing functionality.

 

Regression testing is very important, because in most places these days iterative development is used. In iterative development, shorter cycle is used with some functionality added in every cycle. In this scenario, it makes sense to have regression testing for every cycle to make sure that new features are not breaking any existing functionality.

Whenever there are any changes in the system, regression testing is performed. Regression testing can be performed for the complete product or it can be selective. Normally, full regression cycle is executed during the end of testing cycle and partial regression cycle is executed between the test cycles. During the regression cycle it becomes very important to select the proper test cases to get the maximum benefits. Test cases for regression testing should be selected based on the knowledge of 

  • What defect fixes, enhancement or changes have gone into the system?
  • What is the impact of these changes on the other aspect of system?

Focus of the regression testing is always on the impact of the changes in the system. In most of the organizations, priority of the regression defect is always very high. It is normally included in the exit criteria of the product that it should have zero regression defects. 

Regression testing should always happen after the sanity or smoke testing. Sanity/Smoke can be defined as the type of testing which make sure that software is in testable state. Normally, sanity test suite will contain very basic and core test cases. These test cases decide the quality of the build and any failure in the sanity test suite should result in the rejection of build by the test team.

Regression testing is a continuous process and it happens after every release. Test cases are added to the regression suite after every release and repetitively executed for every release. 

Because test cases of regression suite are executed for every release, they are perfect candidate for the automation. Test automation is discussed separately in the Test Automation section of this website.