Bridging the Gap between Open Source & Commercial Tools

By - http://www.qacampus.com

These opinions are strictly my own based on my own observations and experiences with both kinds of software and trying to understand the dynamics at play in each environment. I’m sure others share similar opinions. Also note that these are sweeping generalities and caricatures, but I think provide a good starting model for critiquing both.
The main Gap between Open Source and commercial tools is licensing. Using the Open Source Edition (under the GPL license) obligates you to share your source code without restrictions with the users of your program. Using the GPL also means you may not demand compensation for or limit subsequent re-use and re-distribution of the source code. You need the commercial license if you want to avoid these obligations.

Open source Functional Tools

The reason many users originally try an open source solution (myself included) is price. An open source functional tool will be significantly cheaper than a commercial functional tool. As with many open source programs, because the code is “open,” the opportunities for customization are also greater than they are for a commercial functional tool. Depending on your functional tool needs, there may very well be an existing open source functional tool that will fulfill your requirements.

The arguments against implementing an open source functional tool are numerous, but are generally tied into one key concern: uncertainty. Product support, documentation, and user training are often subject to the whims of volunteer (read: unaccountable) developers. As a result, there is often no brand name or customer service department to offer assurances or assistance in maintaining functional tool stability and security. Enterprise-level workflow management may therefore be difficult to achieve, and product implementation may take considerably longer than with comparable commercial functional tool products.

Commercial Functional Tools

Buying a commercial functional tool offers a number of distinct advantages, not the least of which is commercial support and well-defined service level agreements. A commercial functional tool may already be ready-built for your needs and will likely be faster to implement than an open source functional tool. Documentation and training for commercial functional tool products are usually significantly stronger than for an open source solution. Your average person also associates a certain degree of safety with commercial software as opposed to open source. If you or your client has the resources to purchase and appropriately license a tool, it can often be the safest bet.

Arguments against buying a commercial functional tool come down to one issue: cost. Commercial functional tool license costs can be prohibitively expensive, and customization/integration expenses can send these prices even higher. Commercial functional tools rarely represent a “budget” solution.

On top of that commercial software needs to demonstrate that they have solved a hard problem that no one else has solved or make competitors solutions to problems seem hackish. So the marketing folks will use words like “You want to solve the problem the right way, don’t you?” which often means “The hard way that only we know how to do.” A corollary to that is if a problem is really hard and you have solved it or can convince people you have solved it, make sure everyone knows that and make sure everyone thinks they have the same problem. That actually holds true for both commercial and open source but is generally easier to pull off for commercial software.

Comparison Parameters

Tools => Record and Play Back => Language support => Application support
Watir => Support with WET => Ruby => Web Based Application
Sahi => Inherently Supported => JavaScript => Web Based Application
AutoIT => Inherently Supported => BASIC-like syntax => Windows Application
QTP => Supported => VB Script,Java script => Windows/Web Application
RFT => Supported => Java,HTML,VB.NET => Windows/Web Application

After Thoughts

In the end, I would just like to say that it’s a trade off between how uncertain is the uncertainty of a free Open Source solution with how costly is the cost of a commercial reliable solution. Many companies are trying to bridge the gap between the two by providing software with the reliability and user friendliness of a commercial solution for an open source tool price. Well my team is also working on the same, so keep looking out for my blog people if you are someone needing that Bridge

Application Security | PCI DSS Overview

By – http://www.qacampus.com

As the number security breaches has increased, regulatory and industry requirements have become more stringent. One of the most popular compliance standard is PCI DSS. It was developed by the major credit card companies as a guideline to help organizations that process card payments prevent credit card fraud, cracking and various other security vulnerabilities and threats. A company processing, storing, or transmitting payment card data must be PCI DSS compliant or risk losing their ability to process credit card payments and being audited and/or fined. Here is brief overview of what PCI DSS is all about.

What is PCI- DSS?

· PCI stands for Payment Card Industry.

· PCI-DSS actually stands for PCI Data Security Standards (DSS), currently at version 1.2. PCI DSS is a set of comprehensive requirements for enhancing payment account data security. It was developed by a council (PCI SSC) which includes American Express, Visa International, MasterCard Worldwide, Japan Credit Bureau (JCB). The council is responsible for developing and managing the PCI DSS standards, establishing and maintaining Qualified Security Assessors (QSA) and Approved Scanning Vendors (ASV).

Who must comply with PCI?

Any company that stores, processes or transmits cardholder data must comply with PCI. Compliance to PCI is assurance to the organization that IT infrastructure and business processes are secure. It can serve as great marketing tool for company and instill greater confidence in customer’s and stakeholders’ minds.

Scope of PCI –DSS

All systems that store, process or transmit Cardholder’s data.

a) Applications processing Cardholder’s data ( e.g. e-commerce application, sales processing application)

b) Network Infrastructure

c) Storage Area Networks

d) Data Extracts including Cardholder’s data.

e) Backups

f) Log Files

g) Paper records

h) People

i) Org wise processes and structure

j) Third parties that stores or transmit Cardholder’s data on Organization’s behalf such as suppliers and dealers.

Who can help you get PCI DSS?

a) Consulting Agencies: Consulting agencies can help you find gaps, implement processes to fill the gaps and do a pre audit to make you prepare for final audit by QSA.

b) QSA: A security company qualified by PCI SSC to assess compliance to the PCI DSS standard. QSA’s are certified by PCI SSC to perform on site security assessments for verification of compliance with PCI DSS.

A list of QSA’s can be found at

http://www.pcisecuritystandards.org/pdfs/pci_qsa_list.pdf

Updates about QTP 10 (I)

By – http://www.qacampus.com

QTP 10 revolves around 3 pivotal features, alongside several minor features (which turned out to be quite revolutionary):

I. QC integration – which (mostly) boils down to Resource Management and Source Control:

Resource Management: Although you could keep saving your resources as attachments (for backward compatibility), you can upgrade to a new, fuller mode of work. This includes a whole new Resource module in QC, and allows for some very neat tricks on Function Libraries, Tests, Object Repositories etc.

It should be noted, though, that other types of files (external excel / XML files, for example), remain as unmanaged attachments.

1. Resources have full meta-data, and have a special view pane – you can view Object-Repositories, data-tables, and function libraries code right from QC.

2. Resources are aware of their dependencies – Who relies on them, and who do they rely on. This enables a very strong warning system – when changing / deleting a resource, you’ll be alerted to the repercussions – namely, which tests, if any, might break. Also, the ability to immediately know who uses a share object repository is very useful, nearly revolutionary.

3. A very neat trick is a live, automatically updated path system – When moving a function library between folders, QC will automatically update all the tests which depend on it, so they will use it at its new location. This makes the once critical problem of hard-path-link a non-issue. Very impressive.

4. A word about the user interface – when opening a QC resource / test from QTP, the file dialog shows the items with large, crisp icons, very similar to Word’s save dialog. Everything is very clear and intuitive, as is the ability to revert back to saving / opening a test from the File-System.

5. And what about your existing projects? Well, when upgrading to QC 10, a wizard will automatically transform all you unmanaged attachments to managed resources (if you’d like it to).

Source Control: This includes a very rich line of features which are very well executed, and effectively allow you to manage a QTP project as any other code project:

1. First, the basics – QTP and QC 10 introduce a new Check-in/Check-out ability. It works similar to what you’d expect – a checked out item will be locked to all other users, and you can immediately know an item’s status by looking at its icon (green/red locks).

2. An interesting twist regards manner in which a test / resource is locked – it’s at the user level (not the local machine level). This means that if you log into QC from a different machine, you’ll have access to all your checked-out items, even if they were originally checked-out on a different local machine. The ability is implemented very well, both from QTP’s end, as well as from QC’s end.

A major enabler for source control is the new versioning feature of QC. It manifests both with a kind of instant versioning for a single resource, and with a project-wide “base-line version”, which allows you to revert your entire test framework to a previous version. Both types of versioning are supported by a surprisingly robust comparison mechanism. You can select two versions of a resource / test, a see a very detailed comparison of their respective changes. For function libraries this amounts to a “simple” text comparison, but this feature truly shines in full test comparisons.

It will present changes in the different actions and their resources (data-table, object repositories, inner code), as well as in the global test-settings, associated function libraries, recovery scenarios, and pretty much anything you could think of. The ability to drill-down into the data is very impressive; and the effective, concise manner in which the data is presented in the top level view is downright unbelievable. A nice touch is a small screen capture of the change, in case you don’t remember what “Run all rows –>Changed into-> Run a single iteration only” means (for example).

Now to the versioning mechanism itself: Whenever you check and item in, a new “version” will be created, and you’ll be able to revert back to it with ease. The snapshots are visible both from QC and QTP, and you can very easily choose which one to open. This allows you a kind of an instant undo to a single file which worked in the past, but is broken in the present.

The second mechanism presents the ability to select several folders, and create a full blown “base-line version” of them and everything they relate to. Defects, inner-connections, tests, history data, resources – all these and more will be “frozen” and preserved as a base-line. You can then choose to revert back to an old baseline, and truly regain all the abilities that were present at that time. As all the resources, attachments tests and reports will be restored, you don’t have to worry about forgetting anything, or leaving some minor resource at the wrong version. This is versioning with a vengeance – it allows you to track the AUT’s versions with your own automation versions, enabling, among other things, running automation efforts on several AUT versions at once.

For conclusion – The new abilities inherit in the connection of QTP and QC Atlantis are (or at least seem to be) revolutionary. At last, QTP projects can be natively managed as code projects; and some of the supporting infrastructure is surprisingly robust and useful.

Agent Controller Issue on Starting RAServer Process

By – http://www.qacampus.com

By Kuldeep Singh

Introduction: This document has been prepared in order to resolve the issue that might occur during Invocation of RPT- Agent Controller process on Linux Machine.

Requirement: Our requirement was to generate the load from Linux machine (client) on the application server

For this, we have installed Load Generating tool (Rational Performance Tool version – 7.0.2) on window machine (OS: Window XP 2000 Profession SP-2) and RPT-Agent controller Process (version-7.0.2.1) on Linux machine (OS-Red Hat Enterprise Linux AS release 4-Nahant).

Below are enlisted some of the issues which were encountered during the load distribution through load generating machine (RPT) to Linux machine.

On executing the performance schedule we were getting the following error “Connection failed on host 172.23.244.207”.

————————————————————

Security Message

Connection failed on host 172.23.244.207

Reason:
IWAT0284E The agent controller is not available on host 172.23.244.207
Make sure that:
*the agent controller is installed.
*the agent controller is configured to communicate with your machine
*you have the correct host name and port number for the agent controller.

————————————————————–

Possible reason: The above error might have occurred due to Agent Controller is not installed or is not running on Linux machine.
Since, on Linux machine Agent Controller process (RAServer) process is not started automatically. So, we have to start this process manually.

Starting and Stopping Agent Controller on Linux machine:
• To start the Agent Controller process (RAServer) on Linux machine, move to the Installation location’s bin directory (for e.g. /opt/IBM/AgentController/bin). Then execute the following command
./RAStart.sh

• To stop the Agent Controller process (RAServer) on Linux machine, move to the Installation location’s bin directory ( for e.g. /opt/IBM/AgentController/bin) and then execute the following command
./RAStop.sh

On trying to start the Agent Controller process on Linux machine we may get the following Errors. (Below section describe the Error description, reason and resolution for the same)

Error:
1) Starting Agent Controller
“RAServer: error while loading shared libraries: libstdc++-libc6.2-2.so.3: cannot open shared object file: Error 40 No such file or directory.
RAServer failed to start.” Error

Possible Reason: Since the Agent Controller is compiled using libstdc++-libc6.2-2.so.3 shared library. Ensure that this shared library exists under the /usr/lib directory. If it does not exist, then you have to install the RPM package compat-libstdc++ that comes with the operating system installation media.
Note: – To make sure that libstdc++-libc6.2-2.so.3 shared library is available in the /usr/lib directory:
Move to the /usr/lib directory and execute the following command at the Shell prompt.
# ls –l libstdc*

Resolution:
The solution is to install the standard C++ compatibility libraries in order to satisfy this library dependency. The version of Linux on the client machine will determine what RPM or software package needs to be installed.
In our case, since we are using Red Hat Enterprise Linux As Release 4 (Nahant) Operating System on Linux machine, we need to install compat-libstdc++-296-2.96.132.7.2.i386.rpm package that is located on the Red Hat 4.0 Installation Disc 3.
Note: For more which rpm package required installing, browse the following link
http://seer.entsupport.symantec.com/docs/267077.htm

We can also download required rpm package from the following link
http://rpmfind.net/linux/rpm2html/search.php?query=libstdc%2B%2B-libc6.2-2.so.3&submit=Search
http://rpmfind.net/linux/RPM.

How to Install Required RPM Package:
1) Insert the required disc in CD-ROM and change the directory Red Hat/RPMS from Shell command.
cd media/CDROM/Red Hat/RPMS/
2) Enter the following command and execute
rpm –ivh compat-libstdc++-296-2.96-132.7.2.i386.rpm
If installation is successful, you see the following message:
Preparing… ########################################### [100%]
1:compat-libstdc++-296 ################################## [100%]
RPM prints out the name of the package and then prints a succession of hash marks as the package is installed as a progress meter.
Note: For more information on RPM package browse the following link
http://www.faqs.org/docs/securing/chap3sec20.html

Now we can start the Agent Controller Process (RAServer) on Linux machine. Following message should be displayed on successfully start the Agent Controller Process.

Starting Agent Controller
RAServer Started Successfully
RPM prints out the name of the package and then prints a succession

2) “RAServer failed to Start” Error
Possible Reason: This failure is usually caused when TCP/IP port 1002 is not free. Agent Controller listens on this port by default. Agent controller was just stopped and restarted before the port could be released.
• If Agent Controller failed to start. You can start it as follows:
If port 10002 is being used by another process, you can change the port number by editing the serviceconfig.xml file. Serviceconfig.xml file is located in Installation Location Config’s directory /opt/IBM/AgentController/Config/

• If Agent Controller was just stopped, wait a few minutes and try to start it again.

VSTS Test Edition - A General Writeup

VSTS Test Edition - A General Writeup

By Pankaj Goel

Visual Studio Team System testing tools provides several test types that can be used for specific software testing purposes. Section below describes those test types.

Unit Test
Deals with generating and authoring unit tests, including ASP.NET unit tests and data-driven unit tests.

Web Tests
Works with Integrated Web Test Recorder to record the actions you perform while you browse a Web site.

Load Tests
Refers to Simulating load of multiple users accessing the web application simultaneously

Manual Tests
Manual test steps can be written in text format and can be referred by functional Testing Team

Generic Tests
Wrap an existing test, program, or third-party tool as a generic test.

Ordered Tests
Refers to Ordered test that would execute in a specific sequence.

About Web Test

Web Test is a tool with VSTS Test edition that is used for automating functional tests. The scripts for Web test can be used for performance testing as well.
A Web test simulates how an end user might interact with a Web application. Web test is created by recording HTTP requests using the Web Test Recorder in a browser session. However Web tests can also be building manually using the Web Test Editor.

In contrast to most of the test automation tools on the market, Web Test records HTTP transactions, not Windows API calls. For instance when we click on a [Submit] button, what’s recorded is not the click but the transaction going to the server as a result.

The language used with this tool is C# or VB dot net

We can
• Parameterize recorded transactions with test data,
• Add or remove transactions,
• Add validation points (called Validation Rules in VSTS lingo)

We can not
• Perform GUI validation with this tool
• Perform Database Validation using this tool
• Use the flow control and looping constructs that are provided by more advanced programming languages.

Pros of Web test

• Script is more reliable in terms of being oblivious to minor changes in the application user interface
• VSTS scripts can also detect defects that would be missed with GUI tools.

Cons of Web test

• Web test is not a classical test automation tool. It does not work on GUI elements
• Learning curve is significantly higher. You require Coders to do the effective test automation


Info

http://www.qacampus.com

http://www.crestech.in

Software Testing

Australia

Script Services

Script Services

By Renu Bala

Script Support Functions enables you to insert code into Functional Tester Script to perform variety of tasks. These tasks include:

• Calling a script from a Functional Test Script
• Inserting Log Messages into Functional Test Script
• Using timers with Functional Test Script
• Setting delays and sleep states for Functional Test Script Playback
• Inserting comments into Functional Test Script

The Script Support Functions dialog box has following tabs:

1. Call Script for Calling a script from a Functional Test Script
2. Log Entry for Inserting Log Messages into Functional Test Script
3. Timer for Using timers with Functional Test Script
4. Sleep for Setting delays and sleep states for Functional Test Script Playback
5. Comment for Inserting comments into Functional Test Script

To use script support commands Click on Insert Script Support Commands button on Recording Monitor Toolbar.

Call Script
While recording or editing a functional test script, you can insert a call to a previously recorded script. This lets you avoid repeatedly recording similar actions on the AUT by taking advantage of scripts that already exist.
CallScript inserts following code in the script at the cursor location:

callScript (“scriptname”)
Where scriptname is the name you selected in the script name field.

Log Entry
You can insert a log message into a functional test script and indicate whether it is a message, a warning or an error. During playback, RFT inserts this information into the log.
LogEntry inserts the following code into the script based on the option you selected (message, warning, error):

logInfo(“Message”)
logWarning(“Message”)
logError(“Message”)
Where Message is the text you entered

Timer
You can insert timers with different names into the same script to measure time it takes to perform a variety of separate tasks. You have to explicitly stop the timer to calculate the time taken by the set of events.
The code generated is:
timerStart(“name”)
timerStop(“name”)
Where name is the variable name you want to use for the timer.

Sleep
You can insert sleep command into Functional test script to delay the script for specified time.
The code generated is:
Sleep(seconds)
Where seconds is the time entered in seconds fields.

Comment
During recording or editing, you can insert lines of comment text into RFT script. Comments are helpful for documenting and editing scripts. It inserts the text with appropriate comment delimiter (//) preceding each line.


Info

http://www.qacampus.com

http://www.crestech.in

Software Testing

Australia

Script Maintenance

Script Maintenance

By Renu Bala

Script maintenance is a great challenge faced by automation testers. As the course of development proceeds, applications undergo changes, which cause automation test scripts to break. For example, Developers change underlying attributes (like object name) that the test script rely upon to identify the object. So testers need to rework on scripts.

Script maintenance at some level is a fact of automation but too much maintenance can be a drain on team’s time, resulting in decreasing value of automation. Automation testing tool that build scripts that are resistant to breaking are extremely valuable.

Testers must constantly update scripts to handle minor application changes but testers may make mistakes when they update scripts.

One solution to these changes is to update the object map so that all scripts can reference updated object map and identify objects.

Second solution to these changes is Script Assure Technology introduced by Rational Functional Tester. This technology makes test scripts more resilient to changes in applications. With script assure technology, RFT evaluates the closeness of objects that it finds and attempts to match something that is closest. If objects are close enough, test continues with no intervention from tester and will log a warning so that tester is made aware of a possible discrepancy. So more tests can execute with less intervention, even with significant changes being made to objects in application. In this way RFT automatically finds changed target objects in application without object remapping. Benefit of this technology is reduced maintenance and more reliable scripts. Additionally, this flexibility is completely under tester’s control. Tester can determine what level of closeness is sufficient enough for the project

Info

http://www.qacampus.com

http://www.crestech.in

Software Testing

Australia