In today’s environment of plummeting cycle times and budget pressures, test automation has become an increasingly critical and strategic necessity. Software test automation has the capability to decrease the overall cost of testing, and improve software quality. Test automation raises people’s hopes yet often frustrates and disappoints them. Most testing organizations are not been able to achieve the full potential of test automation. Many groups that implement test automation programs run into a number of common obstacles. These problems can lead to test automation plans being completely scrapped with the tools purchased for test automation becoming expensive “shelf-ware”. Often teams continue their automation effort, building up huge costs of maintaining large suites of automated test scripts that are of questionable value.

Many teams acquire a test automation tool and begin automating right away, with little consideration of how they can structure their automation to make it scalable and maintainable. Little consideration is given to managing the test scripts and test results, creating reusable functions, separating data from tests, and other key issues which allow a test automation effort to succeed. After some time, the team realizes that they have hundreds or thousands of test scripts, and thousands of separate test result files. The combined work of maintaining the existing scripts while continuing to automate new ones requires a larger and larger team with higher costs and no additional benefit. As teams drive towards their goal of automating as many existing test cases as possible, they often don’t consider what will happen to the automated tests when the application under test (AUT) undergoes a significant change.

The need of automation strategy is quite clear in these situations. For example, automation would be a waste of effort if the tests would never be repeated. Automation requires a proper methodology in architecting and implementing it. For example, writing automation for a user interface that is not stable is clearly a throw away effort. People sometimes view automation is a programming project — which is another recipe for disaster. Automation requires much different and rigorous treatment than other programming projects.

This paper discusses Aztecsoft iTest’s Test Automation Strategy Service which offers a complete solution to this challenge. The paper discusses a systematic methodology for Test Automation. In the strategy phase, participants will understand the test organization, existing test practices, testing problems, and past successes. Test automation goals for critical areas will also be defined (i.e. where automation is the only solution), as well as cost and time benefits wherever possible. the functional test coverage will be analyzed, and if applicable, additional automation test cases to improve the coverage will be proposed. A plan to implement automation which would deliver the greatest ROI will be put together. The plan includes a well-defined test automation vision, answers questions such as what, when and how of automation, and proposes a prioritized test automation plan. The strategy will also include recommendations on test automation tools and harnesses to assist with the “how” question.

Audience:

This paper is intended to benefit Test Leads and Managers on how to build an effective automation test strategy and ensure a good ROI to the organization.