Types of Test Automation Frameworks

In a previous posting, we examined the evolution of automation frameworks.
How are frameworks being implemented today by various QA organizations? Here’s a basic summary of the types of test automation currently in use:
Ad-Hoc

• Scripting developed in reactionary mode to test a single issue or fix
• Test case steps are part of each Action script: high maintenance, low reusability
• Contains some data inputs stored in test script’s datasheet, but not true data-driven

Data-Driven

• Scripts are an assembly of function calls
• Data for test cases read in from external source (e.g., Excel spreadsheet, ODBC database)
• Results can be captured externally per script execution (i.e., spreadsheet, database)

Keyword-Driven

• Test cases are expressed as sequence of keyword-prompted actions
• A Driver script executes Action scripts which call functions as prompted by keywords
• No scripting knowledge necessary for developing and maintaining test cases

Model-Driven

• Descriptive programming is used to respond to dynamic applications (e.g., websites)
• Actually, this is a method which can used within other solution types
• Objects defined by parameterized code (i.e., regular expressions, descriptive programming)
• Custom functions used to enhance workflow capabilities

3rd-Party: HP (Mercury) Quality Center with QuickTest Pro and Business Process Testing

• Similar to keyword-driven but controlled via HP Quality Center database
• Can be used for collaborative work efforts: Business Analysts, Testers, Automaters
• Begins with high-level test requirements:
• Business Requirements defined
• Application Areas (shared resources) defined
• Business Components defined and grouped under Application Areas
• Test steps defined
• Tests can be defined as Scripted Components (i.e., QTP scripts with Expert Mode)
• Business Process Tests and Scripted Components are collected under Test Plan
• Test Runs are organized from Test Plan components and executed from Test Lab
• Test Runs can be scheduled and/or executed on remote/virtual test machines (with QTP)
• Defects can be generated automatically or entered manually per incident
• Dashboard available for interactive status monitoring

Intelligent Query-Driven

• Agile
• Object Oriented
• Constructed as a layered framework
• Test data is compiled at runtime using data-mining techniques

Of course, a combination of these techniques could also be used based on the scope and depth of test requirements.