Software Testing Strategies

Testing Strategy :

-A strategic approach to testing 

- Test strategies for conventional software 

- Test strategies for object-oriented software

- Validation testing 

- System testing 

- The art of debugging

Introduction:

• A strategy for software testing integrates the design of software test cases into a well-planned series of steps that result in successful development of the software 

• The strategy provides a roadmap that describes the steps to be taken, when, and how much effort, time, and resources will be required 

• The strategy incorporates test planning, test case design, test execution, and test result collection and evaluation 

• The strategy provides guidance for the practitioner and a set of milestones for the manager 

• Because of time pressures, progress must be measurable and problems must surface as early as possible

General Characteristics of Strategic Testing:

• To perform effective testing, a software team should conduct effective formal technical reviews 

• Testing begins at the component level and work outward toward the integration of the entire computer-based system 

• Different testing techniques are appropriate at different points in time 

• Testing is conducted by the developer of the software and (for large projects) by an independent test group 

• Testing and debugging are different activities, but debugging must be accommodated in any testing strategy 

Verification and Validation:

• Software testing is part of a broader group of activities called verification and validation that are involved in software quality assurance 

• Verification (Are the algorithms coded correctly?) – The set of activities that ensure that software correctly implements a specific function or algorithm 

• Validation (Does it meet user requirements?) – The set of activities that ensure that the software that has been built is traceable to customer requirements

Organizing for Software Testing:

• Testing should aim at "breaking" the software 

• Common misconceptions 

– The developer of software should do no testing at all

– The software should be given to a secret team of testers who will test it unmercifully – The testers get involved with the project only when the testing steps are about to begin 

• Reality: Independent test group 

– Removes the inherent problems associated with letting the builder test the software that has been built 

– Removes the conflict of interest that may otherwise be present 

– Works closely with the software developer during analysis and design to ensure that thorough testing occurs

A Strategy for Testing Conventional Software:

Software Testing Techniques

Testing Techniques

§  Test techniques normally refers to the techniques used to select the 'best' tests from the infinite number possible. Test techniques cover two most important issues:

•  Test design: the process of identifying test cases

•  Test measurement: the process of setting objective coverage targets and measuring progress towards them.

What are Testing Techniques?

§  Test cases are build on these Test Conditions.

-       Test cases determine ‘how’ the ‘what’ is to be tested

§  Help in Identifying Test Conditions

-       Determines ‘what’ is to be tested

-       reduces by more than a count of 1, the number of other test cases yet providing the same confidence

-       They are systematic, converts the testing Art into a Science

-       They are objective - using the same technique, the same test cases should always be obtained

Types of Testing Techniques

Types of Testing Techniques

Relationship

Relationship

Main Dynamic Techniques

Mappings between Phases of Testing and Dynamic Testing Techniques

Mappings between Phases of Testing and Dynamic Testing Techniques

Mappings between Phases of Testing and Dynamic Testing Techniques

Mappings between Phases of Testing and Dynamic Testing Techniques

Please comment like and share.

Thank you!

Test Case download with steps from HP ALM

There are various methods to do that, below are two methods:

1. VB Script to download test cases with test steps from QC/ALM.

Function EmportTestCases()
    On Error Resume Next  
    Dim QCConnection
    Dim sUserName, sPassword
    Dim sDomain, sProject
    Dim TstFactory, TestList
    Dim TestCase
'Create QC Connection Object to connect to QC  
    Set QCConnection = CreateObject("TDApiOle80.TDConnection")
    sUserName = "your user id"
    sPassword = "your password"
    QCConnection.InitConnectionEx "your QC URL/qcbin"
'Authenticate your user ID and Password
    QCConnection.Login sUserName, sPassword
'Quit if QC Authentication fails
    If (QCConnection.LoggedIn <> True) Then
        MsgBox "QC User Authentication Failed"
        End
    End If
    sDomain = "your domain name"
    sProject = "your project name"
'Login to your Domain and Project
    QCConnection.Connect sDomain, sProject
'Quit if login fails to specified Domain and Project
    If (QCConnection.AuthenticationToken = "") Then
        MsgBox "QC Project Failed to Connect to " & sProject
        QCConnection.Disconnect
        End
    End If
'Now successful connection is made to QC
'Get the test factory
    Set TstFactory = QCConnection.TestFactory
' Your QC Project Path for which you want to download
' the test cases.
    fpath = "your project folder"  
    Set myfilter = TstFactory.Filter()
    myfilter.Filter("TS_SUBJECT") = "^" & fpath & "^"
'Get a list of all test cases for your specified path
    Set TestList = myfilter.NewList()
'Format the header before downloading the test cases
    With ActiveSheet
        .Range("B5").Select
        With .Range("B4:H4")
            .Font.Name = "Arial"
            .Font.FontStyle = "Bold"
            .Font.Size = 10
            .Font.Bold = True
            .HorizontalAlignment = xlCenter
            .VerticalAlignment = xlCenter
            .Interior.ColorIndex = 15
        End With
        .Cells(4, 2) = "Subject (Folder Name)"
        .Cells(4, 3) = "Test Name (Manual Test Plan Name)"
        .Cells(4, 4) = "Description"
        .Cells(4, 5) = "Status"
        .Cells(4, 6) = "Step Name"
        .Cells(4, 7) = "Step Description(Action)"
        .Cells(4, 8) = "Expected Result"
        Dim Row
        Row = 5 '- set the data row from 5
'loop through all the test cases.
        For Each TestCase In TestList
            .Cells(Row, 2).Value = TestCase.Field("TS_SUBJECT").Path
            .Cells(Row, 3).Value = TestCase.Field("TS_NAME")
'QC stores description in html format. So before storing it
'in to excel, StripHTML() will remove all HTML tags and put
'texts only. Also new line tag <br> is replaced with new line
'character chr(10) in excel so that all the new line texts appears properly
            .Cells(Row, 4).Value = StripHTML(Replace(TestCase.Field("TS_DESCRIPTION"), _
 "<br>", Chr(10)))
            .Cells(Row, 5).Value = TestCase.Field("TS_EXEC_STATUS")
'Get the DesignStepFactory for the this testcase
            Dim DesignStepFactory, DesignStep, DesignStepList
            Set DesignStepFactory = TestCase.DesignStepFactory
            Set DesignStepList = DesignStepFactory.NewList("")
'Check if design steps exists for the test
            If DesignStepList.Count <> 0 Then
'loop for all the steps for this test case
                For Each DesignStep In DesignStepList
                    .Cells(Row, 6).Value = DesignStep.StepName
                    .Cells(Row, 7).Value = StripHTML(Replace(DesignStep.StepDescription, _
 "<br>", Chr(10)))
                    .Cells(Row, 8).Value = StripHTML(Replace(DesignStep.StepExpectedResult, _
 "<br>", Chr(10)))
                    Row = Row + 1
                Next 'next Step
            End If
            ' release the design step objects
            Set DesignStepFactory = Nothing
            Set DesignStep = Nothing
            Set DesignStepList = Nothing
        Next ' Next test case
    End With
    'Release the object
    Set DesignStepFactory = Nothing
    Set DesignStep = Nothing
    Set DesignStepList = Nothing
    Set TstFactory = Nothing
    Set TestList = Nothing
    Set TestCase = Nothing
    QCConnection.Disconnect
    MsgBox ("All Test cases are downloaded with Test Steps")
End Function
 
Function StripHTML(sInput As String) As String
    Dim RegEx As Object
    Set RegEx = CreateObject("vbscript.regexp")
    Dim sInput As String
    Dim sOut As String
    sInput = cell.Text
    With RegEx
        .Global = True
        .IgnoreCase = True
        .MultiLine = True
        .Pattern = "<[^>]+>" 'Regular Expression for HTML Tags.
    End With
    sOut = RegEx.Replace(sInput, "")
    StripHTML = sOut
    Set RegEx = Nothing
End Function


2. SQL query to download test cases with test steps from QC/ALM.

SELECT T.TS_NAME, T.TS_TEST_ID, T.TS_DESCRIPTION, T.TS_STEPS, S.DS_STEP_NAME, S.DS_DESCRIPTION, S.DS_EXPECTED
FROM TEST T, DESSTEPS S
WHERE T.TS_TEST_ID = S.DS_TEST_ID;

test status
SQL 1:
SELECT
CF_ITEM_NAME as "Test Lab Folder Name",
CF_ITEM_PATH as "Folder path",
CF_ITEM_ID as "Folder ID"
FROM CYCL_FOLD
order by CF_ITEM_PATH ;

SQL 2:
SELECT
CF_ITEM_NAME as "Test Set Folder Name",
CY_CYCLE as "Test Set Name",
TS_NAME as "Test Case Name",
RN_STATUS as "Test Case Status",
ST_STEP_NAME as "Test Step Name",
ST_STATUS as "Test Step Status",
ST_DESCRIPTION as "Test Step Description",
ST_EXPECTED as "Test Step Expected Result",
ST_ACTUAL as "Test Step Actual Result",
RN_HOST as "Test Host Name",
RN_TESTER_NAME as "Tester Name",
ST_EXECUTION_DATE as "Test Step Execution Date",
ST_EXECUTION_TIME as "Test Step Execution Time"
FROM STEP a, TEST b, CYCLE c,RUN d,CYCL_FOLD e
where
a.ST_TEST_ID=b.TS_TEST_ID and
c.CY_CYCLE_ID=d.RN_CYCLE_ID and
d.RN_TEST_ID=b.TS_TEST_ID and
e.CF_ITEM_ID=c.CY_FOLDER_ID and
RN_HOST IS NOT NULL and
RN_TESTER_NAME IS NOT NULL and
CF_ITEM_PATH like 'AAAAAAAAA%'

ORDER BY TS_NAME,RN_RUN_ID,ST_RUN_ID,ST_EXECUTION_DATE,ST_EXECUTION_TIME ASC;

Software Development Life Cycle

Software Development Life Cycle

Ø      Requirement (gathering & analysis)

Ø      Design (HLD, LLD)

Ø      Develop (Code)

Ø      Test (Validate)

Ø      Deploy (Implement)

Ø      Support (Maintain) 

Requirements

Types of requirements

• Functional Requirements

• Data flow diagrams

• Entity-relationship diagrams

• State transition diagrams

• Data Dictionary

• System parameters

Ø Security, integrity, performance, reliability

• External Interfaces

• Acceptance Criteria

Ø Validation and Performance criteria 

Design

High Level:

• Functional Design

• Activity Flows / Work Flows

• System Flows and Application Flows

• Logical Data Model

• Program Inventory

• Architectural Diagrams

Low Level:

• Program Specs - Database Design

• Architectural diagrams - Data Access Design

• Interface design - Class Specifications

Build (Code)

Ø Environment Setup

Ø Test Data Creation

Ø Coding

Testing

• Test plans

• Test specifications

Ø Test cases + Test data + Expected Results

• Testing

• Test Results

Implement

Ø Data Conversion

Ø Data Migration

Ø Database Creation

Ø Production Tests

Ø Contingency Planning

Ø User Training

Support

Ø Production Support

Ø Enhancements 

Myths of SDLC

»Once the program is written and working, job is done.

»Until I get the program running, I have no way of assessing its quality. 

SDLC Models

Waterfall Model

The waterfall model is a sequential software development model (a process for the creation of software) in which development is seen as flowing steadily downwards (like a waterfall) through the phases of requirements analysis, design, implementation, testing (validation), integration, and maintenance.

V Model

V model is the software development model which encapsulates the steps in Verification and Validation phases for each step in the SDLC. For each phase, the subsequent phase becomes the verification (QA) phase and the corresponding testing phase in the other arm of the V becomes the validating (Testing) phase

Iterative Development Models

Prescribes the construction of initially small but ever larger portions of a software project to help all those involved to uncover important issues early before problems or faulty assumptions can lead to disaster.

Example: RUP, RAD, Prototyping, Agile

Testing within each lifecycle

In any life cycle model, there are several characteristics of good testing:

• For every development activity there is a corresponding testing activity.
• Each test level has test objectives specific to that level.
• The analysis and design of tests for a given test level should begin during the corresponding development activity.
• Testers should be involved in reviewing documents as soon as drafts are available in the development life cycle.

Software Quality

What is Quality?

Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated and implied needs.

Software Quality means: Total conformance to user requirements and specifications. Absence of Defects Fitness of Use.

Other definitions

Errors: Mistakes we make.

Defects: Defects are the results of errors.

Failures: Software failures are the results of Defects - observed while running the software

Quality Assurance vs Quality Control

Quality Assurance

1.Ensures that the product is developed correctly

2.Prevention activity

3.Eg. Process definition, planning

4.Done at the beginning of project

Quality Control

1.Ensures that the product developed is correct

2.Detection and correction activity

3.Eg. Reviews and Testing

4.Carried out after initiation

Importance of QA/QC

“All QC/QA activities are important, because all lead to quality software, if practiced correctly. However, if you allow me to choose one only, my choice is Formal Technical Reviews. If conducted properly, they are the single-most effective ways to uncover and fix defects while they are still inexpensive to find and fix.”

Roger Pressman

What is V&V?

Verification. 

• Determine whether the artifacts of each phase of SDLC fulfill the requirements of the previous phase. 
• Are we building the product correctly? 
• Also called as static technique or Review

Validation.

• Determine whether the artifacts of each phase of the SDLC are in tune with the customer requirements. 
• Are we building the correct product? 
• Also called as dynamic technique or Testing

Static and Dynamic V&V

Static and Dynamic techniques.

Why V&V?

• V&V is required because:
• Software construction is a complex task.
• Software construction is a Human activity. Humans make mistakes.
• Discipline and Methodology for construction reduce mistakes but cannot eliminate mistakes.
• V&V is about evaluating the products and processes used for producing the products.
• V&V is NOT about evaluating the producers.

Types of V&V

Reviews: 

• Project Plans, Schedules, and Costs 
• Requirements, Functional Specifications, and Design 
• Test Plans, Test Specifications, Test Scripts, 
• Source Code

Testing of Software: 

• Unit
• Integration 
• System Customer Acceptance