Practical Measurements For Software Testing

Every software development company focuses on developing quality software. The only way to track the software quality is evaluating it at every stage of its development. It requires some kind of metrics, which is obtained through effective testing methods. Each stage of software testing is effectively monitored for the software QA.
1. Software measurements are used for:
2. Deriving basis for estimates
3. Tracking project progress 4. Determining (relative) complexity
5. Understanding the stage of desired quality
6. Analyzing defects
7. Validating best practices experimentally
Here, some software testing metrics are proposed for black box testing that has real world applications. It discusses:
Importance of software testing measurement
Different techniques/processes for measuring software testing
Metrics for analyzing testing
Methods for measuring/computing the metrics
Advantages of implementing these metrics
These metrics helps in understanding the inadequacies in different software QA stages and finding better correcting practices.
What is measurement and why it is required?
The process of assigning numbers or symbols to attributes of real world entities for describing them according to defined rules is called measurement.
For developing quality software, several characteristics like requirements, time and effort, infrastructural cost, requirement testability, system faults, and improvements for more productive resources should be measured.
Measuring software testing is required:
1. If the available test cases cover all the system s aspects
2. For tracking problems
3. For quantifying testing
Choose the suitable metrics:
Several metrics can measure software-testing process.
Here, the following types of metrics are identified:
Base metrics:
These raw data are collected in a testing effort and applied in formulae used to derive Calculated Metrics.
The Test Metrics comprise of the Number of
Test Cases Passed, Failed, Under Investigation, Blocked, Re-executed and Test Execution Time.
Calculated metrics:
They convert the Base Metrics data into useful information. Every test efforts must implement the following Calculated Metrics:
% Complete
% Defects Corrected
% Test Coverage
% Rework
% Test Cases Passed & Blocked
% Test Effectiveness & Efficiency
% 1st Run Failures
% Failures
Defect Discovery Rate
Defect Removal Cost
Measurements for Software Testing
The corresponding software testing process in software development measures each step for ensuring quality product delivery.
1. Software Size:
The amount of functionality of an application determines this and is calculated by
Function Point Analysis
Task Complexity Estimation Methodology
2. Requirements review:
Before software development, the Software requirement specifications (SRS) from the client are obtained. It must be:
Complete
Consistent
Correct
Structured
Ranked
Testable
Traceable
Unambiguous
Validate
Verified
The Review Efficiency is a metric that offers insight on the review quality and testing.
Review efficiency=100*Total number of defects found by reviews/Total number of project defects
3. Effectiveness of testing requirements:
It is measured by maintaining Requirement Trace-ability matrix and specification of requirements, which should have:
SRS Objective, purpose
Interfaces
Functional Capabilities
Performance Levels
Data Structures/Elements Safety
Reliability
Security/Privacy
Quality
Constraints & limitations
Next comes the updating of the crucial requirement trace-ability matrix or RTM, which determines the number and types of tests.
While measuring the mapping of test cases, the number and priority of requirement it tests, its execution effort and requirement coverage must be determined.
The Requirement compliance factor (RCF) measures the coverage provided by the test cases to one or set of requirement(s).
Mathematically, RCFj=?(Pi*Xi)/(maxXi)*(?Pi)i=1
Where,
j is a set of requirements and (j=1-m);
Xi=2, if the test case (say Tj) tests requirements Ri completely, =1, if it tests partially,
=0, if otherwise. Effectiveness=RCFj/Ej where Ej=Time required for executing a test case
4. Evaluating estimation accuracy
Relative error=(A-E)/A where E is estimate of a value and A is actual value.
For a collection of estimates, the mean RE for n projects is
n RE=1/n?REi i=1
For a set of n projects, the mean magnitude of RE (MRE) is
n MRE=1/n?MREi i=1
Of a set of n projects, an acceptable level for MRE is less than 0.25.
If K is the number of projects whose mean magnitude of relative error is less than or equal to q, then the prediction quality pred(q)=K/n
5. Measurement of Efficiency in testing process
In software testing, we must keep tabs on what we had planned and what we have actually achieved for measuring efficiency. Here, the following attributes play major roles: -
Cost: The Cost Variance (CV) factor measures the risk associated with cost.
CV=100*(AC PC)/PC, AC=Actual Cost, PC=Planned/Budgeted Cost.
Effort: Effort Variance (EV) measures effort.
EV=100*(AE PE)/PE (AE=Actual Effort, PE=Planned Effort)
Schedule: Schedule Variance (SV) is important for project scheduling.
SV=100*(AD-PD)/PD where AD=Actual duration and PD=Planned duration.
Cost of quality: It indicates the total effort expended on prevention, appraisal and rework/failure activities versus all project activities.
Prevention Effort=Effort expended on planning, training and defect prevention. Appraisal Effort=Effort expended on quality control activities.
Failure effort=Effort expended on rework, idle time etc.
COQ=100*(PE + AE + FE)/Total project effort.
Product -
Size variance: It is the degree of variation between estimated and actual sizes. Size Variance=100*(Actual Software Size Initial Estimated Software Size)/Initial Estimated Software Size
Defect density: It is the total number of defects in software with respect to its size.
Defect density=Total number of defects detected/software size Mean Time Between Failures: MTBF is the mean time between two critical system failures or breakdowns.
MTBF=Total time of software system operation/Number of critical software system failures.
Defects: Defects are measured through:
Defect distribution: It indicates the distribution of total project defects. Defect Distribution=100*Total number of defects attributed to the specific phase/Total number of defects.
Defect removal effectiveness: Adding the number of defects removed during the phase to the number of defects found later approximates this.
Benefits of implementing metrics in software testing:
Improves project planning.
Understanding the desired quality achieved.
Helps in improving the processes followed.
Analyzing the associated risks.
Improving defect removal efficiency.
ReadyTestGo is a professional ( http://www.readytestgo.com/readytestgo.htm ) Software Testing Company ( http://www.readytestgo.com/services.htm ) Outsourcing QA . For more details, please contact marketing@readytestgo.com melania kaylee



Bookmark it: