|
What
is Verification and Validation?
FDA Definition and Explanation
VALIDATION
Software validation is "establishing by objective evidence that
all software requirements have been implemented correctly and completely
and are traceable to system requirements" [Ref: NIST 500-234].
FDA considers software validation to
be “confirmation by examination and provision of objective
evidence that software
specifications conform to user needs and intended uses, and that
the particular requirements implemented through software can be
consistently fulfilled.” [Ref: FDA General Principles of Software
Validation, Final Guidance, January 2002]. Software validation
is essentially a design verification function as defined in FDA's
Quality System Regulation (21 CFR 820.3 and 820.30) and includes
all of the verification and testing activities conducted throughout
the software life cycle.
Design validation encompasses software validation, but goes further
to check for proper operation of the software in its intended use
environment.
VERIFICATION
Verification is defined in FDA's Quality System Regulation (21 CFR
820.3) as "confirmation by examination and provision of objective
evidence that specified requirements have been fulfilled." In FDA's
General Principles of Software Validation, software verification
provides objective evidence that the design outputs of a particular
phase of the software development life cycle meet all of the specified
requirements for that phase. Software verification looks for consistency,
completeness, and correctness of the software and its supporting
documentation, as it is being developed, and provides support for
a subsequent conclusion that software is validated. Software testing
is one of many verification activities intended to confirm that
software
development output meets its input requirements. Other verification
activities include various static and dynamic analyses, code and
document inspections, walkthroughs, and other techniques."
In
a software development environment, software verification is confirmation
that the output of a particular phase of development meets all of
the input requirements for that phase.
IEEE Definition
VALIDATION
"Confirmation by examination and provisions of objective evidence
that the particular requirements for a specific intended use are
fulfilled."
VERIFICATION
"Confirmation by examination and provisions of objective evidence
that specified requirements have been fulfilled."
Using the above definitions in software development
Validation, in its simplest terms, is the demonstration that the
software implements each of the software requirements correctly
and completely. In other words, the "right product
was built." Verification is the activity
which ensures the work products of a given phase fully implement
the inputs to that phase, or "the product was built right."
Levels of Verification
There are three levels of software verification: |
| 
|
Software
Unit Testing
Testing conducted to verify the implementation of the design for
one software element (unit, module) or a collection of software
elements. |
|
|
Software
Integration Testing
An orderly progression of testing in which various software elements
and/or hardware elements are integrated together
and tested. This testing proceeds until the entire software
system has been integrated. |
|
|
Software
System Testing
The process of testing an integrated software system to verify that
the software system meets its specified requirements.
|
|
There
are three levels of system validation and verification:
|
|
|
System
Integration Testing
An orderly progression of testing in which various system elements
are integrated together and tested. This testing proceeds until
the entire system has been integrated. |
|
|
System
Testing
The process of testing an integrated hardware and software system
to verify that the system meets its specified requirements.
|
|
|
Acceptance
Testing
Formal testing conducted to determine whether
or not a system satisfies its acceptance criteria and to
enable the customer to determine whether or not to accept the system.
|
| |
Types
of Verification
There are four types of verification that can be applied to the various
levels outlined above: |
|
|
Inspection
Typical techniques include desk checking, walkthroughs, software
reviews, technical reviews, and formal inspections (e.g., Fagan
approach). |
|
|
Analysis
Mathematical verification of the test item, which can include estimation
of execution times and estimation of system resources.
|
|
|
Testing
Also known as "white box" or "logic driven" testing. Given input
values are traced through the test item
to assure that they generate the expected output values, with the
expected intermediate values along the way. Typical techniques include
statement coverage, condition coverage, and decision coverage. |
|
|
Demonstration
Also known as "black box" or "input/output driven" testing. Given
input values are entered, and the resulting
output values are compared against the expected output values. Typical
techniques include error guessing, boundary-value analysis, and
equivalence partitioning. |
|
Explanation
The four types of verification can be used at any of the levels
although some work better than others for a given level of verification.
As an example, the most effective way to find anomalies at the component
level is inspection. On the other hand, inspection is not applicable
at the system level (you don't look at
the details of code when performing system level testing). A logical
approach to testing is to utilize techniques and methods that are
most effective at a given level.
Component level verification can easily get very expensive. Companies
need to avoid making statements like "all paths and branches will
be executed during component testing." These statements make for
a very expensive test program, as all code developed is required
to have one of the most labor-intensive type
of testing performed on it. To minimize the costs of component verification,
the V&V group develops rules for determining the type of verification
method(s) needed for each of the software functions. As an example,
very low complexity software function, which is not on the safety
critical list, may only need informal inspection (walkthrough) performed.
Other complicated functions typically require white box testing
since the functions become difficult to determine how they work.
We recommend performing inspections before doing the white box testing
for a given module as it is less expensive
to find the errors earlier in the development.
The FDA is embracing V&V as the primary way of proving your
system does what you intended and also
meets the needs of the people using it. The resulting V&V effort
has become a significant part of the software development effort
for a medical device. One of the key pieces to demonstrate that
the system is implemented completely
is a Requirements Traceability Matrix (RTM), which documents each
of the requirements traced to design items, code, unit, integration
and system test cases. The RTM is an easy and effective way for
documenting your implementation in a manner to which the FDA is
familiar - what are the requirements, where are they implemented,
and how have you tested them. |
