|
What is Verification
and Validation?
FDA Definition and Explanation
VALIDATION
Software validation is "establishing by objective evidence that all
software requirements have been implemented correctly and completely and are
traceable to system requirements" [Ref: NIST 500-234]. Software validation
is essentially a design verification function as defined in FDA's Quality
System Regulation (21 CFR 820.3 and 820.30) and includes all of the
verification and testing activities conducted throughout the software life
cycle.
Design validation encompasses software validation, but goes further to check
for proper operation of the software in its intended use environment.
VERIFICATION
Verification is defined in FDA's Quality System Regulation (21 CFR 820.3) as
"confirmation by examination and provision of objective evidence that
specified requirements have been fulfilled." In a software development
environment, software verification is confirmation that the output of a
particular phase of development meets all of the input requirements for that
phase.
IEEE Definition
VALIDATION
"Confirmation by examination and provisions of objective evidence that
the particular requirements for a specific intended use are fulfilled."
VERIFICATION
"Confirmation by examination and provisions of objective evidence that
specified requirements have been fulfilled."
Using the above definitions in software development
Validation, in its simplest terms, is the demonstration that the software
implements each of the software requirements correctly and completely. In other words, the "right product was built."
Verification is the activity which ensures the work
products of a given phase fully implement the inputs to that phase, or
"the product was built right."
Levels of Verification
There are four levels of verification:
|
|
|
Component
Testing
Testing conducted to verify the implementation of the design for one software
element (unit, module) or a collection of software elements.
|
|
|
Integration
Testing
An orderly progression of testing in which various software elements and/or
hardware elements are integrated together and tested.
This testing proceeds until the entire system has been
integrated.
|
|
|
System Testing
The process of testing an integrated hardware and software system to verify
that the system meets its specified requirements.
|
|
|
Acceptance
Testing
Formal testing conducted to determine whether or not
a system satisfies its acceptance criteria and to enable the customer to
determine whether or not to accept the system.
|
|
Types of Verification
There are four types of verification that can be applied to the various
levels outlined above:
|
|
|
Inspection
Typical techniques include desk checking, walkthroughs, software reviews,
technical reviews, and formal inspections (e.g., Fagan approach).
|
|
|
Analysis
Mathematical verification of the test item, which can include estimation of
execution times and estimation of system resources.
|
|
|
Testing
Also known as "white box" or "logic driven" testing.
Given input values are traced through the test item
to assure that they generate the expected output values, with the expected
intermediate values along the way. Typical techniques include statement coverage,
condition coverage, and decision coverage.
|
|
|
Demonstration
Also known as "black box" or "input/output driven"
testing. Given input values are entered, and the
resulting output values are compared against the expected output values.
Typical techniques include error guessing, boundary-value analysis, and
equivalence partitioning.
|
|
Explanation
The four methods for verification can be used at any of the levels although some
work better than others for a given level of verification.
As an example, the most effective way to find anomalies at the component
level is inspection. On the other hand, inspection is not applicable at the
system level (you don't look at the details of code
when performing system level testing). A logical approach to testing is to
utilize techniques and methods that are most effective at a given level.
Component level verification can easily get very expensive. Companies need to
avoid making statements like "all paths and branches will be executed
during component testing." These statements make for a very expensive
test program, as all code developed is required to have one of the most
labor-intensive type of testing performed on it. To
minimize the costs of component verification, the V&V group develops
rules for determining the type of verification method(s) needed for each of
the software functions. As an example, very low complexity software function,
which is not on the safety critical list, may only need informal inspection
(walkthrough) performed. Other complicated functions typically require white
box testing since the functions become difficult to determine how they work.
We recommend performing inspections before doing the white box testing for a
given module as it is less expensive to find the
errors earlier in the development.
The FDA is embracing V&V as the primary way of proving your system does
what you intended and also meets the needs of the
people using it. The resulting V&V effort has become a significant part
of the software development effort for a medical device. One of the key
pieces to demonstrate that the system is implemented
completely is a Requirements Traceability Matrix (RTM), which documents each
of the requirements traced to design items, code, unit, integration and
system test cases. The RTM is an easy and effective way for documenting your
implementation in a manner to which the FDA is familiar - what are the
requirements, where are they implemented, and how have you tested them.
|
