The purpose of conformance testing is to determine to what extent a single
implementation of a particular standard conforms to the individual requirements
of that standard.
The important factors which characterize conformance testing are as follows:
the System or Implementation Under Test (SUT or IUT) defines the boundaries (open
interfaces) for testing;
- the conformance test system is a specialized tool (system) built for the purpose of
testing and on which specific test scripts can be run;
- the SUT comes from a single supplier (or, at least, a single product line);
- the tests are executed by a dedicated test system that has full control of the SUT and the
ability to observe all communications from the SUT;
- the tests
are performed at open standardized interfaces that are not (usually) accessible
to a normal user (i.e. they are specified at the protocol level);
- the tests are specified at the detailed protocol level and are not usually based on
functionality as experienced by a user.
Because the conformance tester maintains a high degree of control over the
sequence and contents of the protocol messages sent to the IUT it is able to
explore a wide range of both expected and unexpected (invalid) behaviour.
Development of ETSI Conformance Test Specifications
Most ETSI test specifications are developed according to the well-proven
methodology defined in ISO/IEC 9646. This standard, which covers the entire test
development process, provides an excellent basis for the production of
high-quality test frameworks and specifications.
The following diagram shows the development of test specifications process,
acronyms are explained below.
Building a coherent set of test specifications from disperse requirements
sources can be simplified by gathering the requirements together into a single
catalogue. A Requirements Catalogue should list all implementation requirements
from the various sources and organize them into an appropriate structure.
The Implementation Conformance Statement (ICS) is a checklist of the
capabilities defined in the standard. When filled in by the user, the ICS provides an overview of the features and options that are implemented by any
given IUT. The ICS can be used to select and parameterize test cases and as an
indicator for basic interoperability between different products.
The Implementation eXtra Information for
Testing (IXIT) contains additional
information (specific addresses, timer values etc.) necessary for testing.
In the specific case of protocol standards the ICS is known as the Protocol
ICS or PICS and the IXIT is known as the Protocol IXIT or PIXIT.
Test Purposes (TP) provide an informal, easy-to-read self-contained
description of each test, concentrating on what is to be tested rather than how
the actual test may be achieved. They are derived from the relevant base
standards or from the appropriate Requirements Catalogue. The set of all Test
Purposes is grouped into a logical Test Suite Structure (TSS) according to
suitable criteria (basic interconnection, error handling, functionality etc.).
The standardized notation TPLAN is a good way to write consistent Test Purposes.
The Abstract Test Suite (ATS) is the entire collection of Test Cases,
sometimes known as test scripts. Each Test Case specifies the detailed coding of
the Test Purposes, written in the standardized test specification language TTCN-3.
The actual Executable Test Suite (ETS) can be quickly and easily implemented
from the ATS using the TTCN compilers available on many test tool platforms.
Finding ETSI Test Specifications
ETSI test specifications can be downloaded via the
ETSI Work Programme application.
Keywords can be added to facilitate your search, the following are related to testing:
ATS (Abstract Test Suite)
PICS (Protocol Implementation Conformance Statement)
- ICS (Implementation Conformance Statement)
- TSS&TP (Test suite structure and Test purposes)
To select more than one, hold the Ctrl button and click on the keywords
required. When more than one keyword is selected, you can combine them either
with the OR function ("one of") or the AND function ("all").
The following is a list of recently published and frequently downloaded
standards. Please use the
ETSI WorkProgramme to find further related standards.