Journal of Computer Science 5 (5): 392-397, 2009
ISSN 1549-3636
© 2009 Science Publications
392

Measuring the Software Product Quality during the Software Development Life-Cycle:
An International Organization for Standardization Standards Perspective

Rafa E. Al-Qutaish
Department of Software Engineering, Alzaytoonah University of Jordan,
Airport Street, Amman 11733, Jordan

Abstract: Problem statement: The International Organization for Standardization (ISO) published a
set of international standards related to the software engineering, such as ISO 12207 and ISO 9126.
However, there is a set of cross-references between these two standards. Approach: The ISO 9126 on
software product quality and ISO 12207 on software life cycle processes had been analysed to
investigate the relationships between them and to make a mapping from the ISO 9126 quality
characteristics to the ISO 12207 activities and vers versa. Results: This study presented a set of
comments and suggestions to improve the ISO 9126. Conclusion: The weaknesses of the cross-
references between the two ISO standards had been highlighted. In addition, this study provided a
number of comments and suggestions to be taken into account on the next version of the ISO 9126
international standard.

Key words: ISO 9126, ISO 12207, ISO software engineering standards, software life cycle processes,
software product quality evaluation, software product measurement

INTRODUCTION

The last decade of 20th century has seen a rapid
increase in the use of the software products in a variety
of application areas and their correct operation is often
critical for business success and human safety. This
trend is expected to continue in light of the progress and
utilization of the measurement evaluation of software
product quality, particularly from the international
standardization perspective.
In the area of software engineering, the concept of
software measurement (or what is commonly called
software ‘metrics’) is not new. Since 1972, a number of
so-called software ‘metrics’, or ‘measures’, have been
developed.
In order to standardize the software product quality
measurement process, in 1991, the ISO published its
first international consensus on the terminology for the
quality characteristics for software product evaluation:
This standard was called “software product evaluation-
quality characteristics and guidelines for their use”
(ISO 9126: 1991)[5]. From 2001-2004, the ISO
published an expanded version, containing both the ISO
quality models and inventories of proposed measures
for these models (ISO 9126 parts 1, 2, 3 and 4)[9,11,12].
Since 1991, ISO produced a number of
International Standards (IS) and Technical Reports
(TR) related to the software engineering. However, in
this study we have highlighted the ISO 9126 metrics
and their relationships to the ISO 12207 software life
cycle processes and activities. In other words, this study
explains which software metrics you can compute
during the software life cycle.
In 2002, ISO produced the ISO 15939 international
standard[8] which contains the definitions of the terms to
be used in the software measurement process, including
the term ‘measure’ instead of the term ‘metrics’.
However, in this study we will use the term ‘metrics’ in
order to be aligned with the four parts of the ISO 9126
international standard.
The current version of the ISO 9126 international
standard consists of inventories of proposed metrics to
measure the quality of the internal, external and in-use
software product. However, for each of these metrics
there is a cross-reference on where they could be
measured during the ISO 12207 Software Life Cycle
Processes and activities (SLCP). This study provides a
mapping between these two standards to highlights the
weaknesses of these cross-references and proposes a
way to address them and it provides an approach to use
the ISO 9126 metrics to measure the software quality
during the software life cycle.
The International Organization for Standardization
(ISO) published a set of international standards related
to the software engineering. However, some of these
international standards have many relationships
between them. This study investigates the relationships
between ISO 9126 on software product quality and ISO
12207 on software life cycle processes.

J. Computer Sci., 5 (5): 392-397, 2009

393
However, the relationships between them are
many-to-many relationships since each of ISO 9126
characteristic/subcharacteristic could be measured in
different ISO 12207 software life cycle processes and
activities and in each ISO 12207 software life cycle
process or activity we can measure different ISO 9126
characteristics/subcharacteristics. Furthermore, this
study highlights the weaknesses and the cross-
references between the two ISO standards. Finally, a
number of suggestions and comments have been
addressed.
Recently, the ISO has recognized a need for
further enhancement of ISO 9126 International
Standard, primarily as a result of advances in the
fields of information technologies and changes in
environment[3]. Therefore, the ISO is now working on
the next generation of software product quality
standards[12], which will be referred to as Software
Product Quality Requirements and Evaluation
(SQuaRE-ISO 25000 series). This series of standards
will replace the current ISO 9126 International
Standard. However, many researches have focused on
some weaknesses on the current ISO 9126 and even
on the draft versions of the upcoming new ISO 25000
series of standards (SQuaRE)[1,2].

MATERIALS AND METHODS

ISO 9126 on software quality measurement: The
ISO 9126 series of standards now consists of one
international standard[7] and three technical
reports[11,12]:

• ISO 9126-1: Quality model[7]
• ISO TR 9126-2: External metrics[9]
• ISO TR 9126-3: Internal metrics[10]
• ISO TR 9126-4: Quality in-use metrics[11]
The first document of the ISO 9126 series-Quality
model-contains two-part quality model for software
product quality[7]:

• Internal and external quality model
• Quality in-use model

The first part of the two-part quality model
determines six characteristics in which they are
subdivided into twenty-seven subcharacteristics for
internal and external quality, as in Fig. 1[7]. These
subcharacteristics are a result of internal software
attributes and are noticeable externally when the
software is used as a part of a computer system. The
second part of the two-part model indicates four quality
in-use characteristics. All the quality characteristics and
their corresponding sub-characteristics are defined in
ISO 9126-1[7].
The 2nd, 3rd and 4th documents of the ISO 9126
series provide the following information[11,12]:

• Sets of metrics for each external quality sub-
characteristic, internal quality sub-characteristic
and quality in-use characteristic
• Explanations of how to apply and use these sets of
metrics
• Examples of how to apply these metrics during the
software product lifecycle

ISO 12207 on software life cycle processes: It consists
of processes, activities for each process and tasks for
each activity[5,7]. Figure 2 shows the software life cycle
processes, the number of activities in each process and
the number of tasks in each process. The full list of the
process, activities and tasks can be seen in ISO 12207[6]
and IEEE/EIA 12207[4] (the IEEE/EIA 12207 is the
IEEE version of the ISO 12207).



Fig. 1: ISO 9126 quality model for external and internal quality (characteristics and sub-characteristics)[7]

J. Computer Sci., 5 (5): 392-397, 2009

394


Fig. 2: ISO 12207 software life cycle processes, activities and tasks

The ISO 12207 software life cycle processes are
grouped into three broad classes: Primary; supporting
and organizational. Primary processes are the prime
movers in the life cycle; they are acquisition, supply,
development, operation and maintenance. Supporting
processes are documentation, configuration
management, quality assurance, joint review, audit,
verification, validation and problem resolution. A
supporting process supports another process in
performing a specialized function. Organizational
processes are management, infrastructure, improvement
and training. An organization may employ an
organizational process to establish, control and improve
a life cycle process.

RESULTS

For each metric of the internal, external and in-use
metrics, the ISO 9126 parts 2-4 provides the following
information: metric name, purpose of the metric,
method of application, measurement formula,
interpretation of measured value, metric scale type,
measure type, input to measurement, ISO 12207 SLCP
Reference and target audience[11,12].
Hereinafter we will discuss the detailed mappings
between the ISO 9126 quality metrics of the Internal,
external and in-use software product and the ISO 12207
software life cycle processes and activities will be
provided. Furthermore, this mapping will focus on an
investigation of the “ISO 12207 Software Life Cycle
Processes (SLCP) References” provided by ISO 9126
for each of its metrics.

Internal quality metrics: Within the ISO 9126-3 on
software product internal quality metrics, there are 70
metrics. These metrics can be measured during the
software life cycle. Internal quality defined in ISO
9126-1 as the totality of characteristics of the software
product from an internal view. Internal quality is
measured and evaluated against the internal quality
requirements. Details of software product quality can
be improved during code implementation, reviewing
and testing, but the fundamental nature of the software
product quality represented by internal quality remains
unchanged unless redesigned[10].
Table 1 shows the number of internal quality
metrics which can be measured during each of the ISO
12207 software life cycle processes. For example,
within the “verification process” (of the “supporting
processes”) 59 metrics can be measured.

J. Computer Sci., 5 (5): 392-397, 2009

395
Table 1: Mapping between the ISO 9126 internal quality metrics and
the ISO 12207 processes
ISO 12207 processes No. of related ISO 9126 metrics
5.0 Primary life cycle processes:
5.1 Acquisition 0
5.2 Supply 0
5.3 Development 1
5.4 Operation 0
5.5 Maintenance 0
6.0 Supporting life cycle processes:
6.1 Documentation 0
6.2 Configuration management 0
6.3 Quality assurance 2
6.4 Verification: 59
6.5 Validation 13
6.6 Joint review 59
6.7 Audit 0
6.8 Problem resolution 4



Fig. 3: Example on the ISO 9126-3 internal quality
metrics and where they could be measured during
the ISO 12207 software life-cycle processes and
activities

As an example, Fig. 3 shows a detailed structure of
the software product internal quality metrics’ names
and where they can be measured during the verification
process (which is a part of the ISO 12207 supporting
process) along with the corresponding characteristic
and subcharacteristic reference number for each of
those metrics (for example 1.2 means the accuracy
subcharacteristic of the functionality characteristic). In
Fig. 3 only the software life processes/activities which
have internal quality metrics are mentioned.
However, from Table 1 we can note that there is no
any metric which could be measured during 4 out of 5
primary life cycle processes. This means that there is no
any metric from ISO 9126 external quality metrics
could be useful during the acquisition, supply,
operation and maintenance primary life cycle processes.
Table 2: Mapping between the ISO 9126 external quality metrics and
the ISO 12207 processes
ISO 12207 Processes No. of related ISO 9126 Metrics
5.0 Primary life cycle processes:
5.1 Acquisition 0
5.2 Supply 0
5.3 Development 14
5.4 Operation 93
5.5 Maintenance 84
6.0 Supporting life cycle processes:
6.1 Documentation 0
6.2 Configuration management 0
6.3 Quality assurance 14
6.4 Verification 0
6.5 Validation 47
6.6 Joint review 0
6.7 Audit 0
6.8 Problem Resolution 1

Moreover, there is no any metric which could be
measured during 3 out of 8 of the supporting life cycle
processes; that is, documentation, configuration
management and audit processes.

External quality metrics: Within the ISO 9126-2 on
software product external quality metrics, there are 110
metrics. These metrics can be measured during the
software life cycle. External quality defined in ISO
9126-1 as the totality of characteristics of the software
product from an external view. It is the quality when
the software is executed, which is typically measured
and evaluated while testing in a simulated environment
with simulated data using external metrics. During
testing, most faults should be discovered and
eliminated. However, some faults may still remain after
testing. As it is difficult to correct the software
architecture or other fundamental design aspects of the
software, the fundamental design usually remains
unchanged throughout testing[9].
Table 2 shows the number of external quality
metrics which can be measured during each of the ISO
12207 software life cycle processes. For example,
within the “operation process” of the “primary
processes”, 93 metrics can be measured.

Quality in-use metrics: Within the ISO 9126-2 on
software product quality in-use metrics, there are 15
metrics. These the 15 metrics can be measured during
the software life cycle. Quality in-use defined in ISO
9126-1 as the user’s view of the quality of the software
product when it is used in a specific environment and a
specific context of use. It measures the extent to which
users can achieve their goals in a particular
environment, rather than measuring the properties of
the software itself. The term ‘user’ refers to any type of
intended users, including both operators and
maintainers and their requirements can be different[11].

J. Computer Sci., 5 (5): 392-397, 2009

396
Table 3: Mapping between the ISO 9126 quality in-use metrics and
the ISO 12207 processes
ISO 12207 Processes No. of related ISO 9126 metrics
5.0 Primary life cycle processes:
5.1 Acquisition 0
5.2 Supply 0
5.3 Development 12
5.4 Operation 15
5.5 Maintenance 0
6.0 Supporting life cycle processes:
6.1 Documentation 0
6.2 Configuration management 0
6.3 Quality assurance 0
6.4 Verification 0
6.5 Validation 11
6.6 Joint review 0
6.7 Audit 0
6.8 Problem resolution 0

Table 3 shows the number of quality in-use metrics
which can be measured during each of the ISO 12207
software life cycle processes. For example, during the
“software qualification testing” activity of the
“development process” of the “primary processes”, 12
metrics can be measured.

DISCUSSION

In ISO 9126-3, there are some external quality
metrics-as in Table 4 which have been referred to be
measured during the ‘integration’ activity of the
‘development process’ of the ‘primary processes’. But,
within the ‘development’ process, there are two
activities related to the ‘integration’, that is, ‘system
integration’ and ‘software integration’. However, this
document (ISO 9126-3) did not specify during which
‘integration’ activity those metrics can be measured.
It is clearly noted that through the ISO 12207
‘organizational processes’ none of the 195 quality
metrics-from the ISO 9126 series of international
standards-can be measured. Therefore, the ISO 9126 is
not usable for the ISO 12207 organizational processes.
As mentioned in ISO 9126-1, the quality in-use
metrics should be measured during the execution of the
software product in an actual working environment.
However, from Table 4 we can see that there are 12
metrics which could be measured through the ‘software
qualification testing’ activity. But since ISO 12207
mentioned that the “software qualification testing”
activity is a part of the ‘development process’ Thus, it’s
strange and make no sense to measure that 12 metrics.
The ‘joint review process’ of the ‘supporting
processes’ consists of three activities: One of these
activities is the ‘technical reviews’ activity. This
activity contains one task, that is, “Technical reviews
shall be held to evaluate the software products or
services under consideration and provide evidence that
they are complete[6]. Now, if we go back to Table 1,
Table 4: External on quality metrics which is not clear where to be
measured
1 Estimated latent 9 Fault removal
fault density 10 Restartability
2 Incorrect operation avoidance 11 Mean time between
3 Failure density 12 User support
against test cases failures (MTBF)
4 Availability functional consistency
5 Failure resolution 13 Breakdown
6 Mean down time 14 Restore effectiveness
7 Fault density 15 Failure avoidance
8 Mean recovery time 16 Restorability

we will find that there are 59 internal quality metrics
that could be measured during the ‘joint review’
process. Whereas, from Table 2 and 3, it is seen that
there is no any external quality or quality in-use metrics
that can be measured during ‘joint review’ process.

CONCLUSION

The current edition of the ISO 9126 consists of
inventories of proposed metrics to measure the quality
of the internal, external and in-use software product.
However, for each of these metrics there is a cross-
reference on where it could be measured during the ISO
12207 software life cycle processes and activities. This
study provided a mapping between those two standards
to investigate the cross-references between them. Based
on this mapping, the following comments and
suggestions for the upcoming new ISO 25000 series of
standards (SQuaRE) can be concluded:

• There is no any metric can be measured during the
‘organizational processes’
• A number of external quality metrics where
mentioned in ISO 9126-2 to be measured during
the ‘integration’ activity. However, within the ISO
12207 there are two activities labeled ‘system
integration’ and ‘software integration’, Table 4 for
the names of these metrics. However, this will
make the user of the ISO 9126 confused
• Many of the ISO 9126 quality metrics referred to
processes. However, as known, each process in
ISO 12207 contains a number of different activities.
Thus, it is more usable for the ISO 9126 users to
refer to the activities of the ISO 12207. This can be
done using cross-reference numbers from ISO
12207. For example, the cross-reference number
5.3.9 is referring to “primary processes”,
‘development process’ and ‘software qualification
testing’ activity, respectively, Fig. 3 for an example
on where the ISO 9126 internal quality metrics
could be measured within the ISO 12207
verification activities

J. Computer Sci., 5 (5): 392-397, 2009

397
In addition to the mapping in this study, as a future
work, it is a good idea to investigate where to collect
the data for each of the ISO 9126 quality metrics in the
ISO 12207 software life cycle processes and activities.
This will save time and assure that the data have been
completely collected before the measurement of the
metrics is performed.

REFERENCES

1. Abran, A., R.E. Al-Qutaish and J.M. Desharnais,
2005. Harmonization issues in the updating of the
ISO standards on software product quality. Metr.
News J., 10: 35-44.
http://www.gelog.etsmtl.ca/publications/pdf/975.pdf
2. Abran, A., R.E. Al-Qutaish, J.M. Desharnais and
N. Habra, 2005. An information model for
software quality measurement with ISO standards.
Proceedings of the International Conference on
Software Development, (SWDC-REK'05),
Reykjavik, Iceland, pp: 104-116.
http://www.lrgl.uqam.ca /publications/pdf/904.pdf
3. Azuma, M., 2001. SQuaRE: The next generation of
ISO/IEC 9126 and 14598 international standards
series on software product quality. Proceedings of
the European Software Control and Metrics
Conference, (ESCOM’01), London, UK., pp: 337-346.
http://miro.sztaki.hu/husei/events/SQUAD/reload
/SQUAD-SQuaRE-Budapest-vers1.ppt
4. IEEE, 1996. IEEE/EIA-12207: Information
technology-software life cycle processes. The
Institute of Electrical and Electronics Engineers,
New York, USA.
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&a
rnumber=707581&isnumber=15306
5. ISO., 1991. ISO/IEC IS 9126-software product
evaluation-quality characteristics and guidelines
for their use. International Organization for
Standardization, Geneva, Switzerland. http://www.tbs-
sct.gc.ca/its-nit/standards/tbits26/crit26-eng.asp
6. ISO., 1995. ISO/IEC 12207-information
technology-software life cycle processes.
international organization for standardization.
Geneva, Switzerland.
http://www.iso.org/iso/isocatalogue/cataloguetc/cat
aloguedetail.htm?csnumber=21208







7. ISO., 2001. ISO/IEC 9126-1: Software
engineering-product quality-part 1: Quality model.
International Organization for Standardization,
Geneva, Switzerland.
http://www.iso.org/iso/isocatalogue/cataloguetc/cat
aloguedetail.htm?csnumber=22749
8. ISO., 2007. ISO/IEC IS 15939: Software
engineering-software measurement process.
International Organization for Standardization,
Geneva, Switzerland.
http://www.iso.org/iso/isocatalogue/cataloguetc/cat
aloguedetail.htm?csnumber=44344
9. ISO., 2003. ISO/IEC TR 9126-2: Software
engineering-product quality-part 2: External
metrics. International Organization for
Standardization, Geneva, Switzerland.
http://www.iso.org/iso/iso_catalogue/catalogue_tc/
catalogue_detail.htm?csnumber=22750
10. ISO., 2003. ISO/IEC TR 9126-3: Software
engineering-product quality-part 3: Internal
metrics. International Organization for
Standardization, Geneva, Switzerland.
http://www.iso.org/iso/iso_catalogue/catalogue_tc/
catalogue_detail.htm?csnumber=22891
11. ISO., 2004. ISO/IEC TR 9126-4: Software
engineering-product quality-part 4: Quality in use
metrics. International Organization for
Standardization, Geneva, Switzerland.
http://www.iso.org/iso/iso_catalogue/catalogue_tc/
catalogue_detail.htm?csnumber=39752
12. Suryn, W., A. Abran and A. April, 2003. ISO/IEC
SQuaRE: The second generation of standards for
software product quality. Proceedings of the 7th
IASTED International Conference on Software
Engineering and Applications, (ICSEA’03),
California, USA., pp: 1-9. http://paginaspersonales.
deusto.es/cortazar/doctorado/articulos/2003/square
799.pdf