30 E. Knauss, C. El Boustani, and T. Flohr
help to enhance the quality of SRS, but do not help to quantifiable compare it.
Nevertheless, the quality aspects in these textbooks are the foundation of our
work.
Davis [3] gives some suggestions on how to quantify the quality of a require-
ments document: Findings are weighted according to their severity. Accordingly,
the overall quality of a document is the amount of findings multiplied by their
weight. We integrated this suggestion as well as the proposed weights into our
approach. In order to compare two SRS, we had to normalize the result by the
amount of their requirements.
Besides the rather basic quality-metrics for requirements discussed above,
many more sophisticated ones were suggested in science. For example, the clear-
ness of terms is discussed in the CLEAR-method [7]. Another example is the
discussion of the ambiguity of and respectively or in natural language [8]. Based
on our measurement goals we had to discard these promising metrics, because
they either constructively influenced the RE-process or were simply too difficult
to measure.
3 Study Goals
The GQM (Goal-Question-Metric) method suggests a top-down way of goal-
oriented measurement. The basic steps are defining measurement goals, describ-
ing the goals in a more detailed way using a table (the Abstraction Sheet), to
formulate questions, and to derive metrics from the Abstraction Sheet that help
to answer the questions in a quantifiable way (see [9]).
By filling out the Abstraction Sheets we formulated hypotheses about how
good the quality goals are reached at the moment. Those hypotheses are expected
measurements results. After the elicitation of data we are able to verify the
hypotheses and determine if they were correct or not. We only give a sketch of
our GQM-tree, examples of metrics, and our main hypotheses here.
Formal Req.
Quality
Content-related Req. Quality
#
C
r
i
t
i
c
a
l

T
y
p
o
s
G
r
a
m
m
a
r
R
u
l
e
s

o
f

E
x
p
r
e
s
s
i
o
n
A
m
b
i
g
u
o
u
s

t
e
r
m
s
E
x
i
s
t
.

I
d
e
n
t
i
f
i
e
r
U
n
e
x
p
l
a
i
n
e
d

t
e
c
h
.

t
e
r
m
s
C
o
n
t
r
a
d
i
c
t
o
r
i
n
e
s
s
C
o
m
p
l
e
t
e
n
e
s
s
V
e
r
i
f
i
a
b
l
e

g
o
a
l
s

o
f

r
e
q
.
C
o
r
r
e
c
t
n
e
s
s
R
e
d
u
n
d
a
n
c
y
F
e
a
s
i
b
i
l
i
t
y
N
e
c
e
s
s
a
r
y
C
o
n
t
r
a
d
i
c
t
o
r
i
n
e
s
s

(
b
e
t
.

r
e
q
.
)
L
e
g
a
l
l
y

c
l
a
s
s
i
f
i
e
d
A
s
s
i
g
n
e
d

p
r
i
o
r
i
t
y
O
u
t

o
f

d
a
t
e
R
e
q
.

w
i
t
h
o
u
t

C
a
t
e
g
o
r
y
#
T
e
c
h
n
i
c
a
l

C
a
t
e
g
o
r
i
e
s
E
x
i
s
t
.

U
I

d
e
s
c
r
i
p
t
i
o
n
E
x
i
s
t
.

U
s
e
r

p
r
o
f
i
l
e
I
/
O
-
D
e
v
i
c
e
s

d
e
s
c
r
i
p
t
i
o
n
E
x
i
s
t
.

Q
u
a
l
i
t
y

M
o
d
e
l
S
p
e
c
i
f
i
e
d

Q
u
a
l
i
t
y

G
o
a
l
s
E
x
i
s
t
.

m
e
t
r
i
c
s
Fig. 1. Measurement goals and metrics for Requirements Quality

40 E. Knauss, C. El Boustani, and T. Flohr
or less the same. Therefore, we expect that the ranking of the projects will be
more or less the same, despite different quality-scores. Bad projects will be rated
worse than good projects.
Another problem is the complexity and long duration of the assessment. An
assessment of a typical SRS took between 120 and 180 minutes. In addition, some
of the measurements are rather complex, like finding inconsistencies between
different requirements.
Based on these threats to validity there are also some issues concerning the
mapping between SRS quality and project success. Because of the different scores
the reference project got depended on the reviewer, there is no absolute threshold
for the quality-score. However, with all reviewers creating a consistent ranking
of the projects, we can give a relative quality thresholds.
Table 5 shows types of measurement faults we identified. The first column
gives the type and the second column a typical example. The third column
shows how we would address this problems in future work.
Table 5. Overview of Measurement Faults
Type Example Mitigation Strategy
1 False Positives found passive voice ? training
but was active voice ? keyword-lists
2 Scope of Interpretation identification of ? heuristic tool
process words support
3 Generally subjective understandability ? quantify and
measures limit the effects
6 Conclusion and Outlook
In this work we had two goals. On the one hand we wanted to measure the
quality of software requirements specifications. On the other hand we wanted to
depict project risk based on this quality.
In order to do this, we had to investigate a set of software projects. Based
on the GQM-method we could support our hypotheses that the quality of a
requirement specification influences the probability of project success. In our
setting we were even able to give a threshold: projects that are worse than this
value are very likely to fail.
With this assessment of the SRS’ quality we have a powerful instrument for
our software projects: Based on the results we can decide whether a project is
allowed to go on or whether its SRS needs major rework. Based on our assessment
we also observed that the overall quality of our projects’ SRS increased since
last year. Such observations are essential for software organizations that want to
improve themselves, because the effects of process improvements become visible.
As a side-effect we found out that even simple verbalization policies (e.g. the
requirement template [6] or the use case template [19]) strongly improved the
requirement quality. Such policies simply let fewer room for errors.