Verification and Validation of Knowledge Based Systems

Abstract

In the real world, a knowledge-based system (KBS) must often accommodate a considerable number of references which support the particular knowledge domain. The size of such a knowledge repository makes its detailed verification challenging and subsequent maintenance onerous. New technology can help improve both the verification and maintenance of these knowledge repositories. To investigate the effectiveness of new technologies for verification and maintenance, we developed two subsequent versions of a KBS designed to improve the consistency of software measurement using ISO 19761 (the COSMIC-FFP measurement method for software functional size) and COSMIC- FFP guide [3]. The COSMIC-FFP KBS consists of a hybrid knowledge system built on case-based and ruled-based approaches. The first prototype was built in 2000 using Microsoft technology (Visual Basic 6, Access 2000, hyperlink facilities for RTF files). This prototype included 105 case problems and almost 800 files (hyperlinks) for the required references. Because of the high number of files, the verification and validation of this KBS was, of course, very challenging. This led us to design a second KBS, a Web-based prototype (XML, XSL, Java Server Page) which is much easier to verify and validate, leading to considerably improved maintainability and expandability. This paper presents an overview of the selected hybrid KBS approach and of the first and second prototypes. It also illustrates the transitioning and quantitative benefits for the detailed KBS verification and validation process and the lessons learned during this process.