AI-driven conversational bot test automation using industry specific data cartridges

Abstract

The paper describes an in-house accelerator to generate alternate synonymous sentences and similar intent from sample utterances, the generated data can be applied as test input for conversational AI bots (either text or voice-based). Its NLP-driven sentence generator exposes a RESTful service, which can be consumed by automated testing tools/frameworks such as Katalon, Selenium, and so on. The paper presents building an accelerator to quickly teach and test adaptive conversational AI bots. The approach helps to analyze user inputs and extract intent, the bot developer should ensure a variety of possible utterances are coded. In the traditional manual approach, it is difficult to conceive every possible user utterance before deploying the bot and hence the bot has an early failure rate. This may diminish the usefulness of the bot and the users may stop using the same. Here we propose an AI-driven bot test automation approach using a patent-pending in-house accelerator referenced as LemmaCartridge (LC) in this paper. Testing tools or frameworks can consume LC's data cartridge API for testing the bot AUT and analyze the responses using automated tools/frameworks like Katalon, Selenium and so on until the bot demonstrates desired outcomes under the supervised train, test and adaptive repeatable testing methods yielding quality@speed for the single major goal of testing conversational AI bots. An example of a program used in experiment is described and the results obtained, especially train and test state machines, industry-specific data cartridges that enable to unearth errors in the AI bot under test, are presented.