Cognitive Algorithms and Systems of Error Monitoring, Conflict Resolution and Decision Making

Abstract

There are currently several approaches to decision making in complex systems, particularly in robotics. In most cases, the decision-making process resembles the well-known control or sense–think–act loop: the process output or state is sensed, its deviation (error) from the desired value is continuously monitored and, based on some appropriate algorithm, a control action is picked from the available action set to be applied to the process, so that the loop is closed and the decision-making process moves to its next iteration. The strict-sense control approach typically handles continuous state-space sys- tems, driven by time (either discrete/sampled or continuous), where the controller algorithm implements a closed-form function that maps errors in continuous state- space to control signals from a continuous control-space. On the other hand, the wider-sense approach to decision making often handles discrete state-space, event- driven systems,where the monitored error is used to resolve a conflict among actions from a discrete set, typically following an optimization algorithm that attempts to maximize a payoff function over some finite or infinite horizon of steps. Examples of the former are the control of chemical processes or of mechanical devices, such as robot joints, while the latter concerns, e.g. scheduling in manufacturing systems or conflict resolution while executing conditional plans in robotics. In this chapter, we will address the problem of plan representation, analysis and execution in multi-robot systems using a well-known formal model of computation: Petri nets. Therefore, we will focus on the discrete state-space, event-driven model, so as to view a multi-robot plan as a discrete event system (DES) (Cassandras and Lafortune 2007). Most of the existing robot task models are not based on formal approaches but tailored to the task at hand, usually leading to task plans with few primitive actions, simply because increasing their number and the plan complexity may lead to unexpected results, not predicted by any analysis studies.ApplyingDES concepts and theory to model (multi-)robot tasks provides a systematic approach to modelling, analysis and design, scaling up to realistic applications, and enabling analysis of formal properties, as well as design from specifications. In particular, representingmulti-robot plans by Petri nets enables tackling a con- siderable number of issues:  A plan is seen as control policy that maps states onto actions  Control policies are, in general, non-deterministic, as there may exist more than one possible action for a given state  A plan represented as a Petri net can be executed by simply following Petri net firing rules, where (sequential) decision-making algorithms are used for conflict resolution whenevermore than one action are available for a given state  Using a qualitative untimed Petri net view, plans can be analysed regarding their formal properties, e.g. using algorithms that address Petri net analysis problems (such as conservation, blocking, liveness, invariants)  Using a quantitative stochastic timed Petri net view, plans can be analysed re- garding their performance under uncertainty, e.g. using closed form algorithms and/or Monte Carlo simulations that address Petri net stochastic performance (such as plan success probability, plan robustness) We will review existing Petri net-based approaches to robot plan representation, both froma qualitative untimed viewand froma quantitative stochastic view.Details on the two Petri net views, analysis problems and existing techniques for their solu- tion will be provided.We will then introduce a multi-robot task model composed of primitive actions, behaviours, predicates to represent the system state and events to trigger state transitions. These conceptswill be defined and mapped onto Petri nets. Next, we will explain how the qualitative and quantitative views help analysing dif- ferent problems regarding plan formal properties and performance in the presence of uncertainty in the action effects. Then,we will state when does a stochastic Petri net representation of a plan map onto a Markov chain, enabling the use ofMarkov chain theory to analyse plans performance in closed form. Adding controllable actions to themulti-robot task model finally leads to a controlledMarkov chain or MarkovDe- cision Process (MDP), for which different solutions of the planning problems (seen as the determination of a control policy) exist in the literature. Among those, we will describe how to use reinforcement learning to resolve control policy conflicts between alternative controllable actions for the same state. The chapter is organized as follows: in the next section, we will review some of the Petri net-based approaches to robot task modelling described in the literature, the formal models they introduced and some of the results obtained. Section15.3 introduces our proposed multi-robot task model, the corresponding plan representa- tion by Petri nets, how to analyse plan performance in the presence of uncertainties and some examples of application to robot soccer. The last section will summarize what has been accomplished so far under this line of research, the success stories, limitations found, future challenges and some suggestions on how to address their solution. Throughout the chapter,most provided details concern theoretical aspects, while results are presented only for the purpose of illustrating the main concepts, as some of them are still preliminary. Petri net analysis techniques, Markov chain and sequential decision making under uncertainty (including reinforcement learning) topics will assume basic knowledge about them by the reader and introduce only definitions required for the sake of understanding their relation to the robot task model also introduced here.