IDES: Self-adaptive software with online policy evolution extended from rainbow

Abstract

One common approach or framework of self-adaptive software is to incorporate a control loop that monitoring, analyzing, deciding and executing over a target system using predefined rules and policies. Unfortunately, policies or utilities in such approaches and frameworks are statically and manually defined. The empirical adaptation policies and utility profiles cannot change with environment thus cannot make robust and assurance decisions. Various efficiency improvements have been introduced to online evolution of self-adaptive software itselfhowever, there is no framework with policy evolution in policy-based self-adaptive software such as Rainbow. Our approach, embodied in a system called IDES (Intelligent Decision System) uses reinforcement learning to provides an architecture based self-adaptive framework. We associate each policy with a preference value. During the running time the system automatically assesses system utilities and use reinforcement learning to update policy preference. We evaluate our approach and framework by an example system for bank dispatching. The experiment results reveal the intelligence and reactiveness of our approach and framework. © 2012 Springer-Verlag Berlin Heidelberg.