Developmental learning for user activities

Abstract

This chapter presents a brain-inspired developmental learning system. A personal computer lives with the human user as long as the power is on. It can develop and report some activities of the user like a shadow machine, a virtual machine that runs in the background while the human user is doing its regular activities, on the computer or off the computer. The goal of the teacher of this shadow machine is to enable it to observe human users’ status, recognize users’ activities, and provide the taught actions as desired reports. Both visual and acoustic contexts are used by this shadow machine to infer the user’s activities (e.g., in an office). A major challenge is that the system must be applicable to open domains - without a handcrafted environmental model. That is, there is no handcrafted constraint on office lighting, size, setting, nor requirements of the use of a head-mounted close-talk microphone. A room microphone sits somewhere near the computer. The distance between the sound sources and the microphone varies significantly. This system is designed to respond to its sensory inputs. A more challenging issue is to make the system adapt to different users and different environments. Instead of building all the world knowledge in advance (which is intractable), the system’s adaptive capability enables it to learn sensorimotor association (which is tractable). The real-time prototype system has been tested in different office environments.