Universidad Autónoma Madrid

Dept. Ingeniería Informática

28049 Madrid (SPAIN)

{alejan,corbacho}@ii.uam.es

Tel.: +34-1-397-4319 Fax: +34-1-397-5277

This paper attempts to provide a theory of organization for the analysis and design of Adaptive Autonomous Agents (AAA). By autonomous we mean that the system can perform many of its "survivability functions" as well as play a critical role in selecting its own training set while deciding for itself how to relate perceptions to actions. Different disciplines have attempted to develop design theories for AAA: Artificial Intelligence, Control Theory, and Neural Networks, to name a few. They all have provided with partial success in restricted tasks and domains, nevertheless, none has yet provided an overall integrated theory. On the other hand, biological agents are flexible, adaptable, and highly robust; and they are so at all levels of organization and functionality. These agents "designed" by evolution survive in natural (usually very complex) environments where no current artificial agent could.

So we claim that fundamental principles have yet escaped current theories for AAA. We must rethink in a principled way what are the principles of organization, and what are the unit(s) of organization. In this paper we propose a theory of organization for more robust, flexible and adaptive AAA inspired by the organization of their biological siblings. One of the backbones of our theory is based on the general idea that AAA should be designed to maintain coherence with their environment. Coherence of a particular agent with respect to a particular environment is a measure of the congruence between the result of an interaction with the environment and the expectations the agent has for that interaction. We will then claim that coherence is very much related to survival. In this respect we will introduce the Coherence Maximization Principle (CMP): Given a current interaction with the environment measure the coherence of the expectation with respect to the current results of the interaction, and try to maximize the coherence by adaptation.

We propose that the design of the overall agent (natural or artificial) should be done (has been done by evolution) by aggregation of both units of interaction with the environment, as wells as "dual" units of expectation about those interactions (formed from successful experiences in similar interactions). Thus, by allowing the system to compare its expectations with the actual results obtained after the interactions, the system may focus its adaptation mainly on the incoherent units. We, therefore, introduce the Bootstrap Coherence Principle (BCP) which defines which initial units of interaction and expectation the agent should be initially provided with to achieve a particular chance of survival in a particular environment. Bootstrap coherence provides the agent with the ability to anticipate the results of some of its "seed" interactions, and, thus, expect certain changes in its internal representations accordingly. All the seed schemas included by BCP can be considered the initial bias for the adaptive system. Yet this bias reflects the regularities in the interactions between the agent and the environment.

Schema-Based Learning (SBL) is the architecture implementing these principles of organization. It provides with a generalized framework incorporating general principles of adaptive organization, e.g., CMP and BCP. The basic unit of organization in SBL is the schema. A schema is an evolutionary constructed or experience-based constructed unit of interaction (perceptual, motor and reactive schemas) or expectation (predictive schema). Animal behavior is controlled by significant patterns of interaction which are useful again and again, e.g., grasping. They provide a somehow vague description of a situation and action, and on the other hand are specific enough to be applicable. They represent what is stable and therefore generalizable over variability becoming precise through adaptation (Arbib, 1992).

One of the main goals of SBL is to grow increasingly complex patterns of interactions (schemas) from an initial stock (seed schemas) in an harmonious way which does not destroy the primitive basic functionality but which enhances, and increments it. SBL relies on two related important aspects/assumptions:

i) It is advantageous (efficient) to cope with new experiences based on positive results from previously similar experiences. It is efficient since the agent avoids the statistical estimation required to form the past successful action-perception correlations. The current stock of schemas allows to perform a broad-brush analysis of the action-perception "scene" e.g. prey detection/catching, rude grasping in babies. SBL does not replace statistical estimation, rather, it confines it to a narrow search space (initial islands of reliability), so that the system learns in the right ballpark with minimum scene statistics.

ii) SBL allows for the incremental construction of increasingly more complex schemas by aggregation of schemas (stable units of composition) to reflect increasingly more complex interactions with the environment. These new constructed schemas may then be tuned by experience into a well-adapted integrated unity. Without this, given a new task, the system would start ab initio to train a new unstructured network dedicated to the task. Thus, learning is based on the current stock of schemas (seeds). Ultimately SBL must be able to explain the design of the set of "seed" schemas as they are the basis for the construction of more complex ones.