WebMindä

Self-Organizing, Network-Based Business Intelligence

By Dr. Ben Goertzel, Chief Technology Officer, Chairman of the Board

ben@intelligenesis.net

Contents

• Webmind Network Intelligence versus Conventional AI
• WebMind Yields Dramatic Financial Intelligence Results
• The Network is the Computer is the Mind

• The Psynet Model of Mind
• From Theory to Implementation via Six Simple Questions

• The Psynet
• Basic Node Types
• Language Processing Integrated with Thought Processing
• Genetic Algorithms
• Symbol Grounding
• Reason
• Query Processing
• Making Intelligence Emerge Across Networks

• Financial Decision Support
• Textual Information Retrieval
• Data Mining

Knowledge Management and Knowledge Creation
Enterprise Analysis, Workflow and Business Process Reengineering

WebMindä is a Java-based software system which evolves its own "digital intuition," and using this intuition, poses and answers questions regarding information. It deals with textual and numerical information on an equal footing, freely making generalizations spanning different types of data. Unlike any other AI system, it forms its own intuitions regarding concepts by dynamically "grounding" linguistic concepts in terms of its own nonlinguistic experience. It is the first AI system able to create its own concepts that apply across different contexts.

Derived from a sophisticated, complex-systems-based mathematical theory of intelligence, WebMindä's architecture is that of a massively parallel network, a population of many different static and dynamic agents continually recomputing their relationships with other agents, and acting on other agents in accordance with these relationships

WebMind runs efficiently on powerful stand-alone computers, but is most powerful when run over a network of computers, in which case its sophisticated server-server communication methods allow its internal network structure to harmonize with the connectivity structure of the computer network.

WebMind does not aim to emulate human intelligence in an artificial way, but rather to be a natural intelligence for the digital world -- solving human problems with digital intuition.

Initial tests of WebMind's ability in the area of financial decision support have proved dramatically successful. Further application areas under active development include information retrieval, data mining, and enterprise analysis.

Webmind is a new kind of software system, but it incorporates elements or functions of software technologies typically used for business intelligence. It can serve as a search engine, as a Web visualization tool, or as a data mining engine -- and it can also serve functions not implied by any of these pre-existing categories. Most fundamentally, it is a system for the structuring and dynamic evolution of large bodies of data. It is an intelligent system for knowledge management and knowledge creation, with an almost boundless array of potential applications in business, science and personal computing.

Webmind collects and analyzes information in much the same way as the human mind does, but with different strengths and weaknesses, befitting its physical basis in silicon rather than carbon, and its perceptual embedding in the world of data and text feeds rather than the world of sights, smells, sounds, tastes and touches. In the human mind, everything above the lowest levels of perception and action is dynamic and constantly evolving. The memory constantly studies itself, reorganizing to enhance and inhibit recall and performance. Current information discovery tools impose an artificial distinction between static data repositories and dynamic search and analysis tools. But what is needed, in order to really make use of the vast amount of information we are amassing, is a dynamic information system that mimics, on an abstract level, the structure of the human mind, with all its reflexive, self-analytical capabilities. What is needed is a way to make the data come alive and study itself, rather than lying there inert and being probed by various computational instruments, of various degrees of intelligence.

This is what Webmind accomplishes. It allows collections of data to structure themselves and recognize patterns in themselves. Webmind transforms each informational element within a computer network into an independent agent which cooperates with other agents in organizing and reorganizing data for search and retrieval. It interacts with and builds on the intelligence of the user. Users' queries are treated as informational agents interacting via the collective Webmind agents system to locate information. Webmind agents take advantage of the sum collection of their organization's intelligence to discover context and concepts on behalf of users and their applications.

How does WebMind relate to the architectures and algorithms traditionally used in the AI field? The short answer is: it is completely different from all previous approaches, in that it is the first unified computational mind, complete with integrated systems for perception, action, cognition, language, short and long term memory, etc., all embedded in a data-rich environment.

The discipline of Artificial Intelligence is an extremely ambitious one, defined in the Artificial Intelligence Dictionary as "a multi-disciplinary field encompassing computer science, neuroscience, philosophy, psychology, robotics, and linguistics; and devoted to the reproduction of the methods or results of human reasoning and brain activity." The underlying long-range goal of AI is nothing less than the construction of thinking machines, machines that think as well as human beings, or better. In spite of these lofty ambitions, however, the reality of AI has up to this point been severely disappointing.

AI has not proved useless, but it has adopted a role as one technology among many, to be applied to tasks that

• Do not require a general, intuitive understanding of the world

• Are too non-routine for ordinary software methodologies

• Are too repetitive or too mathematically involved for humans to want to do, or to do well

As such, it has proved particularly useful in the finance industry, and in other specialized areas such as process control.

This is where WebMindä comes in. WebMindä utilizes an original complex-systems model of intelligence, combined with the power of multithreaded, agents-based Java network computing, to overcome these historical limitations of AI technology. Whereas previous AI technologies focus on specific mental qualities such as knowledge representation, perception, reasoning, or learning, WebMindä is based on a comprehensive mathematical model of the mind as a whole. It perceives information, analyzes information, remembers information and acts on information in an integrated way. It adapts to its own environment much as human beings adapt to the physical world. It evolves its own spontaneous structures, rather than relying too heavily on the intuitions of human programmers.

WebMindä does not have a general intuitive understanding of our world, but it does have a general intuitive understanding of its own world, the data stored on the Internet, and the interests and responses of human Internet users. This is the qualitative difference between WebMindä and previous AI systems; and this is why WebMindä has the potential to bring AI beyond the role of a specialized adjunct to unintelligent computer systems, and to make AI the heart of computing itself as we move on into the new era of super-powerful digital networks.

At the present time, the domain in which Webmind has been most systematically and scientifically tested is financial analytics. The goal in these tests was simple, to be able to make better financial predictions and trading decisions than one could without the software. The finance industry has already developed sophisticated tools for projecting trends in markets, using quantitative data about what the markets have done in the past. The problem with these techniques is that they do not encompass the full realm of information available to the markets.

The key advantage of Webmind for this purpose is that it can incorporate a logical analysis of textual material, material from financial news sources. This, of course, is very similar to what market analysts do. They study the statistical data, then interpret it in view of information which they obtain from their reading and conversations with industry leaders. However, there is a tremendous mass of written material available, and not enough time for any analyst to digest it all. Traders for large investment firms have difficulty even finding the time to read the reports generated by their own staff analysts. Webmind Analytics was designed to computerize this process.

The central purpose of Webmind Analytics is to make trading decisions. Webmind begins by incorporating the standard indicators and predictors which are already used in market analysis. An indicator provides a piece of information about the state of play at the time computed, such as the value of the market series at that time, the 5-session variability at that time, the Relative Strength Index of another data series at that time, or the incidence of meaningful text in internet news services at that time. We provide a list of standard and novel indicators which can be selected by the user from readily available data sources. A predictor takes in the past history of a market series, and perhaps other past information, and predicts the next value of the market series. We use standard statistical techniques such as linear and nonlinear regression, pattern searching and weighted historical combinations of indicators. All this is standard, state-of-the-art market analytics. What is new about Webmind is that textual indicators are analyzed and used together with the numerical indicators to make decisions about trading rules.

Initial tests on daily financial data proved dramatically successful. The following table shows our first results, indicating the percentage of correct prediction of market direction with and without the text-based indicators:

Dow Jones 63.5% 90.0%

Eurodollar futures 53.5% 81.3%

Yen futures 59.0% 83.3%

US T-bond futures 53.7% 68.2%

Cocoa futures 60.9% 82.5%

The correct predictions described in this table resulted in a high rate of return for Webmind's simulated trading system, as shown by the following annual profitability results on simulated trades over a year, based on a fixed drawdown and no reinvestment of profits:

Dow Jones 63.8% 172.3%

Eurodollar futures 22.9% 134.4%

Yen futures 35.4% 118.3%

US T-bond futures 7.8% 95.8%

Cocoa futures 19.9% 30.6%

One of the most exciting aspects of Webmind architecture is the way it leverages the intelligence intrinsic in the structure of modern computer networks. Its internal structure is that of a network of interconnected, meaning-sharing, learning agents; and using the power of Java for network programming, this internal network structure is made to harmonize with the network structure of modern intranets and the Internet.

Network-centric computing is commonly portrayed as a third phase in modern computing: first mainframes and terminals, then PC's and local-area networks ... and now large-scale, integrated network computing environments, providing the benefits of both mainframes and PC's and new benefits besides. In fact, though, this view underestimates the radical nature of the network-computing paradigm. What network computing truly represents is a return to the cybernetic, self-organization-oriented origins of computer science. It goes a long way toward correcting the fundamental error committed in the 1940's and 1950's, when the world decided to go with a serial, von-Neumann style computer architecture, to the almost total exclusion of more parallel, distributed, brain-like architectures.

The move to network computing is not merely a matter of evolving engineering solutions; it is also a matter of changing visions of computational intelligence. Mainframes and PC's mesh naturally with the symbolic, logic-based approach to intelligence; network computing environments, on the other hand, mesh with a view of the mind as a network of intercommunicating, intercreating processes. The important point is that the latter view of intelligence is the correct one. From computing frameworks supporting simplistic and fundamentally inadequate models of intelligence, one is suddenly moving to a computing framework supporting the real structures and dynamics of mind.

For mind and brain are fundamentally network-based. The mind, viewed system-theoretically, is far more like a network computing system than like a mainframe-based or PC-based system. It is not based on a central system that services dumb peripheral client systems, nor it is based on a huge host of small, independent, barely communicating systems. Instead it is a large, heterogeneous collection of systems, some of which service smart peripheral systems, all of which are intensely involved in inter-communication. In short, by moving to a network-computing framework, we are automatically supplying our computer systems with many elements of the structure and dynamics of mind.

Viewing computer networks in this way, one sees that network computing opens up fantastic new possibilities for artificial intelligence. Once one steps beyond the single-machine, single-program paradigm, and views the whole Net as a network of applets, able to be interconnected in various ways, it becomes clear that, in fact, every corporate Intranet -- and the Internet itself -- is an outstanding, terribly underutilized supercomputer. Each computer in a corporate intranet is potentially a "neuronal module" in

a company-wide electronic brain. Each computer hooked up to the Internet is potentially a part of the global brain; and each corporate intranet is potentially a lobe of a global business intelligence system.

networked, equipped with Java code or something similar, is potentially a "neuron" in a worldwide brain. The brain metaphor need not be taken too literally -- the point is that network computing, in principle, opens up the possibility for the Net to act as a dynamic, distributed cognitive system.

WebMindä provides a practical software framework within which this vision of "the network is the computer is the mind" is made practical and useful, in a business context. The machines interconnected on an Intranet are leveraged together into a single thinking WebMindä unit, controlled by a "cortex" of one or more dedicated WebMindä machines. And machines interconnected by slower Internet connections are able to interact much like different human beings, exchanging questions and answers and chunks of relevant knowledge. Because WebMindä 's internal structure is that of a self-organizing network, the network structure of intranets and the internet is leveraged to the maximum degree.

2. Conceptual Foundations of Webmind

The WebMindä software architecture is founded on a decade of research in theoretical, cognitive science by IntelliGenesis founder Dr. Ben Goertzel published in four technical books and a series of research articles. The culmination of this research is a mathematical model of mind called the "Psynet model," which draws on the insights of cybernetic and complex systems science, and represents mind as a collection of emergent structures arising from a self-organizing physical system. WebMindä is an implementation of the Psynet model of mind, which is efficient in terms of contemporary computer architectures and effective in terms of contemporary business computing goals.

Underlying the Psynet model is a vision of intelligence as "the capability to achieve complex goals in complex environments." WebMindä's complex goals are firstly posed to it by human users and secondarily evolved by itself; its complex environment consists of the human beings, machines and networks with which it interacts.

According to the Psynet model, the achievement of intelligence does not depend on the implementation of particular mechanisms, but rather on the implementation of mechanisms that give rise to appropriate emergent structures. The unique achievement of the WebMindä design is to cause these emergent structures to emerge from current computers and networks in a computationally efficient and humanly useful way.

Although WebMindä has some neural-nettish features, it is not a "neural network" model in the traditional sense. The constraint of reasonable efficiency on networks of serial processors requires numerous differences from the design of the human brain.

The structures of mind which are identified by the Psynet model as being most essential to intelligent functioning are:

• A topological network of mental entities, each one able to interact with other "nearby" mental entities in a variety of ways. In WebMindä, this topological network is called the "Psynet", and the entities within it are called "nodes"

• A way for each node in the Psynet to know which other nodes it is associated with (related to)

• A way for nodes in the Psynet to control other mental entities in an asymmetric way (giving rise to hierarchy)

• An effective superposition of the associative and hierarchical sub-networks, so that operations in each one are non-destructive with regard to the other (the "dual network")

• A "lineal awareness system" whereby WebMindä takes in a series of data from the environment and encodes them into the Psynet in a way that depends on both the preceding data in the input datastream and the context observed in the Psynet

• A "self system" by which WebMindä observes its own operations and continually queries itself, and modifies its concepts accordingly

• A "social system" by which WebMindä observes the operations of other WebMindäs, and of human users, and represents these entities internally

These structures are coded into WebMindä, but in a highly flexible way, so that each structure may co-adapt with the others, the details of each structure thus ultimately being emergent from the details of all the other structures.

A detailed description of the psynet model cannot be given here, but a capsule summary is as follows:

• A mind is a system of agents which are able to transform, create & destroy other agents

• Many of these agents act by recognizing patterns in the world, or in other agents

• Thoughts, feelings and other mental entities are self-reinforcing, self-producing systems of agents

• These self-producing mental subsystems build up into a complex network of attractors, meta-attractors, etc.

• This network of subsystems & associated attractors is "dual network" in structure, i.e. it is structured according to at least two principles: associativity (similarity) and hierarchy (categorization and category-based control).

• agents pass attention ("active force") to other agents to which they are related

• Because of finite memory capacity, mind must contain agents able to deal with "ungrounded" patterns, i.e. agents which were formed from now-forgotten agents, or which were learned from other minds rather than at first hand -- this is called "reasoning"

• A mind possesses agents whose goal is to recognize the mind as a whole as a pattern -- these are "self"

The Webmind AI system provides a general "agents operating system" for managing systems of software agents that share meaning amongst each other, transform each other, and interact in various ways; the agents may live in the RAM of a single machine, may be run on multiple processors, and may live across many machines connected by high-bandwidth cable. It provides mechanisms for agents to represent patterns they have recognized in other agents, and patterns they have recognized in the overall agents system of Webmind. And it also provides an appropriate assemblage of agents, specialized for such things as

• text processing

• numerical data processing

• text-numerical interrelation

• supervised and unsupervised concept formation

• reasoning

• social interaction

• language processing

These agents have their own localized behaviors but achieve their true intelligence only in the context of the whole Webmind system.

The psynet model of mind is too abstract to lead directly to any particular software program or engineering hardware design.. It can be implemented in many different ways. But, even without further specialization, it does say a great deal about AI. It dictates, for example,

• that an AI system must be a dynamical system, consisting of entities (agents) which are able to act on each other (transform each other) in a variety of ways, and some of which are able to evaluate simplicity (and hence recognize pattern).

• that this dynamical system must be sufficiently flexible to enable the crystallization of a dual network structure, with emergent, synergetic hierarchical and heterarchical subnets

• that this dynamical system must contain a mechanism for the spreading of attention in directions of shared meaning

• that this dynamical system must have access to a rich stream of perceptual data, so as to be able to build up a decent-sized pool of grounded patterns, leading ultimately to the recognition of the self

• that this dynamical system must contain entities that can reason (transfer information from grounded to ungrounded patterns)

• that this dynamical system must be contain entities that can manipulate categories (hierarchical subnets) and transformations involving categories in a sophisticated way, so as to enable syntax and semantics

• that this dynamical system must recognize symmetric, asymmetric and emergent meaning sharing, and build meanings using temporal and spatial relatedness, as well as relatedness of internal structure, and relatedness in the context of the system as a whole

• that this dynamical system must have a specific mechanism for paying extra attention to recently perceived data ("short-term memory")

• that this dynamical system must be embedded in a community of similar dynamical systems, so as to be able to properly understand itself

These criteria, while simple, are not met by any previously designed AI system.

Several questions arise in designing an AI implementation of the psynet model:

• 1. How to represent the agents

• 2. How to manage the dynamics of a collection of agents, on the hardware provided

• 3. What to build in, and what to require the system to learn

• 4. How to embed the system in a rich environment, including other systems of like kind

• 5. What goals to give the system

• 6. What programming language to use?

How one answers these questions determines what kind of system one comes up with. In this section we will describe how these questions were answered in the design of Webmind, on a general level.

In Webmind agents are realized as data structures called "nodes," with particular structural and dynamical properties. A psynet node is defined as a container for one or more of the following:

• possibly some "raw" perceptual data

• a record of the meaning of the current node, in the context of the mind as a whole

• a transformation which acts on a node, and produces a node

• a transformation which acts on a node, and produces a number

• tables of links to nodes with which it shares meaning in certain ways

• mobile agents which seek other nodes to add to the link tables

• information on how much attention is paid to the node currently and over the recent past, in total and from the perspective of various other nodes

• information about how important the node is, and how fast its importance is expected to decay over time

It should be clear that the node data structure, defined in this way, is adequate for expressing the "agent" concept of the psynet model of mind. Transformations taking agents into agents and agents into numbers are here, as required; and the dynamical law of mind, which says that attention follows meaning, is explicitly supported via each node carrying lists of other nodes that share its meaning, and agents searching for nodes that share its meaning.

The view of the mind as a network of psynet nodes is a kind of "middle way" for AI -- or in other words, an "intermediate level" model of intelligence. It stands between, on the one hand, neural network and other physiologically inspired models of intelligence; and on the other hand, abstract logical and psychological models of intelligence that speak of mind in the abstract, completely apart from its physical implementation. Neural network and logical and psychological models all have their particular validity, but considered as a collective, they leave a crucial gap: how do the complex, self-organizing physical dynamics of neural networks and the like give rise to the abstractions of the mind, such as syntax, logic, inference and self? "Psynet" means "mind network," and so it is only appropriate that the network interpretation of the psynet model fills in this gap by positing a network that is not a network of neurons but a network of intermediate-scale, mental entities ... "agent" nodes.

In a neurophysiological context, it may be useful to think of a psynet node as a neuronal group in the sense of Edelman's Neural Darwinism brain theory: each of Edelman's neuronal groups carries out a specific useful function, quite different from a single neuron, which in most cases has no particular function in itself. In some computer implementations of the psynet model, it may be useful to take this mapping fairly literally, and implement nodes in terms of networks of formal neurons -- as we shall see, something like this is done in Webmind, in certain cases, though not in general. But this is not a necessary approach: psynet nodes could be implemented as LISP program trees, as high-degree polynomials, as physical entities outside the realm of digital computing (Bose-Einstein condensates?), etc. The psynet model is abstract: it has to do with the structure and dynamics of the network of agents, not with the internal makeup of the agents themselves. Mind design, in this perspective, is largely about creating the correct agents out of the tools at hand, to ensure the emergence of the desired structures and dynamics from the system as a whole.

In "agent system" terms, we may say that agents in WebMindä fall into several classes. First there are nodes, static agents, which have a relatively persistent state. Nodes may contain information relating to data such as texts or database files or internal information such as categories or other patterns recognized among other nodes. Nodes are not truly static, they update themselves continually, but they represent the backbone of WebMindä's intelligent internal network. Most of the information contained in a node actually consists of a record of the node's relationship to other agents in the Psynet. There are many different types of node, pertaining to different types of information.

In addition to nodes, there are relational agents or links, which establish and retain relationships among nodes. These relationships may fall into many different categories, and may be parameterized by qualities such as intensity, uncertainty, and so forth. Finally, relational agents are formed by the third kind of agent, mobile agents which move around within the internal relational geometry of an individual Psynet, potentially from one WebMindä server to another, actively seeking new relationships to embody in relational agents. Mobile agents associated with nodes can also move from one WebMindä to another, across the Net, but this requires a different brokering mechanism than the free flow of agents within the same WebMindä server, or between different WebMindä servers on the same Intranet.

The dynamics of the network of nodes in Webmind's psynet is as follows. When a node receives attention, it spreads some of its attention to the meaning-sharing nodes that it knows about -- i.e. attention follows the links in its link table. It may also update its record of its own meaning. Finally, it may carry out its transformations to produce new nodes, or it may modify its transformations, with the goal of making them more prominent patterns in the collection of other nodes with which it shares meaning.

Ideally, each node would run independently on its own processor. However, that is not feasible at the present time: massively parallel processors do exist, but are orders of magnitude less efficient than networked top-of-the-line serial computers. Fortunately we are presently blessed with computers that, though serial in nature, possess multiple rapid processors, large amounts of RAM (in the gigabyte and up range), and fast cables (e.g. Ethernet) networking them together. This combination allows us to run large magician systems on serial machines.

The system architecture of Webmind is roughly as follows. A single mind network, may run across a number of machines, interconnected by high-bandwidth connections. Nodes may send messages across machines, and may share meaning with nodes on other machines; they may also move from one machine to another if this is judged dynamically efficient. Each machine contains a managerial process which determines when dispatching of nodes and messages between itself and other machines is appropriate. This managerial process must pay attention to the psynet dynamics on its own machine, for various reasons.

Within a single machine, nodes are divided into groups called pods. Grouping into pods is useful for human saving/loading of groups of nodes -- a pod of nodes may serve a particular human purpose, e.g. it may contain nodes concerning personnel records, nodes concerning financial data, etc. It is also useful for multiprocessing: a pod must live on a single processor, but multiple pods living on multiple processors may be very elegantly and efficiently handled. When a node wants to transform or spread attention to another node, living in a different pod, it sends a request to its pod's managerial process, which then sends the request on to the managerial process of the pod at the receiving end.

During the early development of Webmind, this was the toughest question to answer. One could seek to implement the psynet model of mind by setting up a kind of "primordial soup" of transformations, mapping into numbers and mapping into other transformations. Assuming this soup was hooked up to rich data feeds, given extra resources in return for success at practical tasks, and given a reasonable capacity to mutate itself, it would eventually transform itself into something intelligent, arriving at the structures described above by self-organizing evolution. This methodology, however, is extremely inefficient, and is extremely unlikely to work given the limitations of current engineering technology (it was tried in 1994 and failed miserably!). It mixes up what in biology are three different levels of intelligence: evolution, epigenesis, and learning. In order to arrive at a practical implementation of the psynet model, one must build in more of the desired structure -- just as a human being is born with some of the desired structure of mind already present in the structure of the brain.

The question then becomes how much one can build in without sacrificing the dynamicity, creativity, and self-modification that are at the essence of intelligence. The correct answer, we have found, is to engineer:

• the processing of low-level perception and action

• the types of agents potentially in the system (which determine not which mind you have, but what kind of mind you have)

• the specific configuration of agents, but only as a guide for future learning and not as a fixed, inorganic base of knowledge

The built-in preprocessing of low-level perception and action is necessary because of resource constraints. All that the human brain does by hard-wired visual circuitry -- line extraction, etc. -- could be done by more intelligent parts of the brain, but this would be a waste. The amount of pattern recognized in the world is maximized by allowing pre-processing to take care of the lowest level of pattern-recognition, and focusing the attention of intelligent on the output of the preprocessors. In the current Webmind system this lesson is reflected as follows: some basic text processing routines are built in, as are some numerical data processing routines that recognize dates in numerical data files, etc.

In the current system, a carefully chosen variety of node types is provided also: nodes for text, nodes for numerical data, nodes for words/phrases, nodes for senses of words, nodes for categories, etc. Each node type has specific means for determining shared meanings and for transforming itself and other nodes. The strategy is to build in the node types, and then allow the nodes, of selected types, to self-organize into their own structure based on their inputs. The key principle to remember here is the gradation of specialization. A system consisting of only highly specialized node types will never have the flexibility to be intelligent in the real world; but a system consisting of only highly general node types (not evolved or engineered for any particular world) will have a hard time coming to grips with its environment. Specialized node types are a step up from perceptual/active preprocessing, but serve a similar role: to enable the system to focus most of its attention on problems which require general intelligence.

In the current Webmind system, we have highly specialized node types for two things: language processing, and analysis of numerical data, particularly financial data (e.g. a node type specifically representing a trading system). Preprocessed data from these sensory modalities feeds into specialized nodes at the same time as it feeds into general-purpose nodes, and the interaction between the specialized and general-purpose nodes at this stage of processing is intense.

The interplay of different node types is is in some sense the hardest part of computational mind engineering. The essence of mind, as depicted by the psynet model, is relatively simple: what distinguishes one type of mind from another is mainly the types of agents that live inside it. The interaction between specialized agents and general-purpose agents is something that, in humans, evolved over millions of years; it can be engineered, but only with a firm understanding of the general underlying dynamics of mind, and a healthy respect for the difficulty of the task.

Humans receive a vast amount of perceptual data via their sense organs; and we interact with the same world producing this data via speaking, moving, stinking, etc. In short, we are embedded in our bodies. The body of Webmind consists of the computers on which it is implemented, which have their own sensory and active organs, for reading information from other computers and from human users. Pre-Internet, the data feeds available to computers were fairly sparse, and the ability for a computer program to interact with humans and other programs was strictly limited. At the present time, however, it is possible to hook up Webmind to a vast number of textual and numerical datafeeds, to supply with an unlimited amount of user interaction via corporate intranets and the Web, and to allow it to act by sending messages to individuals and other computers. For the first time, there is now a rich perceptual environment available to a computational intelligence. Furthermore, there is the opportunity to place a community of AI systems within this common environment, giving them access to common perceptual data and the ability to interact with one another in a variety of ways.

This is a somewhat obvious point, but it is one that has not been exploited by the mainstream of AI researchers, who do not accept the essentiality of a rich environment for the emergence of intelligence.

The social interaction of Webminds is explicitly mediated by data structures corresponding to the Self (the Webmind itself), other Webminds (needed for intelligent inter-Webmind querying), and human users (needed for intelligent customized response).

In order for a mind network to evolve effectively, it must be given goals to carry out, and must be given feedback on the quality with which it has carried out these goals. Just as humans are built to want to eat, have sex, drink, exercise, etc., so Webmind is built to want to answer queries effectively. In order to assure this, each time it sends a query, it should receive an assessment of the perceived quality of its response. This may be explicit (e.g., the user may be asked), or it may be implicit (e.g., the client may calculate the number of times the human user needed to rephrase his question to get the answer desired). We all know how hard it is to guess what humans want -- what this means is that answering human questions is certainly a "complex goal" as required in our working definition of intelligence, which is "the ability to achieve complex goals in complex environments."

WebMindä has been designed using rigorous object-oriented methodology, and is being programmed in Java. The value of Java for WebMindä is multifold:

• Automatic garbage collection means that programming and debugging of complex objects is much more rapid than in C++

• Sophisticated networking libraries (Remote Method Invocation; ObjectSpace Voyager) allow flexible network communications to be programmed relatively easily

• Platform independence, for the client and the server!

• Effective and easy to use multi-threading

With the advent of ultra-efficient compilers such as IBM's Formula, the speed gap between C++ and Java is narrowing; our own tests show a ratio of 2.5:1. The WebMindä server will also be able to exploit the added power of the Java chip, to be introduced in early 1999. Finally, the much-publicized "Java browser wars" have no significant effect on WebMindä, as the WebMindä server is not browser-based, and the WebMindä client can be run either as an applet in a Java-1.2 compatible browser, or as a freestanding application.

The details of the WebMindä architecture are proprietary and cannot at this stage be discussed in detail in a public document. However, some important aspects of the design may be discussed at a high level of granularity, and may help the reader to grasp the system's essential nature. This section describes the main functionalities carried out by the standard node types that come with a Webmind system.

The Psynet

A single WebMindä system consists of a collection of nodes called a Psynet, which spans a collection of one or more WebMindä servers, each one running on its own (possibly multiprocessor) machine. The intelligent core of a WebMindä server is its collection of nodes, which is called a PsynetLobe. Each PsynetLobe, in turn, consists of multiple pods, each pod containing a number of nodes.

At the present time, the boundaries of a single Psynet are the boundaries of the intranet or extranet: Ethernet connections are fast enough that WebMindä servers thus connected can act as an emergent whole; whereas Internet connections are too slow. Thus, one has a model of the intranet as the WebMindä individual; and the Internet as a population of WebMindäs interacting in a manner best described as "social", exchanging information and questions and answers but not participating in collective trains of thought on a real-time basis. This of course may change as bandwidth increases over the next decade; at some point the Internet will become as fast as today's Ethernet, and a single global Psynet will become achievable.

Every WebMindä server comes with a standard collection of pods: pods for standard data types such as text and numerical data, several pods for natural language understanding, a pod for categorization, a PsyContext pod for real-time information processing, a pod for query processing, and other pods devoted to such tasks as creative brainstorming, self-reflection, socialization, etc. Other pods will be application-specific: e.g., a WebMindä server aimed at financial applications would contain a pod of historical financial data; an enterprise application would contain a pod of transient e-mail data; etc. Additional pods may be loaded and saved by the user.

The dynamics of the Psynet is fairly simple, though the emergent structures that these dynamics give rise to may be quite complex. Each pod carries its own threads, and these threads cycle through all the agents in the pod, allowing them to carry out "elemental actions." Each node carries a "heat" indicating its importance at a particular time, and hot nodes are given a greater priority.

There is one Pod, the PsyContext pod that bears a special relation to the outside world. This pod is WebMindä's short-term memory, containing information regarding what WebMindä has recently experienced, and other things judged to be associated to these recent experiences. WebMindä's "lineal awareness" system consists of a loop that takes in new data, processes it in conjunction with the agents in the PsyContext, and then places it in the PsyContext, removing some obsolete information.

Specialized pods for dealing with self, social interaction and creativity are dealt with in a fairly similar way to the PsyContext; they contain agents which, rather than being relevant to recent experience, are relevant to particular areas of moderately-recent experience, e.g. experience with oneself, experience with other intelligent entities, and experience with difficult problems.

The starting-point for Webmind's intelligence is a collection of basic node types used to represent data found on current computer networks. There are TextNodes for representing text documents, DataNodes for representing numerical data, and several types of CategoryNodes for representing concepts inferred from text and data. There are also TimeSets, representing collections of data that occur over time; and TimeNodes, representing collections of information considered roughly simultaneous. These node types are the basis for Webmind's intelligence; other processes work with these nodes to generate meaningful information of various forms. Information mined from data warehouses is read into the system and translated into a collection of TextNodes, DataNodes, CategoryNodes, TimeSets and TimeNodes; a flexible format which is amenable to sophisticated intelligent analysis by Webmind's various internal agents.

WebMindä does not have a separate "natural language system" as such. Language processing is carried out by the Lineal Awareness system, and nodes concerned with categorization, and mobile agents concerned with recognizing similarity and other relationsips, in a way that is thoroughly integrated with the Psynet. To disentangle language from thought is to remove most of what is interesting and important about language.

The first part of WebMindä's language understanding is linguistic categorization, which is carried out by category nodes according to the normal dynamics of the Psynet. WebMindä's self-organizing categorization is capable of determining parts of speech for words, and of determining semantic categories such as "house", "action", etc. as well.

Once straightforward categorization has been used to create semantic and syntactic categories, attractor neural network methods are used to refine semantic links between words, based on contextual information. Words are then joined by relational agents if they are related closely in the bodies of information that WebMindä has been exposed to.

Parsing is carried out by a proprietary, original self-organizing parsing method based on L-system grammar. This system takes a sentence, uses the Psynet to determine syntactic and semantic categories for the words in the sentence, and then maps the sentence into a collection of nodes and relational agents, which are placed into the PsyContext and then into the Psynet. If the sentence represented a query, then the collection of nodes and agents corresponding to the sentence is activated rather than merely stored, and a query agent is created which collects the results of this activation after a period of time.

The genetic algorithm (GA) is used in several places within WebMindä, for the purpose of creating new agents satisfying dynamically-arising requirements. For this purpose, a highly generic approach to GA evolution of Java objects has been created. Any node that is to be evolved must implement the interface EvolvableNode, which contains methods such as: mutate(), which mutates the object; cross(), which crosses the object with another EvolvableNode object; fitness(), which reports how fit the object is in its current situation.

The relation between the genetic algorithm and the brain is somewhat uncertain. Gerald Edelman's "Neural Darwinism" theory argues that the brain can be viewed as a genetic algorithm with mutation only; however, a case can also be made for crossover between "neuronal maps." At very least, we may view the use of the GA in WebMindä as a more efficient way of accomplishing, using crossover and mutation, what the brain may accomplish with mutation only.

Mobile agents of various types create associative and hierarchical relationships, thus instantiating the elements of a cognitively powerful "dual network" structure. But what guarantees that these two structures, heterarchy and hierarchy, will work effectively together? There are several answers to this question within the Webmind architecture, but one of the keys is the explicit search for meaning. Each portion of the net, searching for its own meaning, re-expresses itself in terms of the hierarchical and heterarchical connections surrounding it, thus causing the hierarchical and heterarchical connections in its area to function effectively together.

In order for an intelligence to understand the meaning of a word, concept or other entity, this entity must be "grounded" in the system's own experience. Consider, for example, the position of WebMindä on encountering the word "two." In order to deal with "two" in an intelligent way, the system must somehow understand that "two" is not only a pattern in texts, it also refers to the two computer users who are currently logged onto it, the two kinds of data file that it reads in, etc. These are all patterns associated with "two" in various situations. Associating all these patterns with the "two" category node is what fully integrates the "two" node with the rest of the network, guaranteeing that the behavior of the "two" node is in harmony with its surroundings.

Through processing of text alone, part of the fuzzy set that is the meaning of "two" can be acquired: that part which is purely linguistic, purely concerned with the appearance of "two" in linguistic combinations with other words. But the other part of the meaning of "two" can only be obtained by seeing "two" used, and using "two," in various situations, and observing the patterns in these situations that are correlated with correct use of "two." In general, the fuzzy set of patterns that is the meaning of a symbol X involves not only patterns emergent among X and other symbols, but also patterns emergent among X and entities involved in various situations. Recognition of patterns of this type is called symbol grounding.

Symbol grounding is connected in a fascinating way with introspection. In WebMindä, each agent has the ability to look inside itself and report various aspects of its current state. Symbol grounding is achieved by special mobile agents attached to individual nodes representing concepts (e.g. the node for two"). These agents are evolved by the genetic algorithm; they survey the introspections of other nodes and combine the results of these introspections to form "groundings" of the node to which they are attached.

Reasoning, in the WebMindä context, is defined as a process that takes in a collection of relationships between nodes, and outputs a new relationship between nodes. Webmind contains node-resident and mobile agents whose specific purpose is to carry out this dynamic. The specific behavior of these agents is in part borrowed from the NARS (Non-Axiomatic Reasoning System) AI system developed by IntelliGenesis engineer Pei Wang during his academic career. Dr. Wang, in his previous research work, has shown mathematically that reasoning agents such as those used in Webmind are capable of carrying out deductive, inductive and abductive (hypothetical reasoning) in accordance with the requirements of mathematical logic. Unlike in typical AI reasoning systems, however, here reasoning is carried out via self-organizing processes that are integrated with the remainder of Webmind's intelligence.

The importance of reasoning can be well understood in the context of symbol grounding: reasoning allows Webmind to extend conclusions from grounded domains to ungrounded domains. For example, if Webmind has learned that "big" means a certain thing in the context of financial data (where it can ground the symbol "big" in reference to its direct experience), then it can use reasoning to transfer the relations it has derived about "bigness" to domains where it has no direct experience, e.g. to understand a phrase like "a big upheaval in the Japanese government."

Query Processing

WebMindä's query processing is integrated with the rest of its mental activity, just as for a human being, question-answering is not so different from purely internal thought processing.

When a query (be it a series of key words, a paragraph of natural language, or a series of commands requesting particular data operations) is entered into the system, a node is created for it, the query node sends out mobile agents, and these agents create new relational agents joining it and other nodes. Activity related to the query node spreads through the Psynet, and after a certain period of time, the nodes with the highest activity relevant to this particular activation process are collected, and returned as the answers to the query.

The distinction between activity due to a particular query and activity due to general Psynet thought processes or other queries is carried out via an innovative, proprietary technique of "parallel thought processes," which allows WebMindä to do one thing the human mind cannot: carry out hundreds or thousands of simultaneous trains of thought, and keep them all straight!

An individual WebMindä server running in a single Java process is, potentially, an autonomous WebMindä. On a multiprocessor machine with a large amount of RAM, one can achieve quite a powerful WebMindä in this way. Given the current state of computer hardware, however, the most cost-effective way to achieve a very smart WebMindä will be to network a number of WebMindä servers together into a single multi-machine Psynet.

To understand the implications of these hardware restrictions, which push toward a network implementation, one must consider them in a broader context. They overlap with fundamental system-theoretic requirements in an interesting way. Hardware leads us to distinguish three levels of WebMindä system:

• Single machine

• Network of machines connected by fast cable (intranet)

• Network of machines connected by slow connections (internet)

These levels may shift over time, as the Internet becomes faster and faster. On the other hand, analogy with biological intelligence leads us to distinguish two levels of systematic intelligence:

• individual intelligence

• society of individual intelligences

Intra-mental interactions may encompass social interactions, but not vice-versa: parts of the same mind have access to each other in ways that separate minds do not.

We use the term an "individual WebMindä" or simply "a WebMindä" to refer to an individual WebMindä intelligence; and the term "global WebMindä" or "WebMindä society" to refer to a collection of WebMindä intelligences. At the present time, the most natural mapping from social/individual concepts to computer hardware is

• machine = part of individual WebMindä

• intranet = individual WebMindä

• internet = WebMindä society

However, this mapping may not remain ideal forever.

Social interactions between WebMindäs are as follows:

• One WebMindä server can query another server, just as if it were a client.

• One WebMindä server can send a agent to visit another server, to gather information (create relational agents involving agents on the other server).

Determining which server is best suited to receive a query or a visiting node to is done using a fairly sophisticated "socialization" system

Intra-mental interactions encompass social interactions but add an additional components. For example, nodes within a single WebMindä (currently, a single intranet) will reach out to one another when activated and reinforce the relational agents intertwining them; nodes separated by a slower Internet connection will not.

Having reviewed the general nature of the Webmind architecture, we can now outline its application to various particular problem areas to a moderate level of detail. In this section we will consider several broad application domains: financial decision support, text information retrieval, data mining and enterprise analysis.

The application of Webmind to text-based market prediction described above requires a number of different node types. All of these node types have broad uses outside the domain of text-based market prediction, but are combined in this application in a special way. DataNodes were used for numerical data sets, such as financial time series. TextNodes are used for text documents, such as a market news report for a particular days. TimeSets are created to hold series of time-indexed nodes (TimeNodes), such as a series of market news reports over several years. ConceptNodes are created to hold concepts, which are either extracted from the text or learned by Webmind itself. TradingRuleNodes incorporated different financial trading rules which are routinely used by traders. TradingSystemNodes incorporated trading systems, which are collections of trading rules.

Suppose, for example, that the market being traded is the Dow Jones Industrial Average, and the text being used is market news which is issued on a daily basis. In this example, the data structures are a DataNode pointing to the DJIA, and a TimeSet pointing to the series of market news articles. The text indicator learning process is carried out by an agent associated with these two notes. The goal of this agent is to isolate a ConceptNode, representing one of Webmind's internal ideas, with the property that the relevance of the ConceptNode to the TimeNode, on a given day, is a useful indicator for trading the data in the DJIA DataNode on that day. Having carried out this learning process, it then uses the ConceptNode it has isolated to create a new "text indicator" DataNode, each entry of which indicates the relevance of the ConceptNode to the market news DynamicCategoryNode on a certain day. This text indicator DataNode can then by supplied to the user as, quite simply, a series of numbers. Each day, on reading the news, Webmind can supply a new value for the text indicator – just as, on seeing the DJIA itself, it can supply new values for numerical indicators such as moving averages and the like.

The text indicator obtained by the process of internal conceptualization can be used for a variety of different purposes. It can be used by humans to form their own intuitive judgments, or incorporated in any computational trading framework that is sufficiently flexible to incorporate arbitrary numerical indicator. Most of our work in this area, however, has involved incorporating text indicators into Webmind's own intelligent trading processes, which are designed to incorporate text indicator information. A TradingRuleNode, in Webmind, encapsulates a trading rule, which refers to a numerical predictor and a number of trading indicators, some numerical and some textual. A TradingSystemNode encapsulates a number of trading rules, one or more long rule and one or more short rule, which have been found to work well together. A genetic algorithm based learning technique, native to Webmind, is used to derive a collection of TradingRuleNodes which are near-optimal for a given DataNode, representing a certain financial instrument.

Another application area for Webmind has been domain engineered to some extent is information retrieval from document databases. It is important in this context to distinguish intelligent information retrieval from "search engine" software. The notion of searching for documents based on topic descriptions is highly limiting. What is really interesting is having human minds relate with textual and other digital information in creative and productive ways. A final implementation of Webmind for information retrieval will incorporate search engine functionality, but will also involve visual exploration of information, and interactive processing whereby the software asks the user questions to clarify their intentions.

Here we will not go into these user interface matters in detail, but will rather give a cursory description of the internals of Webmind as applied to text information retrieval. This will serve to indicate how drastically different it is from its current competitors, and to illustrate the algorithmic roots of the different behavior that it displays. Each document being indexed is described by a TextNode, which contains links to various nodes representing the meaningful information in the text. There are also nodes corresponding to categories among documents. At the most abstract level, these categories might correspond to general topics such as "Business" or "Computing." At a level closer to the bottom, these might correspond to very particular groupings of pages, reflecting topics such as "artificial intelligence and finance." The process of category formation follows immediately from Webmind's self-organizing dynamics.

Each node on every level, when it is sufficiently stimulated, sends out a flock of intelligent agents, which wander the network of nodes in search of other nodes displaying similar or otherwise appropriately related patterns. When a query (be it a series of key words, a simple question, or a paragraph of natural language) is entered into the system, a node called a QueryNode is created for it. Then, the QueryNode is entered into the network much as if it were a new document to be read, classified and understood. First, using higher-level nodes to guide it, it finds a natural position in the network; then it sends out intelligent agents and these agents create new links between it and other nodes, other documents. Then activation spreads through the network, somewhat in the manner of a neural net, and after a certain period of time, the nodes with the highest activation are detected, and returned as the answers to the query.

What is fundamentally different about Webmind, as opposed to search engines, is that its document database is not a static entity, which sits there passively waiting for queries to set it into motion. Instead, it is constantly refining its structure -- and its structure is, like the structure of the human mind/brain, inseparable from its complex dynamics. The categories that it forms are not defined by mere statistical clustering, but by the outcome of self-organizing, reflexive dynamics. When a query is entered into Webmind, it sets off a chain of reactions that ultimately lead to it finding its natural places in the network, i.e., finding those groups of documents that pertain to the same topics as it does, as judged by the network's own complex dynamics.

In an ordinary search engine, a collection of pages is judged to have the "same topics" as each other if they have the same words in them. In a statistical search engine like Excite or SemioText, a collection of pages is judged to have a common topic if they have a mutually similar distribution of words and phrases. In Webmind, on the other hand, common topicality is not defined entirely in terms of structure but also in terms of process. A group of pages has a common topic if they have been judged as related by the self-organizing dynamics of the network. The distribution of words and phrases does play a role in this dynamics, as does the distribution of human-created links between phrases. But these factors only give the initial conditions for the dynamics. In the end, the distribution of links in Webmind is determined as much by the distribution of links in Webmind as by any specific identifiable factor. A query entered into Webmind is not only a request for information, but a perturbation of the complex dynamics of Webmind's constant study of itself.

It was noted above that Webmind can be used to form a more complete and complex version of the Yahoo category system. However, there are significant differences, which are conceptually relevant. The Yahoo category structure is very deep, for one thing; Webmind's structures tend to be somewhat flatter. Also, the Yahoo structure is strictly hierarchical, whereas, Webmind , like the human mind, is capable of operating with a patchwork of overlapping, complementary categorization schemes. In other words, the hierarchy of nodes can be dissociated, branching off in a number of different directions as it rises. Without Webmind's integrative dynamics, this could lead to confusion, but the dynamics are capable of integrating diverse points of view into a single coherent system -- in fact, this is what they do best. The different views within Webmind will not contradict but rather enhance each other. Ultimately, in a situation of extremely favorable computational resources, one can envision each user creating their own view of the Web, and allowing Webmind to crystallize a universal categorization system reflecting their individual perspective. The synthesis of all these individual views, according to Webmind dynamics, would be a kind of collective mentality -- not an average, but a complex, polyphonic blending of the different subjective world-views of the different members of the human race.

Webmind can use the same text-based interface as existing search engines. However, it also lends itself extremely naturally to visual, dynamic-image-directed search. Instead of returning lists of pages, a Webmind search engine can just as naturally return visual images of those portions of the Webmind network containing relevant pages. Different levels of magnification reveal lower and lower levels of the network, different colors represent complementary categorization schemes, etc. There are endless possibilities.

Next, regarding Webmind and the network on which it runs, it should be noted that, just as the general Webmind architecture is perfectly designed for network implementation, so is the Webmind search engine in particular. Today, it is assumed that a search engine database is a single, monolithic mirror of the Web. Already, however, some firms are breaking with this model; for example, FunnelWeb has constructed a network of search engines, one for each continent. When the Australian FunnelWeb engine wants information about the European FunnelWeb engine, it calls it up and asks it. This kind of networked search, on the level of passing queries from one engine to another, is relatively primitive and unexciting. However, Webmind opens up a whole new spectrum of possibilities.

Imagine, as the simplest case, that one set up two separate Webmind databases corresponding to .com and .edu domain names. Each one would have its own integrity, its own hierarchical/associative structure. But there could be links from one database to the other. And if agents from one database were allowed to wander through the other, then the two databases would be linked together on the level of dynamic structure: they would be two entrained, complex dynamical systems. The closeness of the link between the two systems would be determined by the propensity of the agents to leave their home database.

Imagine, next, that instead of two separate Webmind databases, one had, say, ten million. Imagine, in other words, that every website was its own Webmind database, and that, in order to do a search, one simply sent out intelligent agents through the Web. This is the opposite extreme from the one-search-engine/one-database model, and it is not yet feasible given current bandwidth limitations. But a search engine distributed across hundreds of sites is feasible today, and given the projected rapid growth of network bandwidth, the millions-of-sites scenario may well be feasible in ten years time. Once it is feasible, one will have something remarkable indeed: one will have the possibility for a World Wide Web which is, structurally and dynamically, a mirror of the human mind. Instead of a World Wide Web, we will have, so to speak, a World Wide Brain, a global network intelligence.

The term "data mining" refers to the process of searching through a large volume of data to discover interesting and useful information. Text information retrieval, as discussed above, may be considered a special case of data mining, in which the data involved is purely textual. Much of human intelligence is "data mining" -- we are concerned with searching through a vast amount of perceptual information, regarding the outside world and our own brains as well, and finding the interesting and useful information obscured wherein. In practice, though, the term "data mining" is generally used to refer to the elicitation of patterns from numerical data such as financial time series, or data stored in databases such as customer information; and it is these applications that will be the focus in the following comments. Most applications labeled "data mining" involve prediction of future trends, or categorization of data collected over an interval of time.

Data mining answers business questions that traditionally were too time-consuming to resolve, finding predictive or categorical information that experts may miss, either because it is too subtle and complicated, or because it lies outside their psychological expectations. It provides answers to questions such as, "Which clients are most likely to respond to my next promotional mailing, and why?" and "Which direction is the price of the Deutschmark likely to move over the next twenty-four hours?" and "What kind of person is hitting my Website?"

What differentiates Webmind from other data mining packages is, first of all, its ability to ferret out subtle patterns in data. It is able to pick up regularities that other, less sophisticated algorithms miss. Just as neural networks and genetic algorithms go beyond linear statistics, Webmind goes beyond neural networks and genetic algorithms. Furthermore, Webmind's sophisticated Internet implementation allows it to deal with large databases in a highly efficient way. Unlike other data analysis algorithms, Webmind is intrinsically designed to deal with tremendously large amounts of information, stored across distributed networks. As demonstrated in the financial experiments reported above, it is also able to analyze textual and numerical information in a unified way -- a fact of the utmost importance given the vast amounts of conceptually interrelated textual and numerical data on the contemporary business computer network.

On a more conceptual level, what differentiates Webmind is the fact that it approaches a body of data in roughly the same way that the human mind approaches the outside world. It takes the body of data into itself, just as perception brings the world into the mind, and it "brings the data to life," enabling the data to transform itself and study itself. The core, low-level pattern recognition routines used by Webmind are not necessarily different from those used by other data mining systems. The difference lies in the way the low-level recognized patterns are combined to form an holistic understanding of the entire body of data. This holistic understanding is embodied in Webmind's link structure, after a period of evolution, and it results in Webmind being able to recognize extremely subtle and sophisticated patterns in its input data, in an efficient way.

In practical terms, the application of Webmind to data mining is fairly straightforward. Data is read into Webmind and forms itself into a population of nodes; at which point the Webmind dynamics goes to work, bringing out the structure implicit in the data. Questions posed to the system are then resolved in terms of the information that Webmind has already discovered, on its own. If one's information is stored on an intranet, then absolutely no pre-processing is required, as Webmind is optimized to process distributed intranet data. If one's information is stored in a conventional database, or some other format, this is not a serious problem either, as the addition of simple additional software modules allows Webmind's to speak SQL and other relevant database languages.

Webmind carries out the two basic tasks of data mining -- prediction and categorization -- in a flexible and unified way. Categorization is taken care of by link update methods, as described above. Categories among, say, customers of a business, or stocks in the market, are found in the same way that the Webmind search engine isolates categories among Web pages. As for prediction, this is essentially a matter of recognizing patterns in one data set (representing, e.g., the past history of a financial market), and then extrapolating these patterns to another, perhaps unknown data set (e.g. the future of a financial market). The recognition of patterns is the crucial part. Neural networks and genetic algorithms pick up patterns that statistics and decision trees miss; and Webmind, with its self-organizing hierarchical/associative structure and its ability to read text and internally relate text with data, picks up patterns that neural networks and genetic algorithms miss.

As a single example of the application of Webmind to commercial data analysis, outside the finance industry, consider the problem of "targeted marketing." A business maintains a list of its customers over a period of time, which contains a fair amount of information about each customer. By studying this list, one can determine the qualities that typify a "good customer" for this business. The business then obtains a large database of names, e.g. census data, and wishes to determine which people from among this large database are likely to be good customers.

The specifics of this problem will vary from case to case, depending on the nature of the database. But the basic principles underlying the application of Webmind to this type of problem are quite general and flexible. Most simply, one may begin by building a Webmind database consisting of the elements of the large database. In the case of census data, each Webmind node might correspond to a single block, or even a single household. One then sets Webmind to work, building informative links between the nodes. Different links specify different types of relationship. For instance, there will be "geographical links," which link two nodes and are weighted by the distance between the blocks represented by the nodes (two nodes will be linked by a geographical link only if they are relatively close together). There may be links tied to particular parameters, e.g. "age structure" links or "income distribution" links. The different types of links are not chosen haphazardly, but are chosen based on the outcome of pattern recognition operations within the nodes. Each type of link is constructed by swarms of agents equipped with an appropriate criterion function, an appropriate measure of similarity between nodes. Higher level nodes are then formed, representing clusters amongst nodes -- natural groupings of "mutually similar" nodes. Finally, the elements of the smaller database are then entered into this network like queries into a text search oriented network. They find their most natural place in the network, extend links, and are activated. The most active nodes are then the "similar" nodes within the larger database.

This differs from typical statistical or AI approaches to market analysis in many essential respects. In terms of performance, the biggest difference is Webmind's ability to carry out highly targeted market analysis. Existing algorithms are effective at weeding out terrible customers from the large database, but only mediocre as isolating the best customers. Webmind, on the other hand, has a deeper understanding of the structure of the large database, because it begins by allowing the structure of the large database to discover itself. This deeper understanding allows it to target precisely those customers who are most similar to the business's best customers -- just as, in the Webmind search engine, a query finds those Web pages that are the most similar to it.

Similar approaches have been worked out for diverse other data mining applications -- no business intelligence problem lies outside its domain. The architecture encompasses the most effective aspects of previous approaches -- such as statistical clustering, neural networks, and genetic algorithms -- into an overarching framework. It integrates the two main objectives of data mining, categorization (carried out by higher-level nodes) and prediction (carried out by pattern-recognition methods within nodes), in a seamless way. As with Web search, it is not a panacea: there is still a certain amount of work required to tailor the prediction and link update methods to fit the nature of particular application domains. And searches for extremely subtle patterns in huge data warehouses can still take time, even with Webmind's sophisticated internetworking capabilities. But Webmind represents a significant step beyond what is out there; and it has the flexibility and reflexive adaptability to take advantage of improvements in Internet technology and data mining algorithms, without straying from its basic framework, which is, after all, nothing else but the structure of mind.

"Information retrieval" and "data mining" are concepts that were created to describe the behavior of software much more limited in its behavioral repertoire than WebMindä. WebMindä, with its ability to understand context and to ground concepts in its own experience, also has the ability to do much more than any mere information-extraction tool. It is able to serve as a true knowledge management and knowledge creation system, integrating information from the document archives and databases of interest to a business, and allowing exploration of this information in all its facets, each piece considered in the context of the whole.

If WebMindä is installed on a company's intranet, then real-time queries regarding relationships between textual, numerical and other data to do with the enterprise may be posed by any employee with computer access at any time. The result is that WebMindä's intelligence is integrated with the social intelligence of the organization, and the individual intelligence of the employees. Each time an employee accesses a document, he may ask WebMindä for related documents, and WebMindä will carry out this task with an understanding of the role of that employee in the company, that employee's particular needs and interests, etc. It will be able to make its own creative suggestions, based on its autonomous thought processes.

Furthermore, the social dynamics of the different WebMindä servers residing in different parts of the company's intranet (governed by WebMindä's highly innovative, proprietary server-server communication methods) will grow to reflect the social dynamics of the individuals using those parts of the intranet. Each WebMindä server will respond most effectively and rapidly to queries involving information which it stores locally; but the information that a certain server stores locally may change over time, depending on user needs and internal Psynet dynamics. Thus, while providing easy access by all users to all information at all times, WebMindä will nevertheless nudge the information at the readiest disposal of individual humans and divisions in certain directions, based on its inferences and its own emergent understanding. WebMindä will do more than just provide an understanding of structures and processes; it will be a participant in processes, in the formation of emergent human and informational structures.

And, as various WebMindä units in various organizations exchange non-proprietary information, in the interest of increased mutual intelligence, WebMindä will be a participant in the formation of human and informational structures on the global scale. This is an exciting new vision of artificial intelligence, in the business context and beyond -- not AI as something separate from humanity, providing us with answers to our questions, but AI as something interacting symbiotically with humanity, participating in our communications, goals, and social structures and processes.

Using its extremely flexible structures and dynamics, WebMindä can easily be trained to perform a variety of interesting "enterprise analysis" functions. For example, WebMindä will be able to:

• produce pictures of the amount of information passing between different people and groups

• make causal inferences: "Because Bob finishes his balance sheets on time, Joe is usually able to give meaningful information to the production department."

• isolate the most important concepts being passed between a particular pair of people or groups

• track the evolution of concepts through written communications

• discover the tightly connected clusters of people or groups, among which large amounts of information commonly passes

• discover the boundaries in the network of people or groups, which information, or particular concepts, have difficulty passing through

• produce a map of the clusters of people in an organization, separated by boundaries, and the amount and type of linkage between the different clusters

Collectively, these functions amount to making a map of an organization as a self-organizing system, and identifying the emergent structures in the organization. These functionalities will play an important role in enabling organizations to become more effective learning systems.

Achieving these enterprise functionalities will not involve adding any new core structures to WebMindä, but only creating new types of query nodes, and tuning the categorization, visualization and concept-evolution processes to work on the type and volume of data found in enterprise applications.