AI is defined in the Artificial Intelligence Dictionary as "a multi-disciplinary field encompassing computer science, neuroscience, philosophy, psychology, robotics, and linguistics; and devoted to the reproduction of the methods or results of human reasoning and brain activity." In spite of these lofty ambitions, however, the reality of AI has up to this point been severely disappointing. Outside the academic world, AI has adopted a role as one technology among many, suitable to be applied to tasks that are too non-routine for ordinary software, and are too repetitive or mathematically involved for humans to do efficiently, but do not require a general, intuitive understanding of the world. As such, various AI technologies have proved particularly useful in the finance industry, and in other specialized areas such as process control.
Webmind, the Java-based AI system under development at Intelligenesis Corp., overcomes many of these historical limitations of AI technology. Whereas previous AI technologies focus on specific mental qualities such as knowledge representation, perception, reasoning, or learning, Webmind is based on a comprehensive mathematical model of the mind as a whole. It perceives information, analyzes information, remembers information and acts on information in an integrated way. It adapts to its own environment much as human beings adapt to the physical world. It evolves its own spontaneous structures, rather than relying too heavily on the intuitions of human programmers.
Webmind does not have a general intuitive understanding of our world, but it does have a general intuitive understanding of its own world, the data stored on the Internet, and the interests and responses of human Internet users. This is the qualitative difference between Webmind and previous AI systems; and this is why Webmind has the potential to bring AI beyond the role of a specialized adjunct to unintelligent computer systems, and to make AI the heart of computing itself as we move on into the new era of super-powerful digital networks.
At the present time, the domain in which Webmind has been most systematically and scientifically tested is financial analytics. The goal in these tests was simple, to be able to make better financial predictions and trading decisions than one could without the software. The finance industry has already developed sophisticated tools for projecting trends in markets, using quantitative data about what the markets have done in the past. The problem with these techniques is that they do not encompass the full realm of information available to the markets.
The key advantage of Webmind for this purpose is that it can incorporate a logical analysis of textual material, material from financial news sources. This, of course, is very similar to what market analysts do. They study the statistical data, then interpret it in view of information which they obtain from their reading and conversations with industry leaders. However, there is a tremendous mass of written material available, and not enough time for any analyst to digest it all. Traders for large investment firms have difficulty even finding the time to read the reports generated by their own staff analysts. Webmind computerizes this process.
Webmind deals with textual and numerical information on an equal footing, freely making generalizations spanning different types of data. Unlike any other AI system, it forms its own intuitions regarding concepts by dynamically "grounding" linguistic concepts in terms of its own nonlinguistic experience. It is the first AI system able to create its own concepts that apply across different contexts. Webmind runs efficiently on powerful stand-alone computers, but is most powerful when run over a network of computers, in which case its sophisticated server-server communication methods allow its internal network structure to harmonize with the connectivity structure of the computer network. Webmind does not aim to emulate human intelligence in an artificial way, but rather to be a natural intelligence for the digital world -- solving human problems with digital intuition. Initial tests of Webmind's ability in the area of financial decision support have proved dramatically successful. Further application areas under active development include information retrieval, data mining, and enterprise analysis.
Though it is a new kind of software system, Webmind incorporates elements or functions of various standard AI technologies. It can serve as a search engine, as a Web visualization tool, or as a data mining engine -- and it can also serve functions not implied by any of these pre-existing categories. Most fundamentally, it is a system for the structuring and dynamic evolution of large bodies of data. It is an intelligent system for knowledge management and knowledge creation, with an almost boundless array of potential applications in business, science and personal computing.
Webmind collects and analyzes information in much the same way as the human mind does, but with different strengths and weaknesses, befitting its physical basis in silicon rather than carbon, and its perceptual embedding in the world of data and text feeds rather than the world of sights, smells, sounds, tastes and touches. In the human mind, everything above the lowest levels of perception and action is dynamic and constantly evolving. The memory constantly studies itself, reorganizing to enhance and inhibit recall and performance. Current information discovery tools impose an artificial distinction between static data repositories and dynamic search and analysis tools. But what is needed, in order to really make use of the vast amount of information we are amassing, is a dynamic information system that mimics, on an abstract level, the structure of the human mind, with all its reflexive, self-analytical capabilities. What is needed is a way to make the data come alive and study itself, rather than lying there inert and being probed by various computational instruments, of various degrees of intelligence. This is what Webmind accomplishes. It allows collections of data to structure themselves and recognize patterns in themselves. Webmind transforms each informational element within a computer network into an independent agent which cooperates with other agents in organizing and reorganizing data for search and retrieval. It interacts with and builds on the intelligence of the user. Users' queries are treated as informational agents interacting via the collective Webmind agents system to locate information. Webmind agents take advantage of the sum collection of their organization's intelligence to discover context and concepts on behalf of users and their applications. In short, Webmind is the first unified computational mind, complete with integrated systems for perception, action, cognition, language, short and long term memory, etc., all embedded in a data-rich environment.
According to the psynet model, as reviewed above, the achievement of intelligence does not depend on the implementation of particular mechanisms, but rather on the implementation of mechanisms that give rise to appropriate emergent structures. The unique achievement of the Webmind design is to cause these emergent structures to emerge from current computers and networks in a computationally efficient and humanly useful way. Underlying the psynet model is a vision of intelligence as "the capability to achieve complex goals in complex environments." Webmind's complex goals are firstly posed to it by human users and secondarily evolved by itself; its complex environment consists of the human beings, machines and networks with which it interacts.
The Webmind system provides a general "agents operating system" for managing systems of software agents that share meaning amongst each other, transform each other, and interact in various ways; the agents may live in the RAM of a single machine, may be run on multiple processors, and may live across many machines connected by high-bandwidth cable. It provides mechanisms for agents to represent patterns they have recognized in other agents, and patterns they have recognized in the overall agents system of Webmind. And it also provides an appropriate assemblage of agents, specialized for such things as
text processing
numerical data processing
text-numerical interrelation
supervised and unsupervised concept formation
reasoning
social interaction
language processing
These agents have their own localized behaviors but achieve their true intelligence only in the context of the whole Webmind system.
Several important questions arise in designing an AI implementation of the psynet model:
How to represent the agents
How to manage the dynamics of a collection of agents, on the hardware provided
What to build in, and what to require the system to learn
How to embed the system in a rich environment, including other systems of like kind
What goals to give the system
What programming language to use?
How one answers these questions determines what kind of system one comes up with. One way to get across the essence of Webmind is to describe how these questions were answered in the design of Webmind, on a general level.
Question 1: How to represent the agents?
Answer: As nodes in an abstract network, and relational and mobile agents associated with nodes
In Webmind agents are realized as data structures called "nodes," with particular structural and dynamical properties. A psynet node is defined as a container for one or more of the following:
possibly some "raw" perceptual data
a record of the meaning of the current node, in the context of the mind as a whole
a transformation which acts on a node, and produces a node
a transformation which acts on a node, and produces a number
tables of links to nodes with which it shares meaning in certain ways
mobile agents which seek other nodes to add to the link tables
information on how much attention is paid to the node currently and over the recent past, in total and from the perspective of various other nodes
information about how important the node is, and how fast its importance is expected to decay over time
It should be clear that the node data structure, defined in this way, is adequate for expressing the "agent" concept of the psynet model of mind. Transformations taking agents into agents and agents into numbers are here, as required; and the dynamical law of mind, which says that attention follows meaning, is explicitly supported via each node carrying lists of other nodes that share its meaning, and agents searching for nodes that share its meaning.
The view of the mind as a network of psynet nodes is a kind of "middle way" for AI -- or in other words, an "intermediate level" model of intelligence. It stands between, on the one hand, neural network and other physiologically inspired models of intelligence; and on the other hand, abstract logical and psychological models of intelligence that speak of mind in the abstract, completely apart from its physical implementation. Neural network and logical and psychological models all have their particular validity, but considered as a collective, they leave a crucial gap: how do the complex, self-organizing physical dynamics of neural networks and the like give rise to the abstractions of the mind, such as syntax, logic, inference and self? "Psynet" means "mind network," and so it is only appropriate that the network interpretation of the psynet model fills in this gap by positing a network that is not a network of neurons but a network of intermediate-scale, mental entities ... "agent" nodes.
In a neurophysiological context, it may be useful to think of a psynet node as a neuronal group in the sense of Edelman's Neural Darwinism brain theory: each of Edelman's neuronal groups carries out a specific useful function, quite different from a single neuron, which in most cases has no particular function in itself. In some computer implementations of the psynet model, it may be useful to take this mapping fairly literally, and implement nodes in terms of networks of formal neurons -- as we shall see, something like this is done in Webmind, in certain cases, though not in general. But this is not a necessary approach: psynet nodes could be implemented as LISP program trees, as high-degree polynomials, as physical entities outside the realm of digital computing (Bose-Einstein condensates?), etc. The psynet model is abstract: it has to do with the structure and dynamics of the network of agents, not with the internal makeup of the agents themselves. Mind design, in this perspective, is largely about creating the correct agents out of the tools at hand, to ensure the emergence of the desired structures and dynamics from the system as a whole.
In "agent system" terms, we may say that agents in Webmind fall into several classes. First there are nodes, static agents, which have a relatively persistent state. Nodes may contain information relating to data such as texts or database files or internal information such as categories or other patterns recognized among other nodes. Nodes are not truly static, they update themselves continually, but they represent the backbone of 's intelligent internal network. Most of the information contained in a node actually consists of a record of the node's relationship to other agents in the Psynet. There are many different types of node, pertaining to different types of information.
In addition to nodes, there are relational agents or links, which establish and retain relationships among nodes. These relationships may fall into many different categories, and may be parameterized by qualities such as intensity, uncertainty, and so forth. Finally, relational agents are formed by the third kind of agent, mobile agents which move around within the internal relational geometry of an individual Psynet, potentially from one server to another, actively seeking new relationships to embody in relational agents. Mobile agents associated with nodes can also move from one to another, across the Net, but this requires a different brokering mechanism than the free flow of agents within the same server, or between different servers on the same Intranet.
Question 2: How to manage the dynamics of a collection of agents, on the hardware provided
Answer: On contemporary computers and networks, collections of nodes must be stored in large amounts of RAM; they must be dynamically groupable into sets to allow multiprocessing; and they must be able to migrate and send messages across network cable in order to allow a single mind to span multiple machines
The dynamics of the network of nodes in Webmind's psynet is as follows. When a node receives attention, it spreads some of its attention to the meaning-sharing nodes that it knows about -- i.e. attention follows the links in its link table. It may also update its record of its own meaning. Finally, it may carry out its transformations to produce new nodes, or it may modify its transformations, with the goal of making them more prominent patterns in the collection of other nodes with which it shares meaning.
Ideally, each node would run independently on its own processor. However, that is not feasible at the present time: massively parallel processors do exist, but are orders of magnitude less efficient than networked top-of-the-line serial computers. Fortunately we are presently blessed with computers that, though serial in nature, possess multiple rapid processors, large amounts of RAM (in the gigabyte and up range), and fast cables (e.g. Ethernet) networking them together. This combination allows us to run large magician systems on serial machines.
The system architecture of Webmind is roughly as follows. A single mind network, may run across a number of machines, interconnected by high-bandwidth connections. Nodes may send messages across machines, and may share meaning with nodes on other machines; they may also move from one machine to another if this is judged dynamically efficient. Each machine contains a managerial process which determines when dispatching of nodes and messages between itself and other machines is appropriate. This managerial process must pay attention to the psynet dynamics on its own machine, for various reasons.
Within a single machine, nodes are divided into groups called pods. Grouping into pods is useful for human saving/loading of groups of nodes -- a pod of nodes may serve a particular human purpose, e.g. it may contain nodes concerning personnel records, nodes concerning financial data, etc. It is also useful for multiprocessing: a pod must live on a single processor, but multiple pods living on multiple processors may be very elegantly and efficiently handled. When a node wants to transform or spread attention to another node, living in a different pod, it sends a request to its pod's managerial process, which then sends the request on to the managerial process of the pod at the receiving end.
Question 3: What to build in, and what to require the system to learn?
Answer: Build in "preprocessing" routines for transforming particular kinds of input data into patterns, and for producing particular kinds of output data from node states. Build in a vocabulary of useful magician types. Beyond this, build in other things as short-cuts, but nothing which the system is not capable of learning on its own in a non-absurd period of time
During the early development of Webmind, this was the toughest question to answer. One could seek to implement the psynet model of mind by setting up a kind of "primordial soup" of transformations, mapping into numbers and mapping into other transformations. Assuming this soup was hooked up to rich data feeds, given extra resources in return for success at practical tasks, and given a reasonable capacity to mutate itself, it would eventually transform itself into something intelligent, arriving at the structures described above by self-organizing evolution. This methodology, however, is extremely inefficient, and is extremely unlikely to work given the limitations of current engineering technology. I tried this approach in 1994 and failed miserably; my comoc.com collaborator tried it in a different way in 1997, and also failed. This approach is conceptuall flawed in that mixes up what in biology are three different levels of intelligence: evolution, epigenesis, and learning. In order to arrive at a practical implementation of the psynet model, one must build in more of the desired structure -- just as a human being is born with some of the desired structure of mind already present in the structure of the brain.
The question then becomes how much one can build in without sacrificing the dynamicity, creativity, and self-modification that are at the essence of intelligence. The correct answer, we have found, is to engineer:
the processing of low-level perception and action
the types of agents potentially in the system (which determine not which mind you have, but what kind of mind you have)
the specific configuration of agents, but only as a guide for future learning and not as a fixed, inorganic base of knowledge
The built-in preprocessing of low-level perception and action is necessary because of resource constraints. All that the human brain does by hard-wired visual circuitry -- line extraction, etc. -- could be done by more intelligent parts of the brain, but this would be a waste. The amount of pattern recognized in the world is maximized by allowing pre-processing to take care of the lowest level of pattern-recognition, and focusing the attention of intelligent on the output of the preprocessors. In the current Webmind system this lesson is reflected as follows: some basic text processing routines are built in, as are some numerical data processing routines that recognize dates in numerical data files, etc.
In the current system, a carefully chosen variety of node types is provided also: nodes for text, nodes for numerical data, nodes for words/phrases, nodes for senses of words, nodes for categories, etc. Each node type has specific means for determining shared meanings and for transforming itself and other nodes. The strategy is to build in the node types, and then allow the nodes, of selected types, to self-organize into their own structure based on their inputs. The key principle to remember here is the gradation of specialization. A system consisting of only highly specialized node types will never have the flexibility to be intelligent in the real world; but a system consisting of only highly general node types (not evolved or engineered for any particular world) will have a hard time coming to grips with its environment. Specialized node types are a step up from perceptual/active preprocessing, but serve a similar role: to enable the system to focus most of its attention on problems which require general intelligence.
In the current Webmind system, we have highly specialized node types for two things: language processing, and analysis of numerical data, particularly financial data (e.g. a node type specifically representing a trading system). Preprocessed data from these sensory modalities feeds into specialized nodes at the same time as it feeds into general-purpose nodes, and the interaction between the specialized and general-purpose nodes at this stage of processing is intense.
The interplay of different node types is is in some sense the hardest part of computational mind engineering. The essence of mind, as depicted by the psynet model, is relatively simple: what distinguishes one type of mind from another is mainly the types of agents that live inside it. The interaction between specialized agents and general-purpose agents is something that, in humans, evolved over millions of years; it can be engineered, but only with a firm understanding of the general underlying dynamics of mind, and a healthy respect for the difficulty of the task.
Question 4: How to embed the system in a rich environment, including other systems of like kind?
Answer: The Internet!
Humans receive a vast amount of perceptual data via their sense organs; and we interact with the same world producing this data via speaking, moving, stinking, etc. In short, we are embedded in our bodies. The body of Webmind consists of the computers on which it is implemented, which have their own sensory and active organs, for reading information from other computers and from human users. Pre-Internet, the data feeds available to computers were fairly sparse, and the ability for a computer program to interact with humans and other programs was strictly limited. At the present time, however, it is possible to hook up Webmind to a vast number of textual and numerical datafeeds, to supply with an unlimited amount of user interaction via corporate intranets and the Web, and to allow it to act by sending messages to individuals and other computers. For the first time, there is now a rich perceptual environment available to a computational intelligence. Furthermore, there is the opportunity to place a community of AI systems within this common environment, giving them access to common perceptual data and the ability to interact with one another in a variety of ways.
This is a somewhat obvious point, but it is one that has not been exploited by the mainstream of AI researchers, who do not accept the essentiality of a rich environment for the emergence of intelligence.
The social interaction of Webminds is explicitly mediated by data structures corresponding to the Self (the Webmind itself), other Webminds (needed for intelligent inter-Webmind querying), and human users (needed for intelligent customized response).
Question 5: What goals to give the system?
Answer: The goal of giving meaningful answers for questions asked by important entities (human and computer), at present and into the future
In order for a mind network to evolve effectively, it must be given goals to carry out, and must be given feedback on the quality with which it has carried out these goals. Just as humans are built to want to eat, have sex, drink, exercise, etc., so Webmind is built to want to answer queries effectively. In order to assure this, each time it sends a query, it should receive an assessment of the perceived quality of its response. This may be explicit (e.g., the user may be asked), or it may be implicit (e.g., the client may calculate the number of times the human user needed to rephrase his question to get the answer desired). We all know how hard it is to guess what humans want -- what this means is that answering human questions is certainly a "complex goal" as required in our working definition of intelligence, which is "the ability to achieve complex goals in complex environments."
Question 6: What programming language to use?
Answer: Java
The Java language was one of the inspirations for Webmind, so it might seem a 100% obvious choice. Even so, though, serious consideration was given to the possibility of programming Webmind in C++, a language which has the advantage, at this point, that C++ programs run faster than Java programs. But, in spite of its current speed advantage, C++ lost the context, for a number of technical reasons:
Java shares with LISP and other artificial intelligence languages a feature called automatic garbage collection. This means that, when a certain piece of the computer's RAM is no longer being used by a Java program, the Java language will take the initiative to free up this piece of memory so it can be reused by another part of the program. In C/C++, this free-ing up of memory has to be done explicitly by the programmer.
Java is Internet-friendly -- its sophisticated networking libraries allow flexible inter-machine communications to be programmed relatively easily
Platform independence, for the client and the server! Programmed in Webmind can run on PC's, Macs or Unix boxes
Multi-threading is a programming technique used to make programs that act like they're doing a lot of things at once, even though they're really running on a one-thing-at-a-time serial processor. Of all popular programming languages, Java provides the most effective and easy to use multi-threading. In particular, this means that, in Java, it's really easy to make a program that keeps doing its own internal processing in one "thread" of control while communicating with users or with other computers in other "threads."
With the advent of ultra-efficient compilers such as IBM's Formula, the speed gap between C++ and Java is narrowing; our own tests show a ratio of 2.5:1. The Webmind server will also be able to exploit the added power of the Java chip, to be introduced in 1999. Finally, the much-publicized "Java browser wars" have no significant effect on Webmind, as the Webmind server is not browser-based, and the Webmind user interface doesn't need to run in the browser, or even need to be written in Java - in fact, our first user interface is actually written in C++, and talks to the Java portion of Webmind using (what else?) standard Internet communication protocols.
The nitty-gritty details of the Webmind software architecture are proprietary and cannot at this stage be discussed in detail in a public document. However, some important aspects of the design may be discussed at a high level of granularity, and may help the reader to grasp the system's essential nature.
A single Webmind system consists of a collection of nodes called a Psynet, which spans a collection of one or more Webmind servers, each one running on its own (possibly multiprocessor) machine. The intelligent core of a Webmind server is its collection of nodes, which is called a PsynetLobe. Each PsynetLobe, in turn, consists of multiple pods, each pod containing a number of nodes.
At the present time, the boundaries of a single Psynet are the boundaries of the intranet or extranet: Ethernet connections are fast enough that Webmind servers thus connected can act as an emergent whole; whereas Internet connections are too slow. Thus, one has a model of the intranet as the individual; and the Internet as a population of Webminds interacting in a manner best described as "social", exchanging information and questions and answers but not participating in collective trains of thought on a real-time basis. This of course may change as bandwidth increases over the next decade; at some point the Internet will become as fast as today's Ethernet, and a single global Psynet will become achievable.
Every Webmind server comes with a standard collection of pods: pods for standard data types such as text and numerical data, several pods for natural language understanding, a pod for query processing, and other pods devoted to such tasks as creative brainstorming, self-reflection, socialization, etc. Other pods will be application-specific: e.g., a Webmind server aimed at financial applications would contain a pod of historical financial data; an enterprise application would contain a pod of transient e-mail data; etc. Additional pods may be loaded and saved by the user.
The dynamics of the Psynet is fairly simple, though the emergent structures that these dynamics give rise to may be quite complex. Each pod carries its own threads, and these threads cycle through all the agents in the pod, allowing them to carry out "elemental actions." Each node carries a "heat" indicating its importance at a particular time, and hot nodes are given a greater priority.
The starting-point for Webmind's intelligence is a collection of basic node types used to represent data found on current computer networks. There are TextNodes for representing text documents, DataNodes for representing numerical data, and several types of CategoryNodes for representing concepts inferred from text and data. There are also TimeSets, representing collections of data that occur over time; and TimeNodes, representing collections of information considered roughly simultaneous. These node types are the basis for Webmind's intelligence; other processes work with these nodes to generate meaningful information of various forms. Information mined from data warehouses is read into the system and translated into a collection of TextNodes, DataNodes, CategoryNodes, TimeSets and TimeNodes; a flexible format which is amenable to sophisticated intelligent analysis by Webmind's various internal agents.
What is universal to all nodes in Webmind is:
attention (activation), current and cumulative
links representing direct, local relationships to other nodes
links containing information on other "distant" nodes that are bound up with the node in emergent dynamical reactions (this is the "halo" of the node)
the process of spreading attention along links
the process of building links to other nodes, by sending out agents called Wanderers
That is, all Webmind nodes manifest Peirce's Law of Mind, which states that attention follows meaning, that each aspect of mind spreads some of its energy to each other aspect of mind which stands to it in a "peculiar relation of affectability."
Wanderer agents, resident at each node, travel through the network of nodes in search of other nodes that stand toward their home node in a peculiar relation of affectibility. Specifically, they seek out symmetric (similarity), asymmetric (inheritance) and emergent relationships. In the case of emergent relationships, they may create new nodes embodying emergent patterns between their home node and the node they are currently visiting. Wanderers move through the net in a stochastic way, following links from one node to another and honing in on those regions of the net containing the most nodes relevant to their home node.
Beyond this, specific capacities are carried by specific node types. The assemblage of built-in node types is important and is considered analogous to the genetic heritage of specialized human brain modules. Humans are born with some pre-existing ability to process language, to socialize, to perceive and to integrate perceptions cross-modally, to form abstract concepts according to certain archetypal templates, etc. Webmind similarly enters the world with these pre-existing abilities, albeit in somewhat different form.
First of all, Webmind starts with two low-level "perceptual" modules: one containing nodes for text processing, another containing nodes for numerical data processing. Text and numbers are Webmind's perceptual world, and it deals with both in great profusion. Specialized preprocessing is necessary and appropriate here, just as our eyes and ears preprocess sensory data for our brains.
In order to intelligently deal with text information, Webmind starts with a natural language module. The nodes contained herein carry out both semantic and syntactic analysis. The NL system takes in text and parses it, and outputs the parsed text into nodes and links, namely "inheritance links" representing relations of categorial inclusion (e.g. from "big cat" to "cat") and ternary links representing general triadic relationships (e.g. joining "big", "cat" and "big cat"; or "to" ,"run" and "to run"). The parser, whose unique and innovative design is described in a separate document, is specifically designed to parse sentences into collections of trilinks.
Note that language processing is not rigidly separated from thought processing here. Webmind does not have a separate "natural language system" as such. Language processing is carried out by nodes representing words and phrases, nodes carrying short term memories of recently read material, nodes concerned with categorization, and mobile agents concerned with recognizing similarity and other relationships. To disentangle language from thought is to remove most of what is interesting and important about language.
The first part of Webmind's language understanding is linguistic categorization, which is carried out by category nodes according to the normal dynamics of the Psynet. Webmind's self-organizing categorization is capable of determining parts of speech for words, and of determining semantic categories such as "house", "action", etc. as well.
Once straightforward categorization has been used to create semantic and syntactic categories, attractor neural network methods are used to refine semantic links between words, based on contextual information. Words are then joined by relational agents if they are related closely in the bodies of information that has been exposed to.
Parsing is carried out by a proprietary, original self-organizing parsing method based on L-system grammar. This system takes a sentence, uses the Psynet to determine syntactic and semantic categories for the words in the sentence, and then maps the sentence into a collection of nodes and relational agents, which are placed into the PsyContext and then into the Psynet. If the sentence represented a query, then the collection of nodes and agents corresponding to the sentence is activated rather than merely stored, and a query agent is created which collects the results of this activation after a period of time.
The ingestion of linguistic as well as numerical data is mediated via the short term memory module. The most recent items read in through a given input stream are stored in short-term memory and the various intercombinations of these items are explicitly represented in a temporary way. This system is crucial, among other things, for disambiguation of linguistic terms.
As important as language understanding is, however, it is not all-powerful. Relations learned through the natural language system are not intrinsically "understood" by the system -- they represent purely formal knowledge. Webmind's grounding module, on the other hand, contains nodes that allow the system to derive knowledge directly from its environment. This module contains GroundedRelationNodes, which evolve automata that recognize relations among data that Webmind has directly at its disposal (e.g., recognize which DataNodes correspond to volatile markets, which TextNodes correspond to novels, which ones to long novels, etc.).
Mobile agents of various types create associative and hierarchical relationships, thus instantiating the elements of a cognitively powerful "dual network" structure. But what guarantees that these two structures, heterarchy and hierarchy, will work effectively together? There are several answers to this question within the Webmind architecture, but one of the keys is the explicit search for meaning. Each portion of the net, searching for its own meaning, re-expresses itself in terms of the hierarchical and heterarchical connections surrounding it, thus causing the hierarchical and heterarchical connections in its area to function effectively together.
In order for an intelligence to understand the meaning of a word, concept or other entity, this entity must be "grounded" in the system's own experience. Consider, for example, the position of Webmind on encountering the word "two." In order to deal with "two" in an intelligent way, the system must somehow understand that "two" is not only a pattern in texts, it also refers to the two computer users who are currently logged onto it, the two kinds of data file that it reads in, etc. These are all patterns associated with "two" in various situations. Associating all these patterns with the "two" category node is what fully integrates the "two" node with the rest of the network, guaranteeing that the behavior of the "two" node is in harmony with its surroundings.
Through processing of text alone, part of the fuzzy set that is the meaning of "two" can be acquired: that part which is purely linguistic, purely concerned with the appearance of "two" in linguistic combinations with other words. But the other part of the meaning of "two" can only be obtained by seeing "two" used, and using "two," in various situations, and observing the patterns in these situations that are correlated with correct use of "two." In general, the fuzzy set of patterns that is the meaning of a symbol X involves not only patterns emergent among X and other symbols, but also patterns emergent among X and entities involved in various situations. Recognition of patterns of this type is called symbol grounding.
Symbol grounding is connected in a fascinating way with introspection. In Webmind, each agent has the ability to look inside itself and report various aspects of its current state. Symbol grounding is achieved by special mobile agents attached to individual nodes representing concepts (e.g. the node for two"). These agents are evolved by the genetic algorithm; they survey the introspections of other nodes and combine the results of these introspections to form "groundings" of the node to which they are attached.
The actual mechanics of symbol grounding is carried out in within Webmind in a unique way: groundings for symbols are evolved by special nodes that carry out processes similar to the genetic algorithm. As noted earlier, the relation between the genetic algorithm and the brain is somewhat uncertain. Gerald Edelman's "Neural Darwinism" theory argues that the brain can be viewed as a genetic algorithm with mutation only; however, as I argued in my book The Evolving Mind, a case can also be made for crossover between "neuronal maps." At very least, we may view the use of the GA in Webmind as a more efficient way of accomplishing, using crossover and mutation, what the brain may accomplish with mutation only.
Both language based understanding and grounded understanding are essential to mind. But with just grounding nodes and natural language nodes, the system would have a divided mind: formal (social) knowledge on the one hand and experiential (individual) knowledge on the other. What bridges the two is reason. The purpose of the Reason module is quite simple: to derive new links from existing links. This is useful not only because some links are derived from the outside, e.g. via the NL system; but also because some purely internal links in the system, at any given time, were derived from information that is currently forgotten. Webmind contains node-resident and mobile agents whose specific purpose is to carry out this dynamic of building new links from old.
The specific behavior of these agents is in part borrowed from the NARS (Non-Axiomatic Reasoning System) AI system developed by Intelligenesis engineer Pei Wang during his academic career. Dr. Wang, in his previous research work, has shown mathematically that reasoning agents such as those used in Webmind are capable of carrying out deductive, inductive and abductive (hypothetical reasoning) in accordance with the requirements of mathematical logic. Unlike in typical AI reasoning systems, however, here reasoning is carried out via self-organizing processes that are integrated with the remainder of Webmind's intelligence.
The importance of reasoning can be well understood in the context of symbol grounding: reasoning allows Webmind to extend conclusions from grounded domains to ungrounded domains. For example, if Webmind has learned that "big" means a certain thing in the context of financial data (where it can ground the symbol "big" in reference to its direct experience), then it can use reasoning to transfer the relations it has derived about "bigness" to domains where it has no direct experience, e.g. to understand a phrase like "a big upheaval in the Japanese government."
Next, the categorization module allows the system to create new concepts from existing ones via statistical grouping. This leads to concepts of broader scope than are obtained through the grounding system. It does not act on perceptual data directly but rather on higher-level nodes in the system. For instance, applied to TextNodes representing Web pages, it might arrive at a Yahoo-like categorization of texts.
There is a text-numerical interrelation module -- Webmind's analogue of the parts of the brain performing crossmodal correlation between the different senses, such as sight and hearing. For example, it evolves special nodes which are linked to both DataNodes and collections of TextNodes, and which are highly useful for such applications as using information in newspapers to predict financial markets. These cross-sensory links allow activation spreading to move between text and data in an intelligent way.
There are also QueryNodes, for mediating interactions with human queriers. Webmind's query processing is integrated with the rest of its mental activity, just as for a human being, question-answering is not so different from purely internal thought processing. When a query (be it a series of key words, a paragraph of natural language, or a series of commands requesting particular data operations) is entered into the system, a node is created for it, the query node sends out mobile agents, and these agents create new relational agents joining it and other nodes. Activity related to the query node spreads through the Psynet, and after a certain period of time, the nodes with the highest activity relevant to this particular activation process are collected, and returned as the answers to the query. The distinction between activity due to a particular query and activity due to general Psynet thought processes or other queries is carried out via an innovative, proprietary technique of "parallel thought processes," which allows to do one thing the human mind cannot: carry out hundreds or thousands of simultaneous trains of thought, and keep them all straight!
Every Webmind has a "self pod", containing a SelfNode, and various nodes linked to by the SelfNode. No Webmind should have multiple self pods, as this would lead to dysfunctions similar to multiple personality disorder (MPD). A SelfNode can have subcategories corresponding to "subselves," a feature which also has the potential to lead to MPD, but only if managed improperly. The purpose of the SelfNode is multiple, including at least the following:
self-control: to enable the system to determine its own resource allocation, apart from the resource allocation directed by queries and by basic activation and wanderer spreading
episodic memory: to enable the system to compare present situations to past situations, and draw inferences therefrom, and summon appropriate past nodes from disk as necessary
self-description: to enable the system to describe itself to other Webminds or humans
social understanding: to enable the system to conceptualize its relationships to other Webminds or humans and draw conclusions therefrom
Webmind also contains a social module, with specialized nodes mediating its interaction with humans and with other Webminds. There is a "friend pod", containing FriendNodes corresponding to other Webminds. And of course a "user pod" containing UserNodes corresponding to users; and a pod of nodes corresponding to other agents acting over time, with whom the system does not have direct experience, but whom the system reads about. UserNodes representing human users are an essential part of the user-interaction system, allowing Webmind to disambiguate query terms based on its knowledge of the particular user who asked the query. FriendNodes representing other Webminds are essential in the task of figuring out, when the answer to a query is not known, which other Webmind to direct the query to.
Finally, a given Webmind potentially contains, at any given time, a number of short-term memory nodes, containing information regarding what has been recently experienced by a certain node, and other things judged to be associated to these recent experiences. The short-term memory node's dynamic is called "lineal awareness," and consists of a loop that takes in new data, processes it in conjunction with the nodes in the short-term memory, and then places it in the short-term memory, removing some obsolete information. Specialized pods for dealing with self, social interaction and creativity are dealt with in a fairly similar way to the PsyContext; they contain agents which, rather than being relevant to recent experience, are relevant to particular areas of moderately-recent experience, e.g. experience with oneself, experience with other intelligent entities, and experience with difficult problems.
In short, we have a system whose immediate perceptual reality is numerical data and linguistic communication, which perceives this reality using a freely combinatorial short-term memory, derives its own concepts based on numerical data using evolving automata, which learns formal ideas from parsing language, and then extends its ideas from its experiential domain to other domains using reasoning, where the focus of its reasoning at each time is guided by the spread of attention in the direction of meaning, and the meaning of each part of the system is determined by both global dynamic relations and local static relations.
An individual Webmind server running in a single Java process is, potentially, an autonomous Webmind. On a multiprocessor machine with a large amount of RAM (say, 4 Gig), one can achieve quite a powerful Webmind in this way. Given the current state of computer hardware, however, the most cost-effective way to achieve a very smart Webmind will be to network a number of Webmind servers together into a single multi-machine Psynet.
To understand the implications of these hardware restrictions, which push toward a network implementation, one must consider them in a broader context. They overlap with fundamental system-theoretic requirements in an interesting way. Hardware considerations lead us to distinguish three levels of Webmind system:
Single machine
Network of machines connected by fast cable (intranet)
Network of machines connected by slow connections (internet)
These levels may shift over time, as the Internet becomes faster and faster.
On the other hand, analogy with biological intelligence leads us to distinguish two levels of systematic intelligence:
individual intelligence
society of individual intelligences
Intra-mental interactions may encompass social interactions, but not vice-versa: parts of the same mind have access to each other in ways that separate minds do not.
We use the term an "individual " or simply "a " to refer to an individual intelligence; and the term "global " or " society" to refer to a collection of intelligences. At the present time, the most natural mapping from social/individual concepts to computer hardware is
machine = part of individual
intranet = individual
internet = society
For now, therefore, when talking about a single "Webmind" we generally mean an intranet, a small network of machines running with a single self. However, this mapping may not remain ideal forever: eventually it will be possible to have coherent Webmind individuals running across thousands or millions of machines, around the world.
Social interactions between Webminds are as follows:
One Webmind can query another Webmind, just as if it were a client.
One Webmind can send a agent to visit another Webmind, to gather information (create relational agents involving agents on the other Webmind).
Determining which Webmind is best suited to receive a query or a visiting node to is done using a fairly sophisticated "socialization" system, involving FriendNodes and the like.
Intra-mental interactions encompass social interactions but add an additional components. For example, nodes within a single Webmind (currently, a single intranet) will reach out to one another when activated and reinforce the relational agents intertwining them; nodes separated by a slower Internet connection will not.
To ensure clarity for the reader with a computer science rather than philosophical perspective, it is perhaps worthwhile to cast these ideas in the language of distributed computing. Gul Agha has isolated the following five design decisions as being essential to the characterization of any system carrying out concurrent computation:
the nature of the computing elements
global synchrony versus asynchronous elements
the mode of interaction between computing elements
the degree of fairness
reconfigurability and extensibility
The "nature of computing elements" is that they are nodes, which are a special kind of actor, in Agha's terminology. Processing is asynchronous, not globally synchronous, as is necessary given the desired Internet/intranet functionality of the system. The mode of interaction between the nodes in Webmind is determined on the mechanical level by the Pods system, that handles node addressing in a URL-like manner and dispatches communication from nodes to their containing Pods, on to other Pods and the nodes contained therein. On a functional level the interactions between nodes are diverse and node-type-dependent, but centrally include neural-net-like activation spreading, genetic-algorithm-like node creation and deletion, and internal reorganization of a node's perceived relationship to other nodes, enabling the node to better represent a category or other relation amongst the nodes around it.
By "fairness" Agha means such things as guaranteed delivery of messages: in this sense Webmind does not place a premium on fairness, except in a probabilistic sense. At any one moment one node may be treated better than another by some fluke of the system; but, overall, the system acts homeostatically to ensure approximate fairness, in the sense that nodes that the system assigns a greater importance will get more CPU time. The system is comfortable with lossy communications in some contexts (e.g. spreading of activations from machine to machine) and not in others (e.g. the return of query answers to human users; the transport of nodes among machines).
Finally, reconfigurability and extensibility are assured by the modular system of node definition -- different functionalities are induced in the overall system by different node types. This is not a system with a population of agents following a fixed structure and dynamics, designed to be ideal and perfect once and for all. Rather, there are core node types which are seen as essential to realizing the structures and dynamics of the psynet model of mind, but these may be altered based on experience, and may be augmented by a limitless variety of specialized agents to deal with particular situations. In this sense Webmind is almost an "agents operating system" -- any type of agent, within quite broad constraints, could be integrated into the Psynet. Half of the task of engineering Webmind is in setting up the "operating system" mediating node interaction, and the other half is in determining the correct node types to give rise to the emergent structures of mind.
In this section, I will run through the 37-Observation version of the psynet model introduced earlier, and briefly indicate how each of the points is addressed within the Webmind architecture and its current implementation.
Observation 1. Mind is a set of patterns, where a pattern is defined as "a representation as something simpler".
Each Webmind node contains tables of links, which represent patterns recognized between that node and other nodes. A link representing a similarity is representing an approximate repeated pattern; a link representing an asymmetric (inheritance) relation is stating that one thing can be used in the place of another, i.e. as a pattern in another. Etc.
Observation 2: To be able to have patterns, one must have entities that can transform entities into numbers (or some other ordered domain, to give simplicity judgements), and one must have entities that transform entities into other entities (so as to enable representation)
Wanderers, agents which create links, are mapping entities into numbers; they are the valuers in Webmind. Transformation is carried out on several levels, but most centrally by Wanderers which emanate from one node but build links at other nodes.
Observation 3: Mind is intrinsically dynamical. The transformation C(B)=A, in the definition of pattern, implies a "change": C changes B into A
Webmind is obviously a dynamical system; its valuers, Wanderers, are dynamical agents.
Observation 4: Magicians can be combined in a way other than transformation; they may be combined in space. The result of joining A and B in space may be denoted A#B.
Special links exist for representing combination; for example the ternary link that joins "big", "cat" and "big cat" to represent the modifier relationship; or the binary "part-of" link that joins the node representing a phrase extracted from a text to the node representing the text itself.
Observation 5: Spatial combination gives rise to the possibility of emergent pattern: patterns that are there in A#B but not in A or B individually.
Wanderers may explicitly search for emergent pattern, and create emergence nodes.
Observation 6: The meaning of an entity may be defined as the set of all patterns associated with that entity -- where a pattern P may be associated with an entity A in several ways: P may be a pattern in A, P may be an emergent pattern in the combination A # B. or P may close to A in spacetime (P and A may have occurred in about the same place at about the same time)
The meaning of a node is effectively taken as its link tables. Link types exist to denote spatial and temporal relationships.
Observation 7: Pattern and meaning are subjective, in that they depend on who is measuring simplicity, and who is defining the set of permissible transformations
Wanderers take the subjective view of the node who sends them out
Observation 8: Meaning is dynamic as well as static. The patterns in an entity include patterns in how that entity acts, and interacts, over time.
The halo of a node encapsulates the dynamic component of meaning
Observation 9: In any given mind, at any given time some magicians are given more attention than others. Attention means that a magician is allowed to carry out transformations on other magicians.
In Webmind, the nodes with the highest heat, the greatest importance to the system, are given more CPU time
Observation 10: A mind always possesses some degree of randomness (i.e., dynamics which has no patterns that it can detect). Randomness occurs wherever attention occurs.
Wanderers use randomness in their motions. The dynamics of activation spreading will often be chaotic and will hence propagate this randomness widely.
Observation 11: The basic law of mind dynamics is: A magician passes some of its attention to other magicians with whom it shares meaning
This is activation spreading. The thresholding adopted from neural net models is simply a way of making attention spreading efficient.
Observation 12: Sharing of meaning may take many forms. Primally, meaning sharing may be of three different kinds:
symmetric
asymmetric
emergent.
Symmetric meaning sharing is encapsulated in "similarity links", asymmetric meaning sharing in "inheritance links" (member-category), emergent meaning sharing in multiple-target inheritance links.
Observation 13: Intelligence may be defined as the ability to achieve complex goals in complex environments
Webmind's goal of answering human questions meaningfully is complex, and the Net is plainly a complex environment
Observation 14: The complexity of an entity may be defined as the total amount of pattern in that entity, or equivalently, the amount of meaning in the entity. Thus, intelligence is the ability to achieve meaningful goals in meaningful environments.
Webmind has the ability to determine its own meanings, to understand human meanings, and to correlate the two.
Observation 15: In order to achieve complex goals in complex environments -- i.e., to be intelligent -- a complex mind is required
Webmind's mind is certainly complex!
Observation 16: A complex mind, implemented in a physical medium, will require continual modification of its internal parameters to assure steady intelligent functioning. This modification must be done intelligently in some cases, and so there must exist certain magicians with a special feedback relation to the physical medium determining the parameters of mental action.
Webmind has a homeostatic system built in, to regulate its parameters for optimum performance.
Observation 17: Pattern recognition in a complex environment is best done by a combination of perception, cognition (internal transformation of perceptions), and action
Webmind's perception is reading text and numbers, its cognition is most of its activity, and its action is providing answers, asking questions, and possibly carrying out electronic commercial transactions.
Observation 18: A substantial amount of a mind's attention must often be allocated to recognizing pattern in its environment, i.e. to this threefold "perceptual/cognitive/active loop."
This means that pods containing lower-level node types must be allocated significant CPU
Observation 19: A "magician system" is a collection of magicians that is self-producing, in the sense that any magician in the system can be produced by the combination of some other magicians in the system. Minds are magician systems, at least to within a high degree of approximation.
This is an experimental hypothesis that can be carried out on Webmind. If we remove a node corresponding to an abstract category -- a CategoryNode, a GroundedRelationNode, etc. -- will the system regenerate something similar on its own? The working hypothesis is YES.
Observation 20: Highly intelligent minds are characterized by hierarchical structures. The definition of hierarchy in this context is: A relates to {B1, B2,...,Bk} hierarchically if each of the B[i] asymmetrically shares much meaning with A. The process of creating hierarchical structure is called "clustering" or "categorization."
CategoryNodes provide hierarchical structure, as do inheritance links inferred from natural language.
Observation 21: Highly intelligent minds are characterized by heterarchical structures, large connected networks of symmetrically meaning sharing entities
Similarity links provide heterarchy.
Observation 22: In a highly intelligent system, the hierarchical and heterarchical structures of mind are aligned, so that in many cases, when A relates to {B[1],...,B[k]} hierarchically, each B[i] relates to a number of other B[i] symmetrically
This alignment is automatic in Webmind because categorial links are formed by inference from similarity links; and similarity links are then formed amongst category nodes.
Observation 23: Minds are finite, so that if they live long enough, they must forget. They will run into situations where they lose the B involved in a representation C(A)=B, but retain the pattern A that was recognized.
Webmind contains explicit node and link forgetting routines.
Observation 24: A mind's intelligence will be enhanced if it forgets strategically, i.e., forgets those items which are the least intense patterns
Each node and link has an "importance" value determining its forgettability.
Observation 25: A system which is creating new magicians, and then forgetting magicians based on relative uselessness, is evolving by natural selection. This evolution is the creative force opposing the conservative force of self-production.
This occurs implicitly in Webmind, and explicitly as well via node types that evolve other node types using the genetic algorithm (this occurs in the text-numerical cross-correlation module, for the evolution of relations between text and numerical data; and in the grounding module, for the evolution of automata representing patterns in experiential data).
Observation 26: A pattern A is "grounded" to the extent that the mind contains entities in which A is in fact a pattern
GroundedRelationNodes represent grounded patterns; many SenseNodes and CategoryNodes derived therefrom represent ungrounded patterns, to do with things like cats and showers that the system has no experience with.
Observation 27: "Reason" is a system of transformations specialized for producing incompletely grounded patterns from incompletely grounded patterns.
ReasonableNode (which CategoryNode extends) contains specific behaviors which look at the links coming out of that node and try to infer new links therefrom. This is Webmind's "reasoning system."
Observation 28: The specialization of a transformation may be defined as the variety of possible entities that it can act on. The magicians in a mind will have a spectrum of degrees of specialization, frequently with more specialized magicians residing lower in the hierarchy.
This is true within Webmind, there are highly specialized and highly general node types.
Observation 29: A mind must contain magicians specialized for mining emergent information recently obtained via perception. This is "short term memory." It must be strictly bounded in size to avoid combinatorial explosion; the number of combinations (possible grounds for emergence) of N items being exponential in N.
ShortTermMemoryNode accomplishes this, which is necessary among other reasons for semantic disambiguation of linguistic input.
Observation 30: The short-term memory may be used for tasks other than perceptual processing, wherever concentrated attention on all possible views of a small number of things is required
ReasonableNode uses short-term-memory to determine where to direct the course of reasoning next. A ReasonableNode maintains a "trail" of the things it has most recently reasoned about, which maps naturally into the contents of short-term memory. We call this a "train of thought."
Observation 31: Syntax is a collection of categories, and "syntactic transformations" mapping sequences of categories into categories. Parsing is the repeated application of syntactic transformations; language production is the reverse process, in which categories are progressively expanded into sequences of categories.
The Webmind NL system represents this specifically in terms of transformations taking pairs of category labels into individual category labels (and vice versa).
Observation 32: Semantics is a collection of categories, and "semantic transformations" mapping: categories into categories, category elements into category elements, transformations into categories, and semantic transformations into semantic transformations.
This is provided automatically by entry of words into Webmind as SenseNodes, and action of the CategoryNode categorization system on SenseNodes.
Observation 33: A key psychological role of syntax is to transfer semantic knowledge from strongly grounded patterns to weakly grounded or entirely ungrounded patterns.
The reasoning system automatically uses links from words corresponding to grounded concepts up to syntactic categories, and from syntactic categories down to words corresponding to ungrounded concepts, in order to build links between grounded and ungrounded concepts.
Observation 34: Language is useful for producing magicians specialized for social interaction. Syntax in particular is crucial for social interaction, because another intelligence's observations are in general ungrounded in one's own experience.
Only through the NL system can Webmind get a grip on human concepts that are outside its experiential domain.
Observation 35: The most intelligent minds have selves, where a "self" S is a pattern which a mind recognizes in the world, with the property that, according to the mind's reasoning, the substitution of S for the mind itself would produce few changes. I.e., the self asymmetrically shares meaning with the entire mind.
Webmind contains a Self node, which contains its image of itself in its own and others' eyes.
Observation 36: The "self" of a mind is a poorly grounded pattern in the mind's own past. In order to have a nontrivial self, a mind must possess, not only the capacity for reasoning, but a sophisticated reasoning-based tool (such as syntax) for transferring knowledge from strongly grounded to poorly grounded domains.
Reasoning automatically operates on the Self node as it does on all other nodes.
Observation 37: The existence of a society of similar minds makes the learning of self vastly easier
Webminds can communicate with one another, using the query language and more intimately using direct exchanges of nodes.
Having reviewed the general nature of the Webmind architecture, I will now outline its application to various particular problem areas, at a very coarse level of detail. In this section we will consider several broad application domains: text information retrieval, data mining and enterprise analysis, and financial decision support.
Textual Information Retrieval
An application area for whichWebmind has been domain-engineered to some extent is information retrieval from document databases. It is important in this context to distinguish intelligent information retrieval from "search engine" software. The notion of searching for documents based on topic descriptions is highly limiting. What is really interesting is having human minds relate with textual and other digital information in creative and productive ways. A final implementation of Webmind for information retrieval will incorporate search engine functionality, but will also involve visual exploration of information, and interactive processing whereby the software asks the user questions to clarify their intentions.
Here we will not go into these user interface matters in detail, but will rather give a cursory description of the internals of Webmind as applied to text information retrieval. This will serve to indicate how drastically different it is from its current competitors, and to illustrate the algorithmic roots of the different behavior that it displays. Each document being indexed is described by a TextNode, which contains links to various nodes representing the meaningful information in the text. There are also nodes corresponding to categories among documents. At the most abstract level, these categories might correspond to general topics such as "Business" or "Computing." At a level closer to the bottom, these might correspond to very particular groupings of pages, reflecting topics such as "artificial intelligence and finance." The process of category formation follows immediately from Webmind's self-organizing dynamics.
Each node on every level, when it is sufficiently stimulated, sends out a flock of intelligent agents, which wander the network of nodes in search of other nodes displaying similar or otherwise appropriately related patterns. When a query (be it a series of key words, a simple question, or a paragraph of natural language) is entered into the system, a node called a QueryNode is created for it. Then, the QueryNode is entered into the network much as if it were a new document to be read, classified and understood. First, using higher-level nodes to guide it, it finds a natural position in the network; then it sends out intelligent agents and these agents create new links between it and other nodes, other documents. Then activation spreads through the network, somewhat in the manner of a neural net, and after a certain period of time, the nodes with the highest activation are detected, and returned as the answers to the query.
What is fundamentally different about Webmind, as opposed to search engines, is that its document database is not a static entity, which sits there passively waiting for queries to set it into motion. Instead, it is constantly refining its structure -- and its structure is, like the structure of the human mind/brain, inseparable from its complex dynamics. The categories that it forms are not defined by mere statistical clustering, but by the outcome of self-organizing, reflexive dynamics. When a query is entered into Webmind, it sets off a chain of reactions that ultimately lead to it finding its natural places in the network, i.e., finding those groups of documents that pertain to the same topics as it does, as judged by the network's own complex dynamics.
In an ordinary search engine, a collection of pages is judged to have the "same topics" as each other if they have the same words in them. In a statistical search engine like Excite or SemioText, a collection of pages is judged to have a common topic if they have a mutually similar distribution of words and phrases. In Webmind, on the other hand, common topicality is not defined entirely in terms of structure but also in terms of process. A group of pages has a common topic if they have been judged as related by the self-organizing dynamics of the network. The distribution of words and phrases does play a role in this dynamics, as does the distribution of human-created links between phrases. But these factors only give the initial conditions for the dynamics. In the end, the distribution of links in Webmind is determined as much by the distribution of links in Webmind as by any specific identifiable factor. A query entered into Webmind is not only a request for information, but a perturbation of the complex dynamics of Webmind's constant study of itself.
It was noted above that Webmind can be used to form a more complete and complex version of the Yahoo category system. However, there are significant differences, which are conceptually relevant. The Yahoo category structure is very deep, for one thing; Webmind's structures tend to be somewhat flatter. Also, the Yahoo structure is strictly hierarchical, whereas, Webmind , like the human mind, is capable of operating with a patchwork of overlapping, complementary categorization schemes. In other words, the hierarchy of nodes can be dissociated, branching off in a number of different directions as it rises. Without Webmind's integrative dynamics, this could lead to confusion, but the dynamics are capable of integrating diverse points of view into a single coherent system -- in fact, this is what they do best. The different views within Webmind will not contradict but rather enhance each other. Ultimately, in a situation of extremely favorable computational resources, one can envision each user creating their own view of the Web, and allowing Webmind to crystallize a universal categorization system reflecting their individual perspective. The synthesis of all these individual views, according to Webmind dynamics, would be a kind of collective mentality -- not an average, but a complex, polyphonic blending of the different subjective world-views of the different members of the human race.
Webmind can use the same text-based interface as existing search engines. However, it also lends itself extremely naturally to visual, dynamic-image-directed search. Instead of returning lists of pages, a Webmind search engine can just as naturally return visual images of those portions of the Webmind network containing relevant pages. Different levels of magnification reveal lower and lower levels of the network, different colors represent complementary categorization schemes, etc. There are endless possibilities.
Next, regarding Webmind and the network on which it runs, it should be noted that, just as the general Webmind architecture is perfectly designed for network implementation, so is the Webmind search engine in particular. Today, it is assumed that a search engine database is a single, monolithic mirror of the Web. Already, however, some firms are breaking with this model; for example, FunnelWeb has constructed a network of search engines, one for each continent. When the Australian FunnelWeb engine wants information about the European FunnelWeb engine, it calls it up and asks it. This kind of networked search, on the level of passing queries from one engine to another, is relatively primitive and unexciting. However, Webmind opens up a whole new spectrum of possibilities.
Imagine, as the simplest case, that one set up two separate Webmind databases corresponding to .com and .edu domain names. Each one would have its own integrity, its own hierarchical/associative structure. But there could be links from one database to the other. And if agents from one database were allowed to wander through the other, then the two databases would be linked together on the level of dynamic structure: they would be two entrained, complex dynamical systems. The closeness of the link between the two systems would be determined by the propensity of the agents to leave their home database.
Imagine, next, that instead of two separate Webmind databases, one had, say, ten million. Imagine, in other words, that every website was its own Webmind database, and that, in order to do a search, one simply sent out intelligent agents through the Web. This is the opposite extreme from the one-search-engine/one-database model, and it is not yet feasible given current bandwidth limitations. But a search engine distributed across hundreds of sites is feasible today, and given the projected rapid growth of network bandwidth, the millions-of-sites scenario may well be feasible in ten years time. Once it is feasible, one will have something remarkable indeed: one will have the possibility for a World Wide Web which is, structurally and dynamically, a mirror of the human mind. Instead of a World Wide Web, we will genuinely have a World Wide Brain, a global network intelligence.
Data Mining
The term "data mining" refers to the process of searching through a large volume of data to discover interesting and useful information. Text information retrieval, as discussed above, may be considered a special case of data mining, in which the data involved is purely textual. Much of human intelligence is "data mining" -- we are concerned with searching through a vast amount of perceptual information, regarding the outside world and our own brains as well, and finding the interesting and useful information obscured wherein. In practice, though, the term "data mining" is generally used to refer to the elicitation of patterns from numerical data such as financial time series, or data stored in databases such as customer information; and it is these applications that will be the focus in the following comments. Most applications labeled "data mining" involve prediction of future trends, or categorization of data collected over an interval of time.
Data mining answers business questions that traditionally were too time-consuming to resolve, finding predictive or categorical information that experts may miss, either because it is too subtle and complicated, or because it lies outside their psychological expectations. It provides answers to questions such as, "Which clients are most likely to respond to my next promotional mailing, and why?" and "Which direction is the price of the Deutschmark likely to move over the next twenty-four hours?" and "What kind of person is hitting my Website?"
What differentiates Webmind from other data mining packages is, first of all, its ability to ferret out subtle patterns in data. It is able to pick up regularities that other, less sophisticated algorithms miss. Just as neural networks and genetic algorithms go beyond linear statistics, Webmind goes beyond neural networks and genetic algorithms. Furthermore, Webmind's sophisticated Internet implementation allows it to deal with large databases in a highly efficient way. Unlike other data analysis algorithms, Webmind is intrinsically designed to deal with tremendously large amounts of information, stored across distributed networks. As demonstrated in the financial experiments reported above, it is also able to analyze textual and numerical information in a unified way -- a fact of the utmost importance given the vast amounts of conceptually interrelated textual and numerical data on the contemporary business computer network.
On a more conceptual level, what differentiates Webmind is the fact that it approaches a body of data in roughly the same way that the human mind approaches the outside world. It takes the body of data into itself, just as perception brings the world into the mind, and it "brings the data to life," enabling the data to transform itself and study itself. The core, low-level pattern recognition routines used by Webmind are not necessarily different from those used by other data mining systems. The difference lies in the way the low-level recognized patterns are combined to form an holistic understanding of the entire body of data. This holistic understanding is embodied in Webmind's link structure, after a period of evolution, and it results in Webmind being able to recognize extremely subtle and sophisticated patterns in its input data, in an efficient way.
In practical terms, the application of Webmind to data mining is fairly straightforward. Data is read into Webmind and forms itself into a population of nodes; at which point the Webmind dynamics goes to work, bringing out the structure implicit in the data. Questions posed to the system are then resolved in terms of the information that Webmind has already discovered, on its own. If one's information is stored on an intranet, then absolutely no pre-processing is required, as Webmind is optimized to process distributed intranet data. If one's information is stored in a conventional database, or some other format, this is not a serious problem either, as the addition of simple additional software modules allows Webmind's to speak SQL and other relevant database languages.
Webmind carries out the two basic tasks of data mining -- prediction and categorization -- in a flexible and unified way. Categorization is taken care of by link update methods, as described above. Categories among, say, customers of a business, or stocks in the market, are found in the same way that the Webmind search engine isolates categories among Web pages. As for prediction, this is essentially a matter of recognizing patterns in one data set (representing, e.g., the past history of a financial market), and then extrapolating these patterns to another, perhaps unknown data set (e.g. the future of a financial market). The recognition of patterns is the crucial part. Neural networks and genetic algorithms pick up patterns that statistics and decision trees miss; and Webmind, with its self-organizing hierarchical/associative structure and its ability to read text and internally relate text with data, picks up patterns that neural networks and genetic algorithms miss.
As a single example of the application of Webmind to commercial data analysis, outside the finance industry, consider the problem of "targeted marketing." A business maintains a list of its customers over a period of time, which contains a fair amount of information about each customer. By studying this list, one can determine the qualities that typify a "good customer" for this business. The business then obtains a large database of names, e.g. census data, and wishes to determine which people from among this large database are likely to be good customers.
The specifics of this problem will vary from case to case, depending on the nature of the database. But the basic principles underlying the application of Webmind to this type of problem are quite general and flexible. Most simply, one may begin by building a Webmind database consisting of the elements of the large database. In the case of census data, each Webmind node might correspond to a single block, or even a single household. One then sets Webmind to work, building informative links between the nodes. Different links specify different types of relationship. For instance, there will be "geographical links," which link two nodes and are weighted by the distance between the blocks represented by the nodes (two nodes will be linked by a geographical link only if they are relatively close together). There may be links tied to particular parameters, e.g. "age structure" links or "income distribution" links. The different types of links are not chosen haphazardly, but are chosen based on the outcome of pattern recognition operations within the nodes. Each type of link is constructed by swarms of agents equipped with an appropriate criterion function, an appropriate measure of similarity between nodes. Higher level nodes are then formed, representing clusters amongst nodes -- natural groupings of "mutually similar" nodes. Finally, the elements of the smaller database are then entered into this network like queries into a text search oriented network. They find their most natural place in the network, extend links, and are activated. The most active nodes are then the "similar" nodes within the larger database.
This differs from typical statistical or AI approaches to market analysis in many essential respects. In terms of performance, the biggest difference is Webmind's ability to carry out highly targeted market analysis. Existing algorithms are effective at weeding out terrible customers from the large database, but only mediocre as isolating the best customers. Webmind, on the other hand, has a deeper understanding of the structure of the large database, because it begins by allowing the structure of the large database to discover itself. This deeper understanding allows it to target precisely those customers who are most similar to the business's best customers -- just as, in the Webmind search engine, a query finds those Web pages that are the most similar to it.
Similar approaches have been worked out for diverse other data mining applications -- no business intelligence problem lies outside its domain. The architecture encompasses the most effective aspects of previous approaches -- such as statistical clustering, neural networks, and genetic algorithms -- into an overarching framework. It integrates the two main objectives of data mining, categorization (carried out by higher-level nodes) and prediction (carried out by pattern-recognition methods within nodes), in a seamless way. As with Web search, it is not a panacea: there is still a certain amount of work required to tailor the prediction and link update methods to fit the nature of particular application domains. And searches for extremely subtle patterns in huge data warehouses can still take time, even with Webmind's sophisticated internetworking capabilities. But Webmind represents a significant step beyond what is out there; and it has the flexibility and reflexive adaptability to take advantage of improvements in Internet technology and data mining algorithms, without straying from its basic framework, which is, after all, nothing else but the structure of mind.
Knowledge Management and Knowledge Creation
"Information retrieval" and "data mining" are concepts that were created to describe the behavior of software much more limited in its behavioral repertoire than Webmind. Webmind, with its ability to understand context and to ground concepts in its own experience, also has the ability to do much more than any mere information-extraction tool. It is able to serve as a true knowledge management and knowledge creation system, integrating information from the document archives and databases of interest to a business, and allowing exploration of this information in all its facets, each piece considered in the context of the whole.
If Webmind is installed on a company's intranet, then real-time queries regarding relationships between textual, numerical and other data to do with the enterprise may be posed by any employee with computer access at any time. The result is that Webmind's intelligence is integrated with the social intelligence of the organization, and the individual intelligence of the employees. Each time an employee accesses a document, he may ask Webmind for related documents, and Webmind will carry out this task with an understanding of the role of that employee in the company, that employee's particular needs and interests, etc. It will be able to make its own creative suggestions, based on its autonomous thought processes.
Furthermore, the social dynamics of the different Webmind servers residing in different parts of the company's intranet (governed by Webmind's highly innovative, proprietary server-server communication methods) will grow to reflect the social dynamics of the individuals using those parts of the intranet. Each Webmind server will respond most effectively and rapidly to queries involving information which it stores locally; but the information that a certain server stores locally may change over time, depending on user needs and internal Psynet dynamics. Thus, while providing easy access by all users to all information at all times, Webmind will nevertheless nudge the information at the readiest disposal of individual humans and divisions in certain directions, based on its inferences and its own emergent understanding. Webmind will do more than just provide an understanding of structures and processes; it will be a participant in processes, in the formation of emergent human and informational structures.
And, as various Webmind units in various organizations exchange non-proprietary information, in the interest of increased mutual intelligence, Webmind will be a participant in the formation of human and informational structures on the global scale. This is an exciting new vision of artificial intelligence, in the business context and beyond -- not AI as something separate from humanity, providing us with answers to our questions, but AI as something interacting symbiotically with humanity, participating in our communications, goals, and social structures and processes.
Enterprise Analysis, Workflow and Business Process Reengineering
Using its extremely flexible structures and dynamics, Webmind can easily be trained to perform a variety of interesting "enterprise analysis" functions. For example, Webmind will be able to:
produce pictures of the amount of information passing between different people and groups
make causal inferences: "Because Bob finishes his balance sheets on time, Joe is usually able to give meaningful information to the production department."
isolate the most important concepts being passed between a particular pair of people or groups
track the evolution of concepts through written communications
discover the tightly connected clusters of people or groups, among which large amounts of information commonly passes
discover the boundaries in the network of people or groups, which information, or particular concepts, have difficulty passing through
produce a map of the clusters of people in an organization, separated by boundaries, and the amount and type of linkage between the different clusters
Collectively, these functions amount to making a map of an organization as a self-organizing system, and identifying the emergent structures in the organization. These functions will play an important role in enabling organizations to become more effective learning systems.
Achieving these enterprise capabilities will not involve adding any new core structures to Webmind, but only creating new types of query nodes, and tuning the categorization, visualization and concept-evolution processes to work on the type and volume of data found in enterprise applications.
Financial Decision Support
As I write these words, a Webmind is running in the Intelligenesis office, producing daily judgement as to which of four major markets are going to go up or down the next day. Each day, it downloads the news from the Net, extracts the important concepts from the news, and uses these, combined with various purely numerical strategies, to make its predictions. Currently the system is doing prediction on a daily scale, not on a second-by-second scale, but this is purely for simplicity's sake; by the time you read these words the principles underlying our analyses of daily financial data will have been extended throughout the financial domain.
The system reads the news, not in a musing kind of way, but with a particular financial data set in mind, say the Dow Jones. It comes up with concepts with the following property: The relevance of the concept to the news on a given day tells you something about what the market's going to do on the next day (or, 2 weeks later, etc.). The financial meaning of the text is thus boiled down to a collection of numbers – one for each concept extracted, representing the relevance of that concept to the text on a certain day. The numbers corresponding to the concepts can be computed anew every day, and used for financial analysis purposes just like numbers coming from any other source. These numbers, representing the relevancies of text-derived concepts to the news at a particular time, are what we call text indicators.
The extraction of text indicators is the crux of Webmind's financial intelligence. It relies on Webmind's ability to intelligently judge relevance, which draws on all of Webmind's abilities at reasoning, language understanding, conceptualization, and so forth. But text indicator extraction is not the end of the story. In addition to the extraction of financially relevant concepts from news, there is an additional process of learning optimal trading models for particular financial markets. Just knowing the news concepts that tend to correlate with a certain financial market (say, knowing that trouble in foreign countries tends to drive the Dow) doesn't tell you enough to make accurate predictions. You have to get at the nonlinear interrelations between news concepts and numerical patterns in the data. This has to be done differently for the Dow, for IBM stock, for the Yen, for 30 year bonds, and so forth. For each market, Webmind derives a trading model that embodies the best way of incorporating text based information into decisions about that particular market.
Webmind's trading models are what computer scientists call "boolean automata," simple logical decision rules, just as Webmind uses for making any kind of decision. They're basically the same kind of rules that are used, within the natural language system, to decide which sense of "Java" is intended in a sentence; or that are used more generally for learning of abstract concepts and categories. As an initial test of the system, we asked Webmind to learn optimal trading models for five major markets, first with and then without news-derived information. When it included the information derived from news archives, the performance increased tremendously, often by a factor of 3 or more. Financial analysis has traditionally been based on fundamental analysis (looking at qualitative, economic and political factors), technical analysis (number-crunching; extrapolation based on past numerical values of behavior within the market), or, more recently, psychological analysis (exploiting commonalities in behavior of the traders who actually take market actions). The approach here contains elements of all of these, since it seeks out and exploits both causal and correlative relationships from many sources within each of the three perspectives. Beyond this, it clearly represents a new, fourth approach, in its use of text resources for integrated decision-making. By linking vast reservoirs of numerical- and text-based data sets in a sophisticated state-of-the-art intelligent learning system, capacities for financial decision-making are greatly enhanced.
These results are also of fairly obvious importance outside the financial arena, because the dynamics underlying Webmind 's performance on these sample tasks are the same ones underlying its more general performance. In addition to demonstrating that Webmind can produce highly profitable trading systems using textual information, these tests also show something more general: that Webmind can meaningfully extract concepts from large amounts of text data, relate these concepts to trends in numerical data, and use the relationships it has inferred to provide highly significant actionable information. This capability is of tremendous value beyond the world of finance, in areas such as risk assessment, demand planning, supply chain management, enterprise management, document retrieval and analysis, primary market analysis, medical diagnosis and research -- the list could be continued almost endlessly.
Wide implementation of this kind of technology will have a globally dramatic effect. The financial component of the World Wide Brain will become vastly more intelligent than it is right now. I would hate to see the World Wide Brain develop with an overly financial bias – the finance industry, for all its breadth, does after all represent a fairly narrowly biased view of the human race and all the information it has to offer. But the financial markets are already a globally integrated, perceiving and acting self-organizing system, and so they are a natural place for Internet intelligence to start.
The precise mechanisms underlying Webmind's text-based market prediction are, obviously, proprietary (and are also patent-pending). But the basic character of the process is nothing but Webmind intelligence, applied to one particular domain. Recall the basics of Webmind architecture. Webmind, internally, consists of a collection of software objects called "nodes," each of which contains links to other nodes, representing inter-node relationships. Some nodes contain raw data such as text or numerical time series; others are more abstract and consist entirely of links to other nodes. The design is superficially similar to a neural network, but is fundamentally different in several respects. A Webmind node is more like a "neuronal module" in the brain than it is like a single neuron. In Webmind , unlike in a neural network, link construction is carried out by a variety of intelligent software agents, and nodes and links are frequently created and destroyed as part of the learning process. Webmind 's internal intelligent agents use a variety of techniques such as genetic algorithms and statistical language processing. Different types of nodes are used for representing different types of data. The node types most directly relevant to simple financial applications are:
DataNode, which refers to a numerical data sets, e.g. financial time series
TextNode, which refers to a text document, e.g. a market news report for a particular day
TimeSet, which refers to a series of time-indexed nodes, e.g. a series of market news reports over several years
ConceptNode, which refers to a concept either extracted from text or learned by Webmind
TradingRuleNode, a particular kind of finite automaton representing a financial trading rule
The process of learning textual indicators to aid in market prediction, trading and analysis is a natural outgrowth of Webmind 's intelligent self-organizing dynamics. Consider, for simplicity, the case where the market being trading is the Dow Jones Industrial Average, and the text being used is market news which is issued on a daily basis. In this case the central data structures are a DataNode pointing to the DJIA, and a TimeSet pointing to the series of market news articles. The text indicator learning process is carried out by an agent associated with these two nodes. The goal of this agent is to isolate a ConceptNode, representing one of Webmind 's internal ideas, with the property that the relevance of the ConceptNode to the TimeSet, on a given day, is a useful indicator for trading the data in the DJIA DataNode on that day. Having carried out this learning process, it then uses the ConceptNode it has isolated to create a new "text indicator" DataNode, each entry of which indicates the relevance of the ConceptNode to the market news TimeSet on a certain day. This text indicator DataNode can then be supplied to the user as, quite simply, a series of numbers. Each day, on reading the news, Webmind can supply a new value for the text indicator -- just as, on seeing the DJIA itself, it can supply new values for numerical indicators such as moving averages and the like.
The text indicator obtained by this process of internal conceptualization can be used for a variety of different purposes. It can be used by humans to form their own intuitive judgements, or incorporated in any computational trading framework that is sufficiently flexible to incorporate arbitrary numerical indicators. Most of our work so far, however, has involved incorporating text indicators into Webmind 's own intelligent trading processes, which are ideally suited for incorporating text indicator information. A TradingRuleNode, in Webmind , encapsulates a trading rule, which is a logical combination of information deriving from a numerical predictor and a number of indicators, some numerical and some textual. These details, however, have more to do with computational finance than Internet intelligence, so I will omit them here. The point is that Webmind's intelligent text analysis feeds into a Webmind subsystem that carries out integrative numerical-textual financial prediction, according to methods that are original but build on standard practices in financial analysis.
Finally, what exactly are these "concepts" that Webmind derives, by reading the news with financial data in mind? Within Webmind, they are represented as abstract data structures. They are associated with various words and documents and numerical data sets, but they are not truly described in any way except as abstract Webmind-internal data structures. Just as, while we can try to describe our intuitions about the world in words, we can never quite capture them. A good boxer, or financial trader, or mathematician can try to explain the intuitive ideas that guide her work, but the articulation never quite matches the reality. For the period 1996-1997, for instance, the most prominent concepts driving the Dow and the Yen had to do with trouble in Asian economies, and Asian banks particularly. "Asian bank trouble" as a focus of the news was highly predictive of downturns in the Dow. One useful concept had to do with the relation between the British government and the Asian financial situation. Another had to do with European unification. But the important thing to remember is that the concept that Webmind found to be related to the market wasn't just "European unification" itself – it was, rather, a certain slant on European unification, a certain sense it got from the news, that had to do with European unification.
These results are fascinating, and are sure to get more so as Webmind becomes more sophisticated, as all the different modules of the code become integrated with each other (as of now, natural language and reasoning are not used "full-strength" in the finance applications; and the self and social aspects of Webmind are not used at all). Ultimately, they will only be a small part of Webmind's utility. However, they will never become trivial: whatever else it may be, the future of the Internet is going to be commercial, which means that the World Wide Brain will have to be financially intelligent!