Experiential Interactive Learning with Novamente:
Further Plans and Comments
Ben Goertzel
August 1, 2004
The Novamente AI project has had the concept of “experiential interactive learning” (EIL) at its heart since the very start. However, due to the nature of the Novamente design, we have not yet done any actual work with EIL and Novamente. This is because for Novamente EIL to be truly successful, a fairly sophisticated learning and memory framework needs to already be in place within Novamente. (Humans, thanks to evolution, are born with such a framework; in an artificial intelligence, it must be engineered.)
The Novamente project began in mid-2001. At this point, the design is roughly 50% implemented. The figure was about 30% in September 2003; the pace of implementation has substantially increased since then due to an increase in the staff devoted to the project. While the design will need to be 100% complete for EIL to take place in a serious and fully powerful way, we are now only a few man-months of engineering away from the point where initial EIL experiments can begin. If all goes well we will take the first steps in EIL during late 2004.
This document reviews some of my current thoughts about EIL in Novamente. These ideas will likely change as Novamente develops further. The document consists of some loosely organized notes, rather than a coherent body of information.
This document should be read in the context of the 2003 document “Virtually Embodied AI: Embedding Novamente in a Simulated World,” which is available online at
http://www.goertzel.org/dynapsyc/2003/NovamenteSimulations.htm
The ideas in this document are extensions of the ones given there; and that document goes into detail on the cognitive-science and cognitive-engineering aspects of EIL, which are not treated here. Reading this document without the context given in that previous document is bound to lead to some serious misconceptions. However, some relevant things have changed since that previous document was written; for instance:
Most of the ideas in this document represent a fleshing-out of the EIL concept based on these recent developments in Novamente engineering.
2. Goals of
Novamente EIL
Novamente EIL (NEIL for short) has two primary goals
These two goals are interrelated. English language understanding is a critical part of our planned approach to teaching Novamente to modify its source-code. On the other hand, learning to understand and produce English descriptions of complex procedures will be an important part of Novamente’s English-language-learning process.
Our recent work on the programming-language side of Novamente (hinted at above, and described a little more fully in the following section) has made Goal 2 seem significantly more palpable and near-term than it did a year ago, though certainly there’s still a lot of work to be done.
3. Some Relevant
Tools
The current NEIL plans relies significantly on three software tools, which are closely related to, and interlinked with, the Novamente AI Engine but are not strictly parts of it. The first two of these were specifically created for Novamente, the third (the Java supercompiler) is the product of an entirely separate project (although applications to Webmind, Novamente’s predecessor, were already under discussion at the time of inception of that project):
4. The NEIL
Framework
This section presents my current vision of the NEIL framework. I will not address Novamente-internal issues here, but only Novamente-external issues such as simulation worlds and user interfaces. Of course, the Novamente-internal issues are the really interesting ones, but they would require a much longer and more technical document; and they are largely covered in the “Virtually Embodied AI” document mentioned above, though that document’s not entirely up to date.
The basic components of the NEIL interface should be:
4.1 Comments
on AGI-SIM
Regarding the simulation world, there are several open questions, such as:
My current feeling is that richness is very important, and that richness will be significantly easier to obtain in a simulation world that is relatively accurate physics-wise. By “richness” what I mean is:
There is certainly no fundamental reason why these cognitive requirements can only be met by a simulation world that is realistic physics-wise. However, making a simulation world with realistic physics and filling it with lots of real-world data, is one fairly sure way to meet both of these requirements.
Another key point is the requirement for social interaction in the simulation world. Animal psychologists, in order to teach parrots language, have used a strategy in which there are two trainers interacting with the parrot – one playing the role of a teacher, the other playing the role of a student. The student basically acts like a smart parrot, doing what the parrot is intended to do. The trainers occasionally swap roles, so that no one individual is too strongly associated with the teacher or student role. This allows the parrot to learn how to be a student, and to learn how to be a teacher as well. A similar approach will probably be useful with Novamente: two human teachers should be used, one playing the role of a teacher and one playing the role of a Novamente-imitating student.
Another interesting & important point to consider the notion of “Friendly AI” – it’s worth asking what we can do, even at this early stage in Novamente teaching, to encourage Novamente to have a positive psychological attitude toward humans, itself and the universe in general. (For my general thoughts on this sort of issue, see my essay “Encouraging a Positive Transcension”, at
http://www.goertzel.org/dynapsyc/2004/PositiveTranscension.htm
One simple strategy that can be used to encourage Novamente Friendliness is to make most of Novamente’s initial tasks social in nature. That is, one doesn’t give the system tasks that it must complete on its own in order to ensure its survival; rather, one gives it tasks involving helping other agents (humans or other Novamentes) to achieve their goals. It thus learns cooperation as part of its process of learning to learn and think. Another point is that Novamente should be allowed to control different avatars in the same simulation world, and to participate in several different simulation worlds. Also, different Novamente instances should be allowed to swap internal “mind matter”, and to tap directly into one another’s perceptions and “feelings” – this will build a psychology of empathy rather than individuality.
4.2 Comments
on the Programming-Language Interface
The overall purpose of the program-editor window in the envisioned NEIL UI is:
In order to support these goals, some special kind of comment notation should probably be introduced, in order to support English descriptions of programming language constructs in a simple way. For instance, one might introduce a special comment notation ##, used as in the example:
## Calculates j!, commonly known as “j factorial” or “the
factorial of j”
int fact(int j) {
## Sets the value of the variable n equal to 1
n = 1
## Makes the variable i take on the integer values
1-9.
## Equivalently, makes the variable i loop from 1 to j+1,
with an increment of 1
for(i=1;i<j+1;i++)
## For each value of i visited by the above loop,
multiply n by i
n = n*i
## Return the value of n to the function that called this
function
return n
}
The convention is that the comment following a ## denotes a simple English description of the line of code immediately following.
The above example is a simple program manipulating numbers, but a more interesting case is where the program describes a series of actions in the simulation world. For example, the following program encapsulates the behavior of fetching a visible object satisfying a certain specified criterion. Of course, this is a very oversimplistic program, much simpler than the programs that will actually control agent behavior. It is given here simply as a brief and concrete example of the thorough English-language annotation of programs in a simulation world context.
## Causes agent to obtain the nearest visible object satisfying the predicate f
void obtainNearbyObject(Predicate f) {
## Store the agent’s current location in the variable Q
Q = getCoordinates()
## Call the function findClosestVisibleRedObject to find
the object
## satisfying f that is closest and is in the line of
sight
Coordinates P = findClosestVisibleObject(Predicate f)
## Move the agent to the location with coordinates P
MoveTo(P)
## Pick up the object at location P
PickUpObject()
## Move back to location Q
MoveTo(Q)
}
This program-editor window will be used in several different ways, namely:
In all cases, the human and Novamente may then talk about the program and its progress in the conversation window.
5. Initial
Examples of Teaching Novababy
This section discusses a couple examples of Novababy-instruction, which may be interesting to play with once we have the NEIL framework set up as described above.
5.1 Teaching
groundings for prepositions and spatiotemporal concepts
This will require a Novamente-controlled agent and two human-controlled agents (which may however be controlled by the same human, using different aliases), the Teacher and the Student. Basically, when any of the agents does anything, the Teacher agent should describe it in English, e.g.
“Novababy walked toward the wall.”
“Student ran toward the door”
“I spun around.”
Multiple redundant descriptions of the same situation will often be useful.
Then, the Teacher should ask the Student questions, such as
“Where did I walk?”
“What did you just walk toward?”
“What did Novababy just walk toward?”
The Student should answer appropriately (thus giving Novababy a model of what appropriate answers look like).
Through this process, a large data store of information will be built up regarding questions and appropriate answers, and situations and appropriate descriptions. These will provide very valuable data for Novamente to mine in its internal cognition process. My belief is that this sort of process will be by far the best way for Novamente to learn the proper use of prepositions, temporal and spatial words, and so forth.
The use of INLINK to disambiguate human-entered natural language, will ensure that Novamente doesn’t make any syntactic or semantic mistakes in understanding English, which means it will have a “clean” database of linguistic/simulation-world relationships to learn from.
5.2 Teaching
Simple Procedures
The goal here will be to teach Novababy how to carry out simple tasks, such as “Go into the other room and bring back the first red object you find.”
One approach here is purely evolutionary: Novababy can try out different procedures, and get feedback on how successful each one has been. If this feedback is provided by an automated “fitness function” this can be a fairly efficient way of learning. Also, the learning process can use inference based on procedures used for prior, related tasks (i.e. it’s not pure “evolutionary learning”).
Another approach is for the human teacher to give Novababy verbal guidelines, e.g.
First, find a doorway that leads to another room.
Then, go through that doorway.
Then, look around until you find a red object.
Then, go to that red object and pick it up.
Then, return to your original location.
The task of Novababy is then to interpret these verbal instructions properly, and devise a procedure corresponding to each instruction. In some cases, these procedures will be there already; in other cases they’ll have to be inferred or learned through experimentation.
A third approach is for the human teacher to give Novababy explicit programs telling it how to carry out the task. Or, the teacher may give Novababy explicit programs telling it how to carry out part of the task, and leave the rest up to learning. For example, the teacher may give a verbal breakdown of the task into 5 steps, but only give an explicit program corresponding to one of the steps.
When the teacher gives an explicit program, Novamente may then be asked to explain the steps of the program in English. The teacher may then correct Novamente’s explanations. The verbal knowledge about the steps of the program may be helpful to Novamente when it needs to use the program as data to draw inferences about how to carry out other tasks.
When Novamente conceives a program for solving the task, the teacher may look at the program explicitly (in this case the supercompiler may be very useful insofar as it can be used as a tool for program simplification), and make comments on the program, or make suggested modifications to the program. The teacher may also suggest an alternate program, after Novamente has already conceived its own.