Lojbanic AI and the Chaotic Committee of Sub-Bens

In 1998 when my dear departed AI software company Intelligenesis Corp. was just getting started, we had a summer "intern" by the name of Mark Shoulson, who was (if memory serves) a grad student at Rutgers University. Mark worked for us for a few summer months then went back to grad school. Although he was extremely bright with broad interests in computing and cognitive science, his work that summer focused on some technical issues in the computational linguistics portions of our software; he didn't really get into the deeper aspects of the AI theory Intelligenesis was pursuing. Informally, many of us referred to Mark as "The Klingon" because of his mastery of the Klingon language. (For the non-nerds in the audience: Yeah, when they created the Klingons in Star Trek, they hired a linguist to design an actual language for them. Cool, huh?) Mark was involved in the translation of Hamlet into Klingon and didn't mind showing off his Klingon fluency to curious colleagues. Mark's Klingon was smooth and fluent but often seemed a bit odd because of his kind and soft-spoken nature -- personality-wise, at least on the surface, Mark was pretty far from a Klingon-ish guy. He also told us about a friend of his who raised his daughter bilingual English-Klingon: speaking to his daughter only in Klingon from birth, while his wife spoke to her in English.

Along the way Mark also mentioned to me a language called Lojban, which he said was based on predicate logic. He observed to me in passing that it might be easier for us to make our AI system understand Lojban than English. I agreed that it might be, if Lojban was more logically structured, but I reckoned this wasn't very practical, since no one on the team except Mark spoke any Lojban. Also, we were interested in creating a real AI incrementally, along a path that involved spinning off commercial apps -- and the commercial applications of a Lojban-speaking AI system seemed rather few.

Well, six and a half years later, Mark's suggestion has started to seem like a pretty good one. In my new AI project Novamente, we have progressed moderately far along the path of computational language understanding. Our progress toward powerful general AI has been painfully slow due to the team's need to pay rent and the lack of any funding oriented toward the grand AI goal, but for 2004 and part of 2003 the Novamente team and I had some funding to build some English language processing software -- and while we didn't build anything profoundly real-AI-ish, we used the opportunity to explore the issues involved in AI language processing in some depth.

The language processing system that we built is called INLINK and is described here. It doesn't understand English that well by itself, but it interacts with a human user, presenting alternate interpretations of each sentence typed into it, until the human verifies it's found a correct interpretation. The interactive process is slow and sometimes irritating but it ultimately works,
allowing English sentences to be properly interpreted by the AI system. We have plans to create a version of the INLINK system called BioCurator, aimed at biological knowledge entry -- this should allow the construction of a novel biology database containing formal-logic expressions representing biological knowledge of a much subtler nature than exists in current online bio resources like the Gene Ontology.

I've had a lot of doubts about the value of computational linguistics research for "real AI" -- there's a moderately strong argument that it's better to focus on perception, action and embodiment, and let the AI learn language as it goes along interacting with humans using its (real or simulated) body. On the other hand, there's also an argument that a certain degree of "cheating" may be helpful -- that building in some linguistic knowledge and facility may be able to accelerate the experiential-language-learning process. I've outlined this argument in an article called Post-Embodied AI.

The work on INLINK has clarified for me exactly what's involved in having an AI system understand English (or any other natural language). Syntax processing is tricky but the problems with it can be circumvented using an interactive methodology as we've done in INLINK; and eventually the system can learn from its errors (based on repeated corrections by human users) and make fewer and fewer mistakes. The result of INLINK is that English sentences are translated into probabilistic logical expressions inside Novamente's memory, which may then be interpreted, reasoned on, data-mined, intercombined, and yadda yadda yadda. Very nice -- but nasty issues of computational efficiency arise.

Novamente's probabilistic-inference module currently exists only in prototype form, but the prototype has proven capable of carrying out commonsense reasoning pretty well on a number of simple test problems. But there's a catch: for the reasoning process to be computationally tractable, the knowledge has to be fed to the reasoning module in a reasonably simple format. For instance, the knowledge that Ben likes the Dead Kennedys, has to be represented by a relationship something like

#likes( #Ben_Goertzel, #Dead_Kennedys)

where the notation #X refers to a node inside Novamente that is linked by a high-strength link to the WordNode/PhraseNode representing the string X. Unfortunately, if one types the sentence

"Ben likes the Dead Kennedys"

into INLINK, the Novamente nodes and links that come out are more complicated and numerous and less elegant. So a process called "semantic transformation" has to be carried out. This particular case is simple enough that this process is unproblematic for the current Novamente version. But for more complex sentences, the process is, well, more complex, and the business of building semantic transformations becomes highly annoying. One runs into severe issues with the fuzziness and multiplicity of preposition and verb-argument relationships, for example. As occurs so many times in linguistics and AI, one winds up generating a whole bunch of rules which don't quite cover every situation -- and one realizes that in order to get true completeness, so many complexly interlocking small rules are needed that explicitly encoding them is bound to fail, and an experiential learning approach is the only answer.

And this is where -- as I just recently realized -- Lojban should come in! Mark Shoulson was right back in 1998, but I didn't want to see it (urrrgghh!! what useful things are smart people saying to me now that I'm not accepting simply because I'm wrapped up in my own approaches?? why it is to hard to truly keep an open mind?? why is my information processing capacity so small??!! wait a minute -- ok -- this is just the familiar complaint that the limitations of the human brain are what make it so damn hard to build a superior brain. And the familiar observation that cutting-edge research has a way of making the researcher feel REALLY REALLY STUPID. People tell me I'm super-smart but while working on AI every day I come to feel like quite a bloody moron. I only feel smart when I re-enter the everyday world and interact with other people ;-p)

What if instead of making INLINK for English, we made it for Lojban (LojLink!)? Of course this doesn't solve all the problems -- Lojban is a constructed language based on formal logic, but it's not equivalent to formal logic; it allows ambiguity where the speaker explicitly wants it, otherwise it would be un-usable in practice. Semantic transformation rules would still be necessary to make an AI system understand Lojban. But the human work required to encode such transformations -- and the AI learning required to learn such transformations -- would clearly be one or two orders of magnitude less for Lojban.

Lojban isn't perfect... in my study of Lojban over the last week I've run up against the expected large number of things I would have designed differently, if I were building the language. But I have decided to resist the urge to create my own Lojban-ish language for AI purposes, out of respect for the several decades of work that have gone into "tuning" Lojban to make it more usable than the original version was.

In some respects Lojban is based on similar design decisions to the knowledge representation inside my Novamente AI Engine. For instance, in both cases knowledge can be represented precisely and logically, or else it can be represented loosely and associatively, leaving precise interpretation reliant on contextual factors. In Lojban loose associations are represented by constructs called "tanru" whereas in Novamente they're represented by explicit constructs called AssociativeLinks, or by emergent associations between activity-patterns in the dynamic knowledge network.

Next, it's worth noting that Lojban was created and has been developed with a number of different goals in mind -- my own goal, easier interfacing between humans and early-stage AGI's, being just one of them.

Some Lojbanists are interested in having a "culturally neutral" language -- a goal which, while interesting, means fairly little to me.

In fact I don't really believe it's possible -- IMO Lojban is far from culturally neutral, it embodies its own culture, a nerdy and pedantic sort of culture which has plusses and minuses. There is a Lojban term "malglico" which translates roughly to "damn English" or "fucking English" -- it refers to the tendency to use Lojban in English-like ways. This is annoying to Lojban purists but really doesn't matter to me. What I care about is being able to communicate in a way that is fluid and simple and natural for me, and easy for an early-stage AI to comprehend. If the best way to achieve this is through a malglico dialect of Lojban, so be it. If malglico interferes with the comprehensibility of Lojban by AI software, however, then I'm opposed to it.

I've printed up a bunch of materials on Lojban and started studying it seriously -- if I keep up with it then in 6 months or so I'll be a decent Lojbanist. Generally I'm not much good at learning languages, but that's mostly because it bores me so much (I prefer learning things with more of a deep intrinsic structure -- languages always strike me as long lists of arbitrary decisions, and my mind wanders to more interesting things when "I" try to force it to study them...). But in this case I have a special motivation to help me overcome the boredom....

If you want to try to learn Lojban yourself, the most useful resources I've found are:

• This set of Lojban lectures
• This lengthy, complete exposition of Lojban grammar
• This mathematical formalization of Lojban grammar
• This Lojban-English dictionary (take the file dict.ps and convert it to .pdf)
• This English-Lojban dictionary
  

If it does happen that we teach Novamente to speak Lojban before English then in order to participate in its "AI preschool" you'll need to know Lojban! Of course once it gets beyond the preschool level it will be able to generalize from its initial language to any language. But the preschool level is my focus at the moment -- since as I'm intensely aware, we haven't gotten there yet!

I remain convinced that with 2-3 years of concentrated single-focused effort by myself and a handful of Novamente experts (which will probably only be possible if we get some pure-AI-focused funding, alas), we can create a Novamente system with the intelligence and creativity and self-understanding of a human preschooler. But I'm trying really hard to simplify every aspect of my plan in this regard, just to be sure that no unexpected time-sinks come along. One advantage of NOT having had pure-AI-focused funding for the last few years is that the AI design has been refined an awful lot during this frustrating period. The decision to take a "post-embodied" approach to linguistics -- incorporating both experiential learning and hard-wiring of linguistic knoweldge -- is not a new one; that was the plan with Webmind, back in the day. But the idea of doing initial linguistic instruction and hard-wiring for Novamente in Lojban rather than English is a new one and currently strikes me as quite a good one.

Ah -- there's a bit of a catch, but not a big one. In order to do any serious "hard-wiring" of Lojban understanding into Novamente or any other AI system, the existing computational linguistics resources for Lojban need to be beefed up a bit. I describe exactly what needs to be done here. It seems to me there's maybe 3/4 man-years of work in making pure Lojbanic resources, and another year of work in making resources to aid in automated Lojban-English translation.

And another interesting related point. While in 1998 when Mark Shoulson first pointed Lojban out to me, I thought there were no practical commercial applications for a Lojban-based AI system, I've now changed my mind. It seems to me that an AI system with a functional Lojban language comprehension module and modest level of inferential ability would actually be quite valuable in the area of knowledge management. If a group of individuals were trained in Lojban, they could enter precise knowledge into a computer system very rapidly, and this knowledge could then be reasoned on using Novamente or other tools. This knowledge base could then be queried and summarized in English -- because processing simple English queries using a system like INLINK isn't very hard, and doing crude Lojban-English translation for results reporting isn't that hard either. In any application where some institution has a LOT of knowledge to encode and several years to do it, it may actually make sense to take a Lojbanic approach rather than a more standard approach. Here you'll find an overview of this approach to knowledge management, which I call LojLink.

One example where this sort of approach to knowledge encoding could make sense is bioscience -- I've long thought that it would be good to have every PubMed abstract entered into a huge database of bio knowledge, where it could then be reasoned on and connected with online experimental biology data. But AI language comprehension tools aren't really up to this task -- all they can do now is fairly simplistic "information extraction." We plan to use a bio-customized version of INLINK to get around this problem, but entering knowledge using INLINK's interactive interface is always going to be a bit of a pain. There's enough biology out there, and the rate of increase of bio knowledge is fast enough, that it makes sense to train a crew of bio knowledge encoders in Lojban, so that the store of bio knowledge can be gotten into computer-comprehensible form at maximum rate and minimum cost. Yes, I realize this sounds really weird and would be a hard idea to sell to venture capitalists or pharma company executives -- but that doesn't mean it doesn't make sense....

As another aside, there is some Lojban poetry on the Net but I haven't found much Lojban music. I like to sing & play the keyboard sometimes (in terms of vocal style, think Bob Dylan meets Radiohead); I'm considering doing some of my future lyrics in Lojban! True, few listeners would understand what I was talking about -- but I reckon that, in many cases, the verbal contents of lyrics aren't all that important -- what's important is the genuineness of feeling attached to them, which is achievable if the words have deep meaning to the singer, whether or not the listener can understand them. Of course, I have some lyrics that violate this rule and succeed at least a bit in communicating poetically (even a bit of transhumanist lyricism here and there -- e.g. "I've got to tell you something / your lonely story made me cry / I wish we all could breathe forever / God damn the Universal Mind"). But even so I think Lojbanic lyrics could really rock....

But -- wow -- how to fit learning a new language into my schedule? Urgggh!! Way too much to do. Fortunately I have a wife who says she's willing to learn this weird language along with me, which will make things much easier; it'd be trickier to learn a language with no one to speak to. But still ... every time something new like this comes up I'm confronted with the multiplicity of Bens in my head: each with different goals and priority rankings on their shared goals ... some of them saying "Yeah! You've got to do this!", others cautioning that it will siphon away the sometimes irritatingly small amount of time currently allocated to enjoying the non-intellectual aspects of human life in the Ben-iverse....

But "I" digress. Or do I?

Perhaps internal multiplicity and the falsehood of the unified "I" is a topic best saved for another blog entry. But yet, it does tie back into Lojban -- which I notice contains a single word for "I" just like ordinary languages. This is an area where I'm tempted to introduce new Lojbanic vocabulary.

I don't know what "I" am. I like the Walt Whitman quote "I contradict myself? Very well then, I contradict myself. I am large, I contain multitudes." Indeed, I do. In From Complexity to Creativity I explored the notion of subselves extensively. This notion should be explicitly embodied in language. You should be able to say "One of my subselves wants X" rather than "I want X" -- easily, via a brief linguistic expression, rather than a complicated multi-phrasal description. The distinction between "Some of my subselves want this very intensely" and "All of my subselves want this moderately strongly" should be compactly and immediately sayable. If these things were compactly and simply expressible in language, maybe we'd get out of the habit of thinking of ourselves as unities when we're really not. At least, I'm definitely not. (Just like I feel idiotic most of the time, then feel more clever when interacting with others; similarly, when I'm on my own I often feel like a population of sub-Bens with loosely affiliated goals and desires, and then I feel more unified when interacting with others, both because others view me as a whole, and because compared to other peoples' subselves, mine all cluster together fairly tightly in spite of their differences... (and then I'm most unified of all when I let all the goals drift away and dissolve, and exist as a single non-self, basking in the 1=0, at which point humanity and transhumanity and language and all that seem no more important than un ... but now I really digress!)). And in an appropriately designed language -- say, a subself-savvy extension of Lojban -- this paragraph would be a lot shorter and simpler and sound much less silly.

And this brings up a potentially very interesting aspect of the idea of teaching AI systems in odd constructed languages. My main motivation for thinking about using Lojban instead of English to teach Novamente is to simplify the semantic mapping process. But, it's also the case that English -- like all other natural languages -- embodies a lot of really irritating illusions ... the illusion of the unified self being one of them. Lojban now also happens to embody the illusion of the unified self, but this is a lot easier to fix in Lojban than in English, because of the simpler and more flexible structure of the Lojban language. I don't buy the strongest versions of the Sapir-Whorf hypothesis (though I think everyone should read Whorf's essay-collection Language, Thought and Reality), but clearly it's true that language guides cognition to a significant extent, and this can be expected to be true of AI's at least as much as of humans.

I can envision a series of extensions to Lojban being made, with the specific objective of encouraging AI systems learn to think according to desired patterns. Avoidance of illusions regarding self is one issue among many. Two areas where Lojban definitely exceeds English are ethics and emotion. English tends to be very confused in these regards -- look at the unnecessary ambiguities of the words "happy" and "good", for example. The current Lojban vocabulary doesn't entirely overcome these problems, but it does so significantly, and could be improved in these regards with modest effort.

Well, as I type these words, my son Zeb is sitting next to me playing "Final Fantasy" (yes my work-desk sits in the livingroom next to the TV, which is mostly used by the kids for videogames, except for their obsessive viewing of South Park... the new season just started, there are new episodes now, and did you know Mr. Garrison is now Mrs. Garrison??!!). As I look over at the manly-chested (scrawny, artistic little 11-year-old Zeb's favorite utterance these days: "Admire my manly chest or go down trying!"), womanly-faced heroes run through their somewhat bleak simulated landscape, and feel really intensely sick of the repetitive background music, I can't help but observe that, obsessed as he is with that game, I'm even more obsessed with my own "final fantasy." Or am I? One of my "I"'s is. Another one feels a lot more like playing the piano for an hour or so before bed, even though clearly working on AI for that time-interval would be more productive in terms of the long-term good of Ben and the cosmos. Or is playing music a while justified by the mental peace it brings, enabling clearer thinking about AI research later? How to ensure against self-delusion in judgments like that? Ah, by 38 years old I have devised an excellent set of mental tools for guarding against delusion of one subself by itself, or of one subself by others -- and these tools are frustratingly hard to describe in the English language! No worries -- the community of sub-Bens remains reasonably harmonious, though in the manner of strange attractors rather than fixed psychological arrangements. The chaos goes on.... (ranji kalsa ... ) ... the human chaos goes on, moving inevitably toward its own self-annihilation or self-transcendence ... and my committee of sub-Bens unanimously agrees that it's worth spending a lot of thought and effort to bias the odds toward the latter ...