The massive suckage of writing academic research papers / the ontology of time / White Sands
I was a professor for 8 years, so I'm no stranger to the weird ways of academia. But I've been pretty much away from that universe for a while, pursuing commercial software development and independent research. Recently I've re-initiated contact with the world of academic research, because it's become clear that getting some current academic publications on my AI and bioinformatics work will be valuable to my scientific and business pursuits. Egads!! The old frustrations are coming back -- badly enough to spill over into a blog entry....
This is a pretty boring blog entry, I'm afraid: just a long rant about how annoying academic research can be. But I got irritated enough to write this stuff down, so I guess I may as well post it....
I've been working on an academic paper together with my former Webmind colleague Pei Wang, on the topic of "why inference theories should represent truth values using two numbers rather than one." For instance, the inference component of my Novamente AI system represents the truth values of statements using a probability and a "weight of evidence" (which measures, roughly, the number of observations on which the probability is based). Pei's NARS reasoning system uses two-component truth values with a slightly different interpretation.
Now, this is a perfectly decent paper we've written (it was just today submitted for publication), but, what strikes me is how much pomp, circumstance and apparatus academia requires in order to frame even a very small and simple point. References to everything in the literature ever said on any vaguely related topic, detailed comparisons of your work to whatever it is the average journal referee is likely to find important -- blah, blah, blah, blah, blah.... A point that I would more naturally get across in five pages of clear and simple text winds up being a thirty page paper!
I'm writing some books describing the Novamente AI system -- one of them, 600 pages of text, was just submitted to a publisher. The other two, about 300 and 200 pages respectively, should be submitted later this year. Writing these books took a really long time but they are only semi-technical books, and they don't follow all the rules of academic writing -- for instance, the whole 600 page book has a reference list no longer than I've seen on many 50-page academic papers, which is because I only referenced the works I actually used in writing the book, rather than every relevant book or paper ever written. I estimate that to turn these books into academic papers would require me to write about 60 papers. To sculpt a paper out of text from the book would probably take me 2-7 days of writing work, depending on the particular case. So it would be at least a full year of work, probably two full years of work, to write publishable academic papers on the material in these books!
For another example, this week I've been reading a book called "The Ontology of Time" by L. Nathan Oaklander. It's a pretty interesting book, in terms of the contents, but the mode of discourse is that of academic philosophy, which is very frustrating to me. It's a far cry from Nietzsche or Schopenhauer style prose -- academic philosophy takes "pedantic" to new heights.... The book makes some good points: it discusses the debate between philosophers promoting the "A-theory of time" (which holds that time passes) and the "B-theory of time" (which holds that there are only discrete moments, and that the passage of time is an illusion). Oaklander advocates the B-theory of time, and spends a lot of space defending the B-theory against arguments by A-theorists that are based on linguistic usage: A-theorists point out that we use a lot of language that implies time passes, in fact this assumption is embedded in the tense system of most human languages. Oaklander argues that, although it's convenient to make the false assumption that time passes for communicative purposes, nevertheless if one is willing to spend a lot of time and effort, one can reduce any statement about time passing to a large set of statements about individual events at individual moments.
Now, clearly, Oaklander is right on this point, and in fact my Novamente AI design implicitly assumes the B-theory of time, by storing temporal information in terms of discrete moments and relations of simultaneity and precedence between them, and grounding linguistic statements about time in terms of relationships between events occurring at particular moments (which may be concrete moments or moments represented by quantified mathematical variables).
There are also deep connections between the B-theory and Buddhist metaphysics, which holds that time is an illusion and only moments exist, woven together into apparent continua by the illusion-generating faculty of the mind. And of course there are connections with quantum physics: Julian Barbour in "The End of Time" has argued ably that in modern physics there is no room for the notion of time passing. All moments simply exist, possessing a reality that in a sense is truly timeless -- but we see only certain moments, and we feel time moving in a certain direction, because of the way we are physically and psychologically constructed.
But Oaklander doesn't get to the connections with Buddhism and quantum theory, because he spends all his time pedantically arguing for fairly simple conceptual points with amazing amounts of detail. The papers in the book go back 20 years, and recount ongoing petty arguments between himself and his fellow B-theorists on the one hand, and the A-theorists on the other hand. Like I said, it's not that no progress has been made -- I think Oaklander's views on time are basically right. What irritates me is the painfully rate of progress at which these very smart philosophers have proceeded. I attribute their slow rate of progress not to any cognitive deficits on their part, but to the culture and methodology of modern academia.
Obviously, Nietzsche would be an outcast in modern academia -- casting his books in the form of journal papers would really be a heck of a task!
And what if the scientists involved in the Manhattan Project had been forced to write up their incremental progress every step of the way, and fight with journal referees and comb the literature for references? There's no way they would have made the massively rapid progress they did....
And the problem is not restricted to philosophy, of course -- "hard" science has its own issues. In computer science most research results are published at least twice: once in a conference proceedings and once in a journal article. What a waste of the researcher's time, to write the same shit up twice ... but if you don't do it, your status will suffer and you'll lose your research grants, because others will have more publications than you!
Furthermore, if as a computer scientist you develop a new algorithm intended to solve real problems that you have identified as important for some purpose (say, AI), you will probably have trouble publishing this algorithm unless you spend time comparing it to other algorithms in terms of its performance on very easy "toy problems" that other researchers have used in their papers. Never mind if the performance of an algorithm on toy problems bears no resemblance to its performance on real problems. Solving a unique problem that no one has thought of before is much less impressive to academic referees than getting a 2% better solution to some standard "toy problem." As a result, the whole computer science literature (and the academic AI literature in particular) is full of algorithms that are entirely useless except for their good performance on the simple "toy" test problems that are popular with journal referees....
Research universities are supposed to be our society's way of devoting resources to advancing knowledge. But they are locked into a methodology that makes knowledge advance awfully damn slowly....
And so, those of us who want to advance knowledge rapidly are stuck in a bind. Either generate new knowledge quickly and don't bother to ram it through the publication mill ... or, generate new knowledge at the rate that's acceptable in academia, and spend half your time wording things politically and looking up references and doing comparative analyses rather than doing truly productive creative research. Obviously, the former approach is a lot more fun -- but it shuts you out from getting government research grants. The only way to get government research money is to move really slowly -- or else to start out with a lot of money so you can hire people to do all the paper-writing and testing-on-toy-problems for you....
Arrrgh! Anyway, I'm compromising, and wasting some of my time writing a small fragment of my research up for academic journal publication, just to be sure that Novamente AI is "taken seriously" (or as seriously as a grand AGI project can possibly be taken by the conservative-minded world we live in).... What a pain.
If society valued AGI as much as it valued nuclear weapons during World War II, we'd probably have superhuman AI already. I'm serious. Instead, those of us concerned with creating AGI have to waste our time carrying out meaningless acts like writing academic papers describing information already adequately described in semi-formal documents, just to be taken seriously enough to ask for research money and have a nonzero chance of getting it. Arrggh!
OK, I promise, the next blog entry won't be as boring as this, and won't be a complaint, either. I've actually been enjoying myself a lot lately -- Izabela and I had a great vacation to New Mexico, where we did a lot of hiking, including the very steep and very beautiful Chimney Canyon route down Mount Sandia, which I'd always wanted to do when I lived in New Mexico, but never gotten around to. Also, we camped out on the dunes in White Sands National Monument, which is perhaps the most beautiful physical location I know of. I can't think of anywhere more hallucinogenic -- psychedelic drugs would definitely enhance the experience, but even without them, the landscape is surprisingly trippy, giving the sensation of being in a completely different universe from the regular one, and blurring the distinction between inside and out....
Most of the time wandering around in White Sands was spent in conversation about the subtleties of the interrelationship between free will and consciousness -- interesting and perhaps valuable ideas that I haven't found time to write down yet, because all my writing-time these last couple weeks has been spent putting already-well-understood ideas into the form of academic papers ;-ppp White Sands is exactly the right place to mull over the structure of your mind, since the landscape itself projects you involuntarily into a kind of semi-meditative state....
Hmmm... maybe I'll write down those ideas about free will and consciousness in the next blog entry. It's tempting to write that stuff now -- but it's 1:25 AM, I think I'll go to sleep instead. Tomorrow, alas, is another day... (I tried to make all the days run into each other by taking Modafinil to eliminate my need for sleep -- but it just wound up upsetting my stomach too much, so I've had to go back to sleeping again: bummer!!)
This is a pretty boring blog entry, I'm afraid: just a long rant about how annoying academic research can be. But I got irritated enough to write this stuff down, so I guess I may as well post it....
I've been working on an academic paper together with my former Webmind colleague Pei Wang, on the topic of "why inference theories should represent truth values using two numbers rather than one." For instance, the inference component of my Novamente AI system represents the truth values of statements using a probability and a "weight of evidence" (which measures, roughly, the number of observations on which the probability is based). Pei's NARS reasoning system uses two-component truth values with a slightly different interpretation.
Now, this is a perfectly decent paper we've written (it was just today submitted for publication), but, what strikes me is how much pomp, circumstance and apparatus academia requires in order to frame even a very small and simple point. References to everything in the literature ever said on any vaguely related topic, detailed comparisons of your work to whatever it is the average journal referee is likely to find important -- blah, blah, blah, blah, blah.... A point that I would more naturally get across in five pages of clear and simple text winds up being a thirty page paper!
I'm writing some books describing the Novamente AI system -- one of them, 600 pages of text, was just submitted to a publisher. The other two, about 300 and 200 pages respectively, should be submitted later this year. Writing these books took a really long time but they are only semi-technical books, and they don't follow all the rules of academic writing -- for instance, the whole 600 page book has a reference list no longer than I've seen on many 50-page academic papers, which is because I only referenced the works I actually used in writing the book, rather than every relevant book or paper ever written. I estimate that to turn these books into academic papers would require me to write about 60 papers. To sculpt a paper out of text from the book would probably take me 2-7 days of writing work, depending on the particular case. So it would be at least a full year of work, probably two full years of work, to write publishable academic papers on the material in these books!
For another example, this week I've been reading a book called "The Ontology of Time" by L. Nathan Oaklander. It's a pretty interesting book, in terms of the contents, but the mode of discourse is that of academic philosophy, which is very frustrating to me. It's a far cry from Nietzsche or Schopenhauer style prose -- academic philosophy takes "pedantic" to new heights.... The book makes some good points: it discusses the debate between philosophers promoting the "A-theory of time" (which holds that time passes) and the "B-theory of time" (which holds that there are only discrete moments, and that the passage of time is an illusion). Oaklander advocates the B-theory of time, and spends a lot of space defending the B-theory against arguments by A-theorists that are based on linguistic usage: A-theorists point out that we use a lot of language that implies time passes, in fact this assumption is embedded in the tense system of most human languages. Oaklander argues that, although it's convenient to make the false assumption that time passes for communicative purposes, nevertheless if one is willing to spend a lot of time and effort, one can reduce any statement about time passing to a large set of statements about individual events at individual moments.
Now, clearly, Oaklander is right on this point, and in fact my Novamente AI design implicitly assumes the B-theory of time, by storing temporal information in terms of discrete moments and relations of simultaneity and precedence between them, and grounding linguistic statements about time in terms of relationships between events occurring at particular moments (which may be concrete moments or moments represented by quantified mathematical variables).
There are also deep connections between the B-theory and Buddhist metaphysics, which holds that time is an illusion and only moments exist, woven together into apparent continua by the illusion-generating faculty of the mind. And of course there are connections with quantum physics: Julian Barbour in "The End of Time" has argued ably that in modern physics there is no room for the notion of time passing. All moments simply exist, possessing a reality that in a sense is truly timeless -- but we see only certain moments, and we feel time moving in a certain direction, because of the way we are physically and psychologically constructed.
But Oaklander doesn't get to the connections with Buddhism and quantum theory, because he spends all his time pedantically arguing for fairly simple conceptual points with amazing amounts of detail. The papers in the book go back 20 years, and recount ongoing petty arguments between himself and his fellow B-theorists on the one hand, and the A-theorists on the other hand. Like I said, it's not that no progress has been made -- I think Oaklander's views on time are basically right. What irritates me is the painfully rate of progress at which these very smart philosophers have proceeded. I attribute their slow rate of progress not to any cognitive deficits on their part, but to the culture and methodology of modern academia.
Obviously, Nietzsche would be an outcast in modern academia -- casting his books in the form of journal papers would really be a heck of a task!
And what if the scientists involved in the Manhattan Project had been forced to write up their incremental progress every step of the way, and fight with journal referees and comb the literature for references? There's no way they would have made the massively rapid progress they did....
And the problem is not restricted to philosophy, of course -- "hard" science has its own issues. In computer science most research results are published at least twice: once in a conference proceedings and once in a journal article. What a waste of the researcher's time, to write the same shit up twice ... but if you don't do it, your status will suffer and you'll lose your research grants, because others will have more publications than you!
Furthermore, if as a computer scientist you develop a new algorithm intended to solve real problems that you have identified as important for some purpose (say, AI), you will probably have trouble publishing this algorithm unless you spend time comparing it to other algorithms in terms of its performance on very easy "toy problems" that other researchers have used in their papers. Never mind if the performance of an algorithm on toy problems bears no resemblance to its performance on real problems. Solving a unique problem that no one has thought of before is much less impressive to academic referees than getting a 2% better solution to some standard "toy problem." As a result, the whole computer science literature (and the academic AI literature in particular) is full of algorithms that are entirely useless except for their good performance on the simple "toy" test problems that are popular with journal referees....
Research universities are supposed to be our society's way of devoting resources to advancing knowledge. But they are locked into a methodology that makes knowledge advance awfully damn slowly....
And so, those of us who want to advance knowledge rapidly are stuck in a bind. Either generate new knowledge quickly and don't bother to ram it through the publication mill ... or, generate new knowledge at the rate that's acceptable in academia, and spend half your time wording things politically and looking up references and doing comparative analyses rather than doing truly productive creative research. Obviously, the former approach is a lot more fun -- but it shuts you out from getting government research grants. The only way to get government research money is to move really slowly -- or else to start out with a lot of money so you can hire people to do all the paper-writing and testing-on-toy-problems for you....
Arrrgh! Anyway, I'm compromising, and wasting some of my time writing a small fragment of my research up for academic journal publication, just to be sure that Novamente AI is "taken seriously" (or as seriously as a grand AGI project can possibly be taken by the conservative-minded world we live in).... What a pain.
If society valued AGI as much as it valued nuclear weapons during World War II, we'd probably have superhuman AI already. I'm serious. Instead, those of us concerned with creating AGI have to waste our time carrying out meaningless acts like writing academic papers describing information already adequately described in semi-formal documents, just to be taken seriously enough to ask for research money and have a nonzero chance of getting it. Arrggh!
OK, I promise, the next blog entry won't be as boring as this, and won't be a complaint, either. I've actually been enjoying myself a lot lately -- Izabela and I had a great vacation to New Mexico, where we did a lot of hiking, including the very steep and very beautiful Chimney Canyon route down Mount Sandia, which I'd always wanted to do when I lived in New Mexico, but never gotten around to. Also, we camped out on the dunes in White Sands National Monument, which is perhaps the most beautiful physical location I know of. I can't think of anywhere more hallucinogenic -- psychedelic drugs would definitely enhance the experience, but even without them, the landscape is surprisingly trippy, giving the sensation of being in a completely different universe from the regular one, and blurring the distinction between inside and out....
Most of the time wandering around in White Sands was spent in conversation about the subtleties of the interrelationship between free will and consciousness -- interesting and perhaps valuable ideas that I haven't found time to write down yet, because all my writing-time these last couple weeks has been spent putting already-well-understood ideas into the form of academic papers ;-ppp White Sands is exactly the right place to mull over the structure of your mind, since the landscape itself projects you involuntarily into a kind of semi-meditative state....
Hmmm... maybe I'll write down those ideas about free will and consciousness in the next blog entry. It's tempting to write that stuff now -- but it's 1:25 AM, I think I'll go to sleep instead. Tomorrow, alas, is another day... (I tried to make all the days run into each other by taking Modafinil to eliminate my need for sleep -- but it just wound up upsetting my stomach too much, so I've had to go back to sleeping again: bummer!!)
posted by Ben at 1:18 AM
9 Comments:
Two things: First of all, I think you pretty much summed up why I dropped out of grad school. I took literally one course: AI 401. I had started grad school at the University of Oregon thinking "Wow here's a school with a great AI lab. I'm sure I'll be surrounded by some of the best." Uh. Nope. Let's just say the disappointment of the reality of academic progress hit me very, very hard. I got an A, but in getting it I lost all respect for the academic view of "writing software".
Regarding Modafinil: Are you one of those lucky few who have managed to get a prescription for it, or did you get it the way I did...Hello, Internet. Mighty expensive stuff, though.
Ah Ben...
As regards your ideas on time it might pay you to re-read your next post on different but equally valid view-points.
I'm now inclined to think that A-Theory and the B-Theory of time are just two different but equally valid perspectives.
I read about Modafinil, am tempted. Perhaps I can bluff my MD into writing me a scrip.
- Tom Buckner, posting anonymously just because I can't be bothered to log in right now.
PS I hope you don't mind too much if I take some of those gripes about publishing and use them in a fiction piece....? They're an interesting glimpse into the culture of those who actually work on AI.
The "toy problems" are one of my pet hates too. I'm not an academic, but I've read a lot of academic papers and year after year these things keep rearing their heads. Optomising an algorithm on a toy problem often ensures that there is no way that it would work on a larger scale in real life situation, mainly because there are too many simplifying (i.e. wrong) assumptions within the model.
I often use the term "cheating" to apply to these sorts of models. With my own robotics and vision experiments I've always quite deliberately resisted the urge to cheat. If I structured the environment in some special way so as to make it trivially easy for the robot to see certain objects, using certain hard-coded colours or textures, I could get the system to *appear* to do some interesting things. But that's not what the real world is like. I think we need AI algorithms which can cope with complex every-day scenarios. The kind of complexity and ambiguity of the real world is really what we need intelligence for, to allow us to generate interpretations.
Well, I've got at least another 1 and a half years of academia. Maybe by that time I'll get reasonably decent at writing papers and can start throwing some out about Novamente. ;)
I'm pretty lucky in regards to modafinil. My stomach feels a little weird, but not really upset or bad. It actually makes me want to eat more - which could be modafinil maintaining my metabolism at a higher level. I still haven't used it to forego sleep completely, just for long days and when I don't get a solid nights rest.
I just started my PhD, so maybe I'm taking all this too much from an ignorant point of view, but wasn't the whole peer-review methodology to ensure two things:
a) New knowledge is indeed new and not just new to a particular group of people.
b) Resources are spent on new things rather then re-inventions of the same thing.
Again maybe I'm just too inexperienced in this regard. I could see how the algorithmic performance pow-how can be regarded more as a marketing thing for your algorithm then "hard-science". But could your really make a case against the peer-review system on this short-coming alone?
Ben
mortgage articles for refinance mortgage home loans
A
Modafinil? The Erdos route works better and is a lot more fun :-)
Alcibiades
Post a Comment
<< Home