People are asking me today about the importance of Google DeepMind’s new video-game AI demo — see e.g. the Guardian article glowingly titled
First off: Yeah, it’s certainly cool!!!
This is progress beyond DeepMind’s former Atari 2600 game demo, and I don’t want to pooh-pooh the amount of work and brilliance that goes into making something like this ACTUALLY WORK !! But this doesn’t especially feel to me like some sort of breakthrough, just good solid work in deep reinforcement learning… In particular, it doesn’t convince me that deep RL as DeepMind has habitually practiced it is an adequate approach to AGI….
The key issue is that these video games are very constrained domains, so that a deep reinforcement learning system can handle them without being able to interpret data in a broader context. This kind of constrained domain can obviously be handled by methods that wouldn’t work for broader, messier, more contextuality-rich real-world domains….
So to me “learning independently in a very constrained world” is not entirely the same problem as “learning independently in a big rich messy world” — the former can be done by a system that ignores a lot of stuff any real-world AI has to pay a lot of attention to…. The universe of simple video games they’ve dealt with in these fascinating experiments, are in the end still a very small and special domain for which a software system can be heavily tuned in various ways (i.e. even if the tuning is independent of which specific game is being learned, it can still be particular to the domain of “games of this type”).
Personally I think DeepMind would need to add a lot of extra stuff to their AGI approach as I understand it (at least, a lot of stuff beyond anything they’ve discussed publicly, and beyond anything hinted at by this demo), to approach AGI seriously. I think the kinds of deep learning and deep reinforcement learning they’ve focused on are only a moderate-sized fraction of the story of what’s needed for human-level cognition. To me, this demo doesn’t show that they have embraced the complexity and multi-aspect nature of what’s needed for human-level AGI. (Maybe they HAVE, behind the scenes, but this video doesn’t show it….)
Another way to put my point is: To really hit a home run their system will need to be able to learn to deal with new DOMAINS, not just new games within a domain for which their system has already been tuned and tweaked and human-configured….
One thing this demo does show is that, since their acquisition by Google, DeepMind is pushing ahead with AGI-oriented R&D according to their own tastes and directions, rather than simply applying their brainpower to Google’s search and ads products and so forth. That’s very nice to see 😉
38 comments
3 pings
Alejandro says:
February 26, 2015 at 3:38 am (UTC 0)
I think what we are seeing is mainly Volodymyr Mnih’s personal project continuing, not DeepMind’s or Google’s strategy into AI. There are very few differences between the architecture and algorithm used in this paper and the previous one that they published seven months before this paper got submitted (bigger, deeper network and delayed updating of Q evaluator, that’s it). Note also how in their recent DRAW paper (http://arxiv.org/abs/1502.04623) they could have used a reinforcement learning paradigm but they chose to go the supervised learning route instead, and they haven’t used reinforcement learning for anything that didn’t involve Mnih as far as I can tell.
Regarding it not dealing with different domains, it of course depends on what you mean by a domain. Even within one single game, there are parts that are different enough that I would call them different domains. On the other extreme, if you want domains to mean something completely general, you’ve got the No Free Lunch theorem telling you you can’t do that (“That’s not a new domain, you are still playing in a Kolmogorov-complexity defined universe”). In between you’ve got humans which can deal with some types of domain adaptations. I understand that you are saying that the commonality between the different environments it does learn in are too big. Your phrasing seems to indicate that you see a much harder line in what constitutes a domain than I can.
Tattoo says:
June 20, 2015 at 1:19 pm (UTC 0)
It’s actually a great and useful piece of info. I’m happy that you simply shared this useful info with us. Please stay us up to date like this. Thanks for sharing.
gamis syar'i online says:
June 30, 2015 at 2:27 am (UTC 0)
Hi friends, its great article regarding teachingand entirely explained,
keep it up all the time.
Halina says:
July 14, 2015 at 6:39 pm (UTC 0)
Empiece sesión si ya dispone de una cuenta como lector registrado, cree una cuenta nueva en un solo
paso.
Stevie says:
July 18, 2015 at 1:12 am (UTC 0)
Thank you for any other wonderful article. The place else may just anybody get that kind
of information in such an ideal manner of writing? I’ve a presentation subsequent week,
and I’m on the search for such information.
joanns coupons says:
August 3, 2015 at 3:41 am (UTC 0)
This piece of writing gives clear idea in favor of the
new users of blogging, that actually how to do running a blog.
Harga Samsung Galaxy Prime SM-G530H says:
August 5, 2015 at 7:34 pm (UTC 0)
Hello, after reading this amazing paragraph i am too happy to share my know-how here
with mates.
aqiqah says:
September 29, 2015 at 10:10 pm (UTC 0)
Hello it’s me, I am also visiting this web page on a regular
basis, this web site is in fact nice and the users are truly sharing fastidious thoughts.
harga ac panasonic 2015 says:
November 3, 2015 at 3:32 am (UTC 0)
Hi there everyone, it’s my first go to see at this web page, and article is truly fruitful in favor of me, keep
up posting such content.
cara mengecilkan perut setelah melahirkan says:
November 10, 2015 at 8:01 am (UTC 0)
Hello to every , as I am truly keen of reading this web site’s post to be
updated daily. It includes nice data.
Jasa Fumigasi says:
November 11, 2015 at 8:50 pm (UTC 0)
Thanks for any other excellent article. The place else
could anyone get that type of information in such a perfect manner of writing?
I have a presentation next week, and I’m on the search for such info.
kurs pajak hari ini says:
November 21, 2015 at 8:48 pm (UTC 0)
I go to see daily a few sites and information sites to read content,
but this website gives quality based posts.
informasi menarik says:
December 10, 2015 at 1:44 am (UTC 0)
I am actually glad to glance at this web site posts which carries tons of helpful facts, thanks for providing
such statistics.
orang terkaya di indonesia says:
December 15, 2015 at 9:13 pm (UTC 0)
Thanks , I’ve just been searching for info about this subject for a long time and yours is the best I have came upon till now.
But, what about the bottom line? Are you certain in regards to the source?
Master cara da foto é bom says:
December 18, 2015 at 6:52 am (UTC 0)
Thanks for finally writing about > Google DeepMinds new Video Game AI goertzel.org < Liked it!
ps says:
December 23, 2015 at 5:44 pm (UTC 0)
We absolutely love your blog and find the majority of
your post’s to be just what I’m looking
for. Does one offer guest writers to write content to suit your needs?
I wouldn’t mind publishing a post or elaborating on some of the subjects you write concerning here.
Again, awesome weblog!
jual mebel jepara says:
December 29, 2015 at 4:46 am (UTC 0)
Quality posts is the main to be a focus for the people
to pay a quick visit the web site, that’s what this web page
is providing.
berita basket says:
January 14, 2016 at 9:39 am (UTC 0)
Fine way of describing, and good paragraph to obtain information on the topic of my presentation focus, which i am going to
present in institution of higher education.
hostmara says:
February 10, 2016 at 9:06 am (UTC 0)
What’s up, constantly i used to check website posts
here early in the daylight, as i enjoy to find out more and more.
ring bearer pillow says:
February 15, 2016 at 6:30 pm (UTC 0)
Marvelous, what a weblog it is! This web site presents helpful data to us, keep it up.
ishihara test says:
February 18, 2016 at 12:54 am (UTC 0)
What’s up to every one, it’s really a fastidious for me to pay a quick visit this site, it contains priceless
Information.
undangan pernikahan says:
February 18, 2016 at 2:14 am (UTC 0)
What i don’t realize is in fact how you’re no longer really much more neatly-preferred than you may be now.
You’re very intelligent. You realize thus significantly in terms of this topic, produced me in my view believe it from numerous various angles.
Its like women and men aren’t involved unless it’s something to accomplish with Girl gaga!
Your own stuffs excellent. All the time maintain it
up!
thymosin says:
March 13, 2016 at 8:58 am (UTC 0)
Acne scars are even more tʏpісal
compaгеԀ tօ yοu cߋᥙⅼԀ ⲣіcturе, ɑs
աᥱlⅼ ɑѕ anyߋne
tһаt ѕtrugglеѕ աіth tɦᥱm աіlⅼ tᥱⅼl
ʏоᥙ, ᥱlіmіnatіng thеm ііѕ еen mоге tߋuǥһ cօmpaгеɗ tօ
іt іѕ tο оbtaіn tҺеm.
Ӎʏ ρsցe thymosin
oral hsv tongue says:
April 16, 2016 at 3:26 pm (UTC 0)
Moved by touch, there is always the danger of capturing the herpes virus when taking part in sex.
Mabel says:
April 16, 2016 at 4:50 pm (UTC 0)
Remember likewise that the spread of herpes is not restricted to
one part of the body.
Houston says:
April 29, 2016 at 10:40 pm (UTC 0)
Utilizing prophylactics reduces your possibilities of
getting or spreading out herpes and other STIs, even if you are already
making use of another birth control technique to prevent pregnancy.
Grant Barbiere says:
May 16, 2016 at 4:07 am (UTC 0)
You are my inhalation, I possess few blogs and rarely run out from brand :). “‘Tis the most tender part of love, each other to forgive.” by John Sheffield.
jasa anti rayap says:
June 5, 2016 at 9:08 pm (UTC 0)
Wіtɦ hhavin ѕo much contᥱnt do you еver run int aany issues oof plagoprism օr copyriight
violation? Ⅿy site hаs a loot oof unique content I’ve eіther ѡritten mysеlf orr outsourced but іt seems a lot of it is popping it up аll oᴠer
the internet witɦoսt my permission. Dо you қnow anny methods to helpp stoр content ftom being ripped off?
I’ɗ trulу ɑppreciate it.
Kunjungi site Kami buat dapat Informasi Menarik mengenai jasa anti rayap .
Suwun
http://www.bandarjudi.xyz says:
September 30, 2016 at 11:13 am (UTC 0)
Yoᥙ cɑn certaionly seee your enthusiasm in the article уou wгite.
The sector hopes fߋr evsn more passionate writers llike үou who
ɑгe not afaid to memtion һow they bеlieve. Aⅼաays go afvter уouг heart.
Ⲏave a look att my web sige … http://www.bandarjudi.xyz
singapore says:
November 7, 2016 at 9:42 pm (UTC 0)
Hey I am so thrilled I found your website, I really found you by error, while I
was looking on Bing for something else, Anyhow I am here
now and would just like to say kudos for a remarkable post and a all round entertaining blog (I also love the theme/design), I don’t have time to
read through it all at the minute but I have bookmarked it and
also added your RSS feeds, so when I have time I will be
back to read a great deal more, Please do keep up the excellent job.
Jerome says:
November 18, 2016 at 3:39 am (UTC 0)
But, like fever blisters, genital herpes recurs, typically approximately four or 5 times a year.
newgirlfriend55.wikidot.com says:
January 3, 2017 at 10:20 am (UTC 0)
Now when I am handling a difficulty, I discover myself comparing it to exactly what
I have actually experienced formerly.
http://mohcaw8.deviantart.com/ says:
January 9, 2017 at 9:47 am (UTC 0)
Possibly the scientific staff even made you feel embarrassed or ashamed about getting herpes.
www.evernote.com says:
January 20, 2017 at 10:58 am (UTC 0)
Portanto as futuras mães devem antes de engravidar
ter uma orientação de uma boa nutricionista, para que sua
saúde esteja apta para formar um ser humano dentro
dela, saudável e que só venha a trazer alegria
para toda a família.
http://www.thefirstdispatch.com/ says:
January 20, 2017 at 10:38 pm (UTC 0)
Intrapartum transmission represent most neonatal infections and occurs with passage of the baby through a contaminated birth canal.
Service Sofa Bekasi says:
April 30, 2017 at 10:57 am (UTC 0)
great article and best loyout backgroud design
i like this
god bles you
umbrellababyshowerinvitations says:
July 24, 2017 at 8:42 am (UTC 0)
Im not gonna test Im going to keep filling the water
www.beautifulbabyshowerinvitations.com says:
August 6, 2017 at 9:57 am (UTC 0)
In case them aliens wanna fuck around and get dunked
Quick links (#28) | Urban Future (2.1) says:
March 2, 2015 at 1:36 am (UTC 0)
[…] The waves of deep learning — the game (with brief expert commentary). […]
Хакатон по глубинному обучению (deep learning) - Терещенко. Просто. Профессионально says:
June 23, 2015 at 1:24 pm (UTC 0)
[…] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]
Хакатон по глубинному обучению (deep learning) | IT-News says:
June 24, 2015 at 3:19 am (UTC 0)
[…] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]