«

»

Google DeepMind’s new Video Game AI

People are asking me today about the importance of Google DeepMind’s new video-game AI demo — see e.g. the Guardian article glowingly titled

Google develops computer program capable of learning tasks independently: ‘Agent’ hailed as first step towards true AI as it gets adept at playing 49 retro computer games and comes up with its own winning strategies

First off: Yeah, it’s certainly cool!!!

This is progress beyond DeepMind’s former Atari 2600 game demo, and I don’t want to pooh-pooh the amount of work and brilliance that goes into making something like this ACTUALLY WORK !! But this doesn’t especially feel to me like some sort of breakthrough, just good solid work in deep reinforcement learning… In particular, it doesn’t convince me that deep RL as DeepMind has habitually practiced it is an adequate approach to AGI….

The key issue is that these video games are very constrained domains, so that a deep reinforcement learning system can handle them without being able to interpret data in a broader context. This kind of constrained domain can obviously be handled by methods that wouldn’t work for broader, messier, more contextuality-rich real-world domains….

So to me “learning independently in a very constrained world” is not entirely the same problem as “learning independently in a big rich messy world” — the former can be done by a system that ignores a lot of stuff any real-world AI has to pay a lot of attention to…. The universe of simple video games they’ve dealt with in these fascinating experiments, are in the end still a very small and special domain for which a software system can be heavily tuned in various ways (i.e. even if the tuning is independent of which specific game is being learned, it can still be particular to the domain of “games of this type”).

Personally I think DeepMind would need to add a lot of extra stuff to their AGI approach as I understand it (at least, a lot of stuff beyond anything they’ve discussed publicly, and beyond anything hinted at by this demo), to approach AGI seriously. I think the kinds of deep learning and deep reinforcement learning they’ve focused on are only a moderate-sized fraction of the story of what’s needed for human-level cognition. To me, this demo doesn’t show that they have embraced the complexity and multi-aspect nature of what’s needed for human-level AGI. (Maybe they HAVE, behind the scenes, but this video doesn’t show it….)

Another way to put my point is: To really hit a home run their system will need to be able to learn to deal with new DOMAINS, not just new games within a domain for which their system has already been tuned and tweaked and human-configured….

One thing this demo does show is that, since their acquisition by Google, DeepMind is pushing ahead with AGI-oriented R&D according to their own tastes and directions, rather than simply applying their brainpower to Google’s search and ads products and so forth. That’s very nice to see 😉

38 comments

3 pings

  1. Alejandro says:

    I think what we are seeing is mainly Volodymyr Mnih’s personal project continuing, not DeepMind’s or Google’s strategy into AI. There are very few differences between the architecture and algorithm used in this paper and the previous one that they published seven months before this paper got submitted (bigger, deeper network and delayed updating of Q evaluator, that’s it). Note also how in their recent DRAW paper (http://arxiv.org/abs/1502.04623) they could have used a reinforcement learning paradigm but they chose to go the supervised learning route instead, and they haven’t used reinforcement learning for anything that didn’t involve Mnih as far as I can tell.

    Regarding it not dealing with different domains, it of course depends on what you mean by a domain. Even within one single game, there are parts that are different enough that I would call them different domains. On the other extreme, if you want domains to mean something completely general, you’ve got the No Free Lunch theorem telling you you can’t do that (“That’s not a new domain, you are still playing in a Kolmogorov-complexity defined universe”). In between you’ve got humans which can deal with some types of domain adaptations. I understand that you are saying that the commonality between the different environments it does learn in are too big. Your phrasing seems to indicate that you see a much harder line in what constitutes a domain than I can.

  2. Tattoo says:

    It’s actually a great and useful piece of info. I’m happy that you simply shared this useful info with us. Please stay us up to date like this. Thanks for sharing.

  3. gamis syar'i online says:

    Hi friends, its great article regarding teachingand entirely explained,
    keep it up all the time.

  4. Halina says:

    Empiece sesión si ya dispone de una cuenta como lector registrado, cree una cuenta nueva en un solo
    paso.

  5. Stevie says:

    Thank you for any other wonderful article. The place else may just anybody get that kind
    of information in such an ideal manner of writing? I’ve a presentation subsequent week,
    and I’m on the search for such information.

  6. joanns coupons says:

    This piece of writing gives clear idea in favor of the
    new users of blogging, that actually how to do running a blog.

  7. Harga Samsung Galaxy Prime SM-G530H says:

    Hello, after reading this amazing paragraph i am too happy to share my know-how here
    with mates.

  8. aqiqah says:

    Hello it’s me, I am also visiting this web page on a regular
    basis, this web site is in fact nice and the users are truly sharing fastidious thoughts.

  9. harga ac panasonic 2015 says:

    Hi there everyone, it’s my first go to see at this web page, and article is truly fruitful in favor of me, keep
    up posting such content.

  10. cara mengecilkan perut setelah melahirkan says:

    Hello to every , as I am truly keen of reading this web site’s post to be
    updated daily. It includes nice data.

  11. Jasa Fumigasi says:

    Thanks for any other excellent article. The place else
    could anyone get that type of information in such a perfect manner of writing?
    I have a presentation next week, and I’m on the search for such info.

  12. kurs pajak hari ini says:

    I go to see daily a few sites and information sites to read content,
    but this website gives quality based posts.

  13. informasi menarik says:

    I am actually glad to glance at this web site posts which carries tons of helpful facts, thanks for providing
    such statistics.

  14. orang terkaya di indonesia says:

    Thanks , I’ve just been searching for info about this subject for a long time and yours is the best I have came upon till now.
    But, what about the bottom line? Are you certain in regards to the source?

  15. Master cara da foto é bom says:

    Thanks for finally writing about > Google DeepMinds new Video Game AI goertzel.org < Liked it!

  16. ps says:

    We absolutely love your blog and find the majority of
    your post’s to be just what I’m looking
    for. Does one offer guest writers to write content to suit your needs?
    I wouldn’t mind publishing a post or elaborating on some of the subjects you write concerning here.
    Again, awesome weblog!

  17. jual mebel jepara says:

    Quality posts is the main to be a focus for the people
    to pay a quick visit the web site, that’s what this web page
    is providing.

  18. berita basket says:

    Fine way of describing, and good paragraph to obtain information on the topic of my presentation focus, which i am going to
    present in institution of higher education.

  19. hostmara says:

    What’s up, constantly i used to check website posts
    here early in the daylight, as i enjoy to find out more and more.

  20. ring bearer pillow says:

    Marvelous, what a weblog it is! This web site presents helpful data to us, keep it up.

  21. ishihara test says:

    What’s up to every one, it’s really a fastidious for me to pay a quick visit this site, it contains priceless
    Information.

  22. undangan pernikahan says:

    What i don’t realize is in fact how you’re no longer really much more neatly-preferred than you may be now.

    You’re very intelligent. You realize thus significantly in terms of this topic, produced me in my view believe it from numerous various angles.
    Its like women and men aren’t involved unless it’s something to accomplish with Girl gaga!
    Your own stuffs excellent. All the time maintain it
    up!

  23. thymosin says:

    Acne scars are even more tʏpісal
    compaгеԀ tօ yοu cߋᥙⅼԀ ⲣіcturе, ɑs
    աᥱlⅼ ɑѕ anyߋne
    tһаt ѕtrugglеѕ աіth tɦᥱm աіlⅼ tᥱⅼl
    ʏоᥙ, ᥱlіmіnatіng thеm ііѕ еen mоге tߋuǥһ cօmpaгеɗ tօ
    іt іѕ tο оbtaіn tҺеm.

    Ӎʏ ρsցe thymosin

  24. oral hsv tongue says:

    Moved by touch, there is always the danger of capturing the herpes virus when taking part in sex.

  25. Mabel says:

    Remember likewise that the spread of herpes is not restricted to
    one part of the body.

  26. Houston says:

    Utilizing prophylactics reduces your possibilities of
    getting or spreading out herpes and other STIs, even if you are already
    making use of another birth control technique to prevent pregnancy.

  27. Grant Barbiere says:

    You are my inhalation, I possess few blogs and rarely run out from brand :). “‘Tis the most tender part of love, each other to forgive.” by John Sheffield.

  28. jasa anti rayap says:

    Wіtɦ hhavin ѕo much contᥱnt do you еver run int aany issues oof plagoprism օr copyriight
    violation? Ⅿy site hаs a loot oof unique content I’ve eіther ѡritten mysеlf orr outsourced but іt seems a lot of it is popping it up аll oᴠer
    the internet witɦoսt my permission. Dо you қnow anny methods to helpp stoр content ftom being ripped off?
    I’ɗ trulу ɑppreciate it.

    Kunjungi site Kami buat dapat Informasi Menarik mengenai jasa anti rayap .

    Suwun

  29. http://www.bandarjudi.xyz says:

    Yoᥙ cɑn certaionly seee your enthusiasm in the article уou wгite.

    The sector hopes fߋr evsn more passionate writers llike үou who
    ɑгe not afaid to memtion һow they bеlieve. Aⅼաays go afvter уouг heart.

    Ⲏave a look att my web sige … http://www.bandarjudi.xyz

  30. singapore says:

    Hey I am so thrilled I found your website, I really found you by error, while I
    was looking on Bing for something else, Anyhow I am here
    now and would just like to say kudos for a remarkable post and a all round entertaining blog (I also love the theme/design), I don’t have time to
    read through it all at the minute but I have bookmarked it and
    also added your RSS feeds, so when I have time I will be
    back to read a great deal more, Please do keep up the excellent job.

  31. Jerome says:

    But, like fever blisters, genital herpes recurs, typically approximately four or 5 times a year.

  32. newgirlfriend55.wikidot.com says:

    Now when I am handling a difficulty, I discover myself comparing it to exactly what
    I have actually experienced formerly.

  33. http://mohcaw8.deviantart.com/ says:

    Possibly the scientific staff even made you feel embarrassed or ashamed about getting herpes.

  34. www.evernote.com says:

    Portanto as futuras mães devem antes de engravidar
    ter uma orientação de uma boa nutricionista, para que sua
    saúde esteja apta para formar um ser humano dentro
    dela, saudável e que só venha a trazer alegria
    para toda a família.

  35. http://www.thefirstdispatch.com/ says:

    Intrapartum transmission represent most neonatal infections and occurs with passage of the baby through a contaminated birth canal.

  36. Service Sofa Bekasi says:

    great article and best loyout backgroud design
    i like this
    god bles you

  37. umbrellababyshowerinvitations says:

    Im not gonna test Im going to keep filling the water

  38. www.beautifulbabyshowerinvitations.com says:

    In case them aliens wanna fuck around and get dunked

  1. Quick links (#28) | Urban Future (2.1) says:

    […] The waves of deep learning — the game (with brief expert commentary). […]

  2. Хакатон по глубинному обучению (deep learning) - Терещенко. Просто. Профессионально says:

    […] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]

  3. Хакатон по глубинному обучению (deep learning) | IT-News says:

    […] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]

Leave a Reply to oral hsv tongue Cancel reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>