Google DeepMind’s new Video Game AI

People are asking me today about the importance of Google DeepMind’s new video-game AI demo — see e.g. the Guardian article glowingly titled

Google develops computer program capable of learning tasks independently: ‘Agent’ hailed as first step towards true AI as it gets adept at playing 49 retro computer games and comes up with its own winning strategies

First off: Yeah, it’s certainly cool!!!

This is progress beyond DeepMind’s former Atari 2600 game demo, and I don’t want to pooh-pooh the amount of work and brilliance that goes into making something like this ACTUALLY WORK !! But this doesn’t especially feel to me like some sort of breakthrough, just good solid work in deep reinforcement learning… In particular, it doesn’t convince me that deep RL as DeepMind has habitually practiced it is an adequate approach to AGI….

The key issue is that these video games are very constrained domains, so that a deep reinforcement learning system can handle them without being able to interpret data in a broader context. This kind of constrained domain can obviously be handled by methods that wouldn’t work for broader, messier, more contextuality-rich real-world domains….

So to me “learning independently in a very constrained world” is not entirely the same problem as “learning independently in a big rich messy world” — the former can be done by a system that ignores a lot of stuff any real-world AI has to pay a lot of attention to…. The universe of simple video games they’ve dealt with in these fascinating experiments, are in the end still a very small and special domain for which a software system can be heavily tuned in various ways (i.e. even if the tuning is independent of which specific game is being learned, it can still be particular to the domain of “games of this type”).

Personally I think DeepMind would need to add a lot of extra stuff to their AGI approach as I understand it (at least, a lot of stuff beyond anything they’ve discussed publicly, and beyond anything hinted at by this demo), to approach AGI seriously. I think the kinds of deep learning and deep reinforcement learning they’ve focused on are only a moderate-sized fraction of the story of what’s needed for human-level cognition. To me, this demo doesn’t show that they have embraced the complexity and multi-aspect nature of what’s needed for human-level AGI. (Maybe they HAVE, behind the scenes, but this video doesn’t show it….)

Another way to put my point is: To really hit a home run their system will need to be able to learn to deal with new DOMAINS, not just new games within a domain for which their system has already been tuned and tweaked and human-configured….

One thing this demo does show is that, since their acquisition by Google, DeepMind is pushing ahead with AGI-oriented R&D according to their own tastes and directions, rather than simply applying their brainpower to Google’s search and ads products and so forth. That’s very nice to see 😉

57 pings

  1. Alejandro says:

    I think what we are seeing is mainly Volodymyr Mnih’s personal project continuing, not DeepMind’s or Google’s strategy into AI. There are very few differences between the architecture and algorithm used in this paper and the previous one that they published seven months before this paper got submitted (bigger, deeper network and delayed updating of Q evaluator, that’s it). Note also how in their recent DRAW paper (http://arxiv.org/abs/1502.04623) they could have used a reinforcement learning paradigm but they chose to go the supervised learning route instead, and they haven’t used reinforcement learning for anything that didn’t involve Mnih as far as I can tell.

    Regarding it not dealing with different domains, it of course depends on what you mean by a domain. Even within one single game, there are parts that are different enough that I would call them different domains. On the other extreme, if you want domains to mean something completely general, you’ve got the No Free Lunch theorem telling you you can’t do that (“That’s not a new domain, you are still playing in a Kolmogorov-complexity defined universe”). In between you’ve got humans which can deal with some types of domain adaptations. I understand that you are saying that the commonality between the different environments it does learn in are too big. Your phrasing seems to indicate that you see a much harder line in what constitutes a domain than I can.

  2. Tattoo says:

    It’s actually a great and useful piece of info. I’m happy that you simply shared this useful info with us. Please stay us up to date like this. Thanks for sharing.

  3. gamis syar'i online says:

    Hi friends, its great article regarding teachingand entirely explained,
    keep it up all the time.

  4. Halina says:

    Empiece sesión si ya dispone de una cuenta como lector registrado, cree una cuenta nueva en un solo

  5. Stevie says:

    Thank you for any other wonderful article. The place else may just anybody get that kind
    of information in such an ideal manner of writing? I’ve a presentation subsequent week,
    and I’m on the search for such information.

  6. joanns coupons says:

    This piece of writing gives clear idea in favor of the
    new users of blogging, that actually how to do running a blog.

  7. Harga Samsung Galaxy Prime SM-G530H says:

    Hello, after reading this amazing paragraph i am too happy to share my know-how here
    with mates.

  8. aqiqah says:

    Hello it’s me, I am also visiting this web page on a regular
    basis, this web site is in fact nice and the users are truly sharing fastidious thoughts.

  9. harga ac panasonic 2015 says:

    Hi there everyone, it’s my first go to see at this web page, and article is truly fruitful in favor of me, keep
    up posting such content.

  10. cara mengecilkan perut setelah melahirkan says:

    Hello to every , as I am truly keen of reading this web site’s post to be
    updated daily. It includes nice data.

  11. Jasa Fumigasi says:

    Thanks for any other excellent article. The place else
    could anyone get that type of information in such a perfect manner of writing?
    I have a presentation next week, and I’m on the search for such info.

  12. kurs pajak hari ini says:

    I go to see daily a few sites and information sites to read content,
    but this website gives quality based posts.

  13. informasi menarik says:

    I am actually glad to glance at this web site posts which carries tons of helpful facts, thanks for providing
    such statistics.

  14. orang terkaya di indonesia says:

    Thanks , I’ve just been searching for info about this subject for a long time and yours is the best I have came upon till now.
    But, what about the bottom line? Are you certain in regards to the source?

  15. Master cara da foto é bom says:

    Thanks for finally writing about > Google DeepMinds new Video Game AI goertzel.org < Liked it!

  16. ps says:

    We absolutely love your blog and find the majority of
    your post’s to be just what I’m looking
    for. Does one offer guest writers to write content to suit your needs?
    I wouldn’t mind publishing a post or elaborating on some of the subjects you write concerning here.
    Again, awesome weblog!

  17. jual mebel jepara says:

    Quality posts is the main to be a focus for the people
    to pay a quick visit the web site, that’s what this web page
    is providing.

  18. berita basket says:

    Fine way of describing, and good paragraph to obtain information on the topic of my presentation focus, which i am going to
    present in institution of higher education.

  19. hostmara says:

    What’s up, constantly i used to check website posts
    here early in the daylight, as i enjoy to find out more and more.

  20. ring bearer pillow says:

    Marvelous, what a weblog it is! This web site presents helpful data to us, keep it up.

  21. ishihara test says:

    What’s up to every one, it’s really a fastidious for me to pay a quick visit this site, it contains priceless

  22. undangan pernikahan says:

    What i don’t realize is in fact how you’re no longer really much more neatly-preferred than you may be now.

    You’re very intelligent. You realize thus significantly in terms of this topic, produced me in my view believe it from numerous various angles.
    Its like women and men aren’t involved unless it’s something to accomplish with Girl gaga!
    Your own stuffs excellent. All the time maintain it

  23. thymosin says:

    Acne scars are even more tʏpісal
    compaгеԀ tօ yοu cߋᥙⅼԀ ⲣіcturе, ɑs
    աᥱlⅼ ɑѕ anyߋne
    tһаt ѕtrugglеѕ աіth tɦᥱm աіlⅼ tᥱⅼl
    ʏоᥙ, ᥱlіmіnatіng thеm ііѕ еen mоге tߋuǥһ cօmpaгеɗ tօ
    іt іѕ tο оbtaіn tҺеm.

    Ӎʏ ρsցe thymosin

  24. oral hsv tongue says:

    Moved by touch, there is always the danger of capturing the herpes virus when taking part in sex.

  25. Mabel says:

    Remember likewise that the spread of herpes is not restricted to
    one part of the body.

  26. Houston says:

    Utilizing prophylactics reduces your possibilities of
    getting or spreading out herpes and other STIs, even if you are already
    making use of another birth control technique to prevent pregnancy.

  27. Grant Barbiere says:

    You are my inhalation, I possess few blogs and rarely run out from brand :). “‘Tis the most tender part of love, each other to forgive.” by John Sheffield.

  28. jasa anti rayap says:

    Wіtɦ hhavin ѕo much contᥱnt do you еver run int aany issues oof plagoprism օr copyriight
    violation? Ⅿy site hаs a loot oof unique content I’ve eіther ѡritten mysеlf orr outsourced but іt seems a lot of it is popping it up аll oᴠer
    the internet witɦoսt my permission. Dо you қnow anny methods to helpp stoр content ftom being ripped off?
    I’ɗ trulу ɑppreciate it.

    Kunjungi site Kami buat dapat Informasi Menarik mengenai jasa anti rayap .


  29. http://www.bandarjudi.xyz says:

    Yoᥙ cɑn certaionly seee your enthusiasm in the article уou wгite.

    The sector hopes fߋr evsn more passionate writers llike үou who
    ɑгe not afaid to memtion һow they bеlieve. Aⅼաays go afvter уouг heart.

    Ⲏave a look att my web sige … http://www.bandarjudi.xyz

  30. singapore says:

    Hey I am so thrilled I found your website, I really found you by error, while I
    was looking on Bing for something else, Anyhow I am here
    now and would just like to say kudos for a remarkable post and a all round entertaining blog (I also love the theme/design), I don’t have time to
    read through it all at the minute but I have bookmarked it and
    also added your RSS feeds, so when I have time I will be
    back to read a great deal more, Please do keep up the excellent job.

  31. Jerome says:

    But, like fever blisters, genital herpes recurs, typically approximately four or 5 times a year.

  32. newgirlfriend55.wikidot.com says:

    Now when I am handling a difficulty, I discover myself comparing it to exactly what
    I have actually experienced formerly.

  33. http://mohcaw8.deviantart.com/ says:

    Possibly the scientific staff even made you feel embarrassed or ashamed about getting herpes.

  34. www.evernote.com says:

    Portanto as futuras mães devem antes de engravidar
    ter uma orientação de uma boa nutricionista, para que sua
    saúde esteja apta para formar um ser humano dentro
    dela, saudável e que só venha a trazer alegria
    para toda a família.

  35. http://www.thefirstdispatch.com/ says:

    Intrapartum transmission represent most neonatal infections and occurs with passage of the baby through a contaminated birth canal.

  36. Service Sofa Bekasi says:

    great article and best loyout backgroud design
    i like this
    god bles you

  37. umbrellababyshowerinvitations says:

    Im not gonna test Im going to keep filling the water

  38. www.beautifulbabyshowerinvitations.com says:

    In case them aliens wanna fuck around and get dunked

  39. etsy says:

    This site certainly has all of the info I needed about this subject and didn’t know
    who to ask.

  40. Jasa fumigasi Jakarta says:

    terima kasih atas info nya

  41. ซื้อไลค์ says:

    ปั้มไลค์มือถือ, รับปั้มไลค์รูป, ปั้มไลค์รูป Facebook, ปั้มเพจ, จ้างปั้มไลค์, ไลค์แฟนเพจเฟสบุ๊ค, รับเพิ่ม Like Fanpage Facebook ราคาพิเศษ เหมาะสำหรับผู้ที่ต้องการเพิ่ม Like Fanpage ให้ดูเป็นความน่าเชื่อถือของร้าน, ปั้มไลค์ฟรี, ปั้ม Like ฟรี, โกงไลค์, ปั้มไลค์, จ้างไลค์, กดไลค์รูป, บริการเพิ่มไลค์แฟนเพจฟรี, ปั้มไลค์แฟนเพจ, เพิ่มไลค์คนไทย, รับทำเพจสินค้าพร้อมยอดไลค์, Add Like Fanpage, ปั้มไลค์รูปเฟส, ปั้มโพสต์, บริการเพิ่ม Like ให้แฟนเพจของคุณ โดยทีมงานระดับมืออาชีพ ในระยะเวลาอันสั้นและราคาถูก รับประกันไลค์จากคนไทย 100%, ไลค์เพจ, ปั้มไลค์ง่ายๆ, ไลค์แฟนเพจ, รับทำไลค์แฟนเพจจำนวนมาก, ปั้มรูป, กดไลค์แฟนเพจ, รับจ้างเพิ่มไลค์, ปั้มไลค์ครบวงจร, ปั้มยอดไลค์เป็นคนไทย 100%, ปั้มไลค์ ในโทรศัพท์, แฟนเพจพร้อมยอดไลค์, รับจ้างกดไลค์, Up Like Fanpage, เพิ่ม Like, Pump Like, ปั้มไลค์สถานะ, Like Fanpage, กดไลค์, เพิ่มไลค์แฟนเพจ, รับสร้างแฟนเพจพร้อมยอดไลค์, รับไลค์แฟนเพจ, เพิ่มไลค์แฟนเพจ ราคาถูก, ปั้มไลค์ Facebook, ปั้มไลค์เพจฟรี, ปั้มเพจเฟสบุ๊ค, ปั้มไลค์เฟสบุ๊ค, รับทำเพจเฟสบุ๊ค, เพิ่มไลค์ฟรี, ปั้มไลค์เพจ, ปั้มไลค์เพจ Facebook, Auto Like, ไลค์แฟนเพจได้สูงสุดหนึ่งล้านไลค์, รับปั้มไลค์เพจ, รับจ้างปั้มไลค์, ปั่นไลค์, บริการปั้มไลค์แฟนเพจฟรี, บริการปั้มไลค์ฟรี, การเพิ่ม Like เป็นการเพิ่มโอกาสทางการขายสินค้าของคุณนั่นเอง ด้วยงานบริการเพิ่มไลค์แฟนเพจระดับมืออาชีพ ประสบการณ์การทำงานหลายปี, รับทำไลค์ครบวงจร, ปั้มไลค์เพจคนไทย, ปั้มไลค์รูป, กด Like, ไลค์แฟนเพจราคาถูก, เพิ่มไลค์เพจ ราคาถูก, บริการเพิ่มไลค์เฟสบุ๊คให้แฟนเพจ, เพิ่มไลค์, ไลค์คนไทย 100%, รับไลค์ครบวงจรทั่วไทย, ปั้มไลค์ราคาถูก, รับเพิ่มไลค์แฟนเพจ เป็นการโปรโมทแฟนเพจ เพื่อเพิ่มยอดขายสินค้าได้, เพิ่มไลค์เพจ, ปั้มไลค์ทั่วไป, รับ Like Fanpage, ปั้มเพจ เหมาะสำหรับผู้ที่ต้องการยอดไลค์เพจเพจเยอะๆ, จ้างเพิ่มไลค์, รับเพิ่มไลค์แฟนเพจเฟสบุ๊ค, ซื้อไลค์แฟนเพจ, รับเพิ่มไลค์, จ้างไลค์รูป, จ้างกดไลค์

  42. https://Nellie-cobra.blogspot.com says:

    Very good article. I am going through some of these issues as well..

  43. https://yeoncheon-softmassage.blogspot.com says:

    Hiya very nice blog!! Guy .. Excellent .. Wonderful .. I will bookmark
    your web site and take the feeds also? I am happy
    to search out numerous helpful information here in the put up,
    we need work out extra techniques in this regard, thanks for sharing.
    . . . . .

  44. https://damyang-opmassage.blogspot.com says:

    I was extremely pleased to uncover this website.

    I need to to thank you for ones time for this particularly fantastic read!!
    I definitely savored every part of it and i also have you bookmarked to
    see new stuff in your website.

  45. https://dongducheon-opanma3.blogspot.com says:

    always i used to read smaller articles or reviews that
    as well clear their motive, and that is also happening
    with this paragraph which I am reading at this time.

  46. 바카라 게임 다운로드 says:

    I visit day-to-day some web sites and sites to read content,
    however this web site offers quality based articles.

  47. 시카고 슬롯 머신 says:

    My relatives every time say that I am killing my time here at net,
    except I know I am getting familiarity every day by reading thes fastidious articles or reviews.

  48. 블랙 잭 게임 says:

    This blog was… how do you say it? Relevant!! Finally I have found
    something that helped me. Thanks!

  49. 바카라 승률 says:

    It’s actually a great and useful piece of information. I am happy that you simply shared this helpful info with us.
    Please keep us informed like this. Thanks for sharing.

  50. 해적맞고게임 says:

    What’s up everyone, it’s my first go to see at this website, and post is really fruitful for me, keep up posting these content.

  51. m 카지노 says:

    Every weekend i used to pay a quick visit this web page,
    for the reason that i want enjoyment, for the reason that this this site conations really pleasant funny
    data too.

  52. 해적게임바둑이 says:

    Do you mind if I quote a few of your posts as
    long as I provide credit and sources back to your website?
    My website is in the very same niche as yours and
    my users would truly benefit from a lot of the
    information you provide here. Please let me know if this alright with you.

  53. divorceBeeva says:

    Even in case you feel positive on your final decision to divorce, then you’re probably stressing telling your own children. Nobody loves telling kids that their parents ‘ are becoming a divorce, so even though everybody else in the family agrees it is for the ideal. Be ready for your own children to become somewhat distressed by this news, or for these to be less upset when you are. Quite simply, you must be ready for just about any possible reaction by the own children. These basic tips should choose the terror from the’big talk’ and help you prepare responses for the children’ tricky inquiries. The first item of advice is additionally the most straightforward: tell the kids the facts in what’s happening. This doesn’t signify that you ought to tell your kids the most romantic details of why you and your partner are dividing. However, your young ones want to be aware that you and your better half will likely be living apart, that it’s maybe not momentary, and also that their own lives will probably alter as a outcome.
    Go to site top online divorce sites

  54. jtvdScawn says:

    find someone to write my college paper http://writemypaperbuyxvv.com/ – write my sociology paper pay someone to write my paper will someone write my paper for me can someone write my paper for me

  1. Quick links (#28) | Urban Future (2.1) says:

    […] The waves of deep learning — the game (with brief expert commentary). […]

  2. Хакатон по глубинному обучению (deep learning) - Терещенко. Просто. Профессионально says:

    […] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]

  3. Хакатон по глубинному обучению (deep learning) | IT-News says:

    […] (Yann LeCun) на CVPR15, заметка Ю. Шмидтхубера (Jürgen Schmidhuber), пост Б. Гёртцеля (Ben Goertzel), уже упомянутые в статье на Хабре […]

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>