Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
102 views

Deep Learning

Deep learning has current limitations in achieving human-level artificial intelligence. It relies on pattern recognition through mapping inputs to outputs based on large datasets without strong abstraction abilities. To achieve extreme generalization on new complex tasks, future AI will need richer models that blend deep learning with symbolic reasoning, stronger metrics to measure generalization beyond training data, and stronger priors through lifelong multi-task learning of reusable modules across domains. This will move AI development towards resembling how humans write computer programs and scientific discovery.

Uploaded by

Richie Wibisono
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
102 views

Deep Learning

Deep learning has current limitations in achieving human-level artificial intelligence. It relies on pattern recognition through mapping inputs to outputs based on large datasets without strong abstraction abilities. To achieve extreme generalization on new complex tasks, future AI will need richer models that blend deep learning with symbolic reasoning, stronger metrics to measure generalization beyond training data, and stronger priors through lifelong multi-task learning of reusable modules across domains. This will move AI development towards resembling how humans write computer programs and scientific discovery.

Uploaded by

Richie Wibisono
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 41

Deep learning:

current capabilities, limitations,


and future perspectives

François Chollet
@fchollet
Strong AI?

2019: mostly deep learning


I’ve been thinking about human-level AI for a long time
(e.g. what I was doing at Todai in 2012)
Human-level AI is harder
than it seemed in 2010
“Human-level AI is harder
than it seemed in 1955”
- John McCarthy (2006)
Strong AI?

2019: mostly deep learning


What are we talking about when we
talk about deep learning?
Not
this
More like this
The most surprising thing about
deep learning is how simple it is
Deep Net: a smooth
morphing function
from X to Y

Layers
(learned affine
transforms with fixed
nonlinearity)

Target space: another set of


Input space: a set of vectors vectors (usually lower-dimensional)
Deep net:
A smooth morphing from X to Y
expressed as a series of
Input space X Target space Y
simple geometric
transformations (layers)
a DENSE SAMPLING of the
`input X output` space

With “enough data”,


such a mapping fitted with SGD can
solve arbitrarily complex problems
● Mapping pixels to labels
● Mapping soundwaves to speech transcripts
● Mapping situation to action
● Anything to anything, really
So it doesn’t scale very well
The limitations of deep learning
Extreme sensitivity to adversarial perturbations
Extreme sensitivity to any input change not seen in the training data
It can only make sense of what it has seen before
Don’t anthropomorphize deep learning models

Source:
Samim Winiger
Deep learning is
pattern recognition
Models memorize a data manifold and
perform local generalization around training examples
Local generalization vs. extreme generalization
Task: finding launch parameters
for a moon rocket
Deep learning:
Humans:
● Hard-code space of launch
parameters ● Develop abstract model of the
● Learn point-by-point mapping problem (rocket science)
between launch parameters and ● Adapt it empirically (few launches)
rocket outcome
● Launch millions of times
Task: avoiding getting hit by a car

Deep learning: Humans:

● Learn point-by-point mapping ● Learn from others (imitation,


between sensorimotor space and spoken instructions)
vital outcome ● Model the world in an abstract
● Die millions of times way (e.g. understand physics to
● Need to re-learn most of it in a predict collisions in a new context)
new environment ● Die ~0 times
What does it take to achieve
extreme generalization?
Extreme generalization via strong abstraction:
the next frontier for AI
Better metrics.
Richer models.
Stronger priors.
We’ll need better metrics
First step: an ambitious new benchmark to measure
progress towards extreme generalization
experience
intelligence skill (data) priors

Posit: I = S / (E + P)
Measure I instead of S:
● Control for experience
● Control for priors
We’ll need richer models
More like current computer programs,
less like paper ball uncrumpling
Programs
humans can write
manually Learnable deep
learning models:
a tiny slice of
program space

Programs that one may want to develop


The future of AI

Programs
humans can write
manually Learnable deep
learning models:
a tiny slice of
program space

Programs that one may want to develop


“Learning” in “Machine Learning” will be
more program synthesis than
tuning the parameters of a hard-coded
geometric transform
But we’re not going to
throw away deep learning
Future AI systems will blend
pattern recognition (geometric intelligence)
with abstraction & reasoning
(symbolic intelligence)
Early examples (with handcrafted symbolic modules)
Blending symbolic AI (programming)
& geometric AI (deep learning)
We can’t learn these modules on every new
task (too complex, too little data)

We’ll need a library of reusable


symbolic & geometric modules,
shared across many tasks,
many systems
We’ll need stronger priors
Lifelong learning across many tasks and many domains

Learn subroutines reusable across diverse problem domains


Lifelong learning and multi-task learning
The long-term vision:
AI resembling current human-driven
software development workflows

You can also compare it to science, which itself is


a form of non-biological Strong AI
Thank you <3
François Chollet
@fchollet

You might also like