Deep Learning: Professor Geoffrey Hinton FRS, 25 June 2015

Duration: 1 hour 6 mins

Share this media item:

Embed this media item:

<iframe width="_width_" height="_height_" src="https://sms.cam.ac.uk/media/2017973/embed" frameborder="0" scrolling="no" allowfullscreen></iframe>

Choose size:

About this item

Available Formats

About this item

Deep Learning: Professor Geoffrey Hinton FRS, 25 June 2015's image

Description:	Deep Learning, Professor Geoffrey Hinton, FRS, University of Toronto and Google


Created:	2015-06-29 14:45
Collection:	Division F Talks
Publisher:	University of Cambridge
Copyright:	Geoffrey Hinton
Language:	eng (English)


Abstract:	I will describe an efficient, unsupervised learning procedure for a simple type of two-layer neural network called a Restricted Boltzmann Machine. I will then show how this algorithm can be used recursively to learn multiple layers of features without requiring any supervision. After this unsupervised “pre-training”, the features in all layers can be fine-tuned to be better at discriminating between classes by using the standard backpropagation procedure from the 1980s. Unsupervised pre-training greatly improves generalization to new data, especially when the number of labelled examples is small. Ten years ago, the pre-training approach initiated a revival of research on deep, feedforward neural networks. I will describe some of the major successes of deep networks for speech recognition, object recognition and machine translation and I will speculate about where this research is headed. The fact that backpropagation learning is now the method of choice for a wide variety of really difficult tasks means that neuroscientists may need to reconsider their well-worn arguments about why it cannot possibly be occurring in cortex. I shall conclude by undermining two of the commonest objections to the idea that cortex is actually backpropagating error derivatives through a hierarchy of cortical areas and I shall show that spike-time dependent plasticity is a signature of backpropagation.

Abstract:

I will describe an efficient, unsupervised learning procedure for a simple type of two-layer neural network called a Restricted Boltzmann Machine. I will then show how this algorithm can be used recursively to learn multiple layers of features without requiring any supervision. After this unsupervised “pre-training”, the features in all layers can be fine-tuned to be better at discriminating between classes by using the standard backpropagation procedure from the 1980s. Unsupervised pre-training greatly improves generalization to new data, especially when the number of labelled examples is small. Ten years ago, the pre-training approach initiated a revival of research on deep, feedforward neural networks. I will describe some of the major successes of deep networks for speech recognition, object recognition and machine translation and I will speculate about where this research is headed. The fact that backpropagation learning is now the method of choice for a wide variety of really difficult tasks means that neuroscientists may need to reconsider their well-worn arguments about why it cannot possibly be occurring in cortex. I shall conclude by undermining two of the commonest objections to the idea that cortex is actually backpropagating error derivatives through a hierarchy of cortical areas and I shall show that spike-time dependent plasticity is a signature of backpropagation.

Available Formats

Format	Quality	Bitrate	Size
MPEG-4 Video	1280x720	2.99 Mbits/sec	1.45 GB	View
MPEG-4 Video	640x360	1.94 Mbits/sec	961.61 MB	View
WebM	640x360	876.95 kbits/sec	423.92 MB	View
iPod Video	480x270	522.46 kbits/sec	252.56 MB	View
MP3	44100 Hz	250.09 kbits/sec	120.90 MB	Listen
Auto *	(Allows browser to choose a format it supports)