MetaLearning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
This topic contains 0 replies, has 1 voice, and was last updated by arXiv 1 year, 4 months ago.

MetaLearning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
Learning to learn is a powerful paradigm for enabling models to learn from data more effectively and efficiently. A popular approach to metalearning is to train a recurrent model to read in a training dataset as input and output the parameters of a learned model, or output predictions for new test inputs. Alternatively, a more recent approach to metalearning aims to acquire deep representations that can be effectively finetuned, via standard gradient descent, to new tasks. In this paper, we consider the metalearning problem from the perspective of universality, formalizing the notion of learning algorithm approximation and comparing the expressive power of the aforementioned recurrent models to the more recent approaches that embed gradient descent into the metalearner. In particular, we seek to answer the following question: does deep representation combined with standard gradient descent have sufficient capacity to approximate any learning algorithm? We find that this is indeed true, and further find, in our experiments, that gradientbased metalearning consistently leads to learning strategies that generalize more widely compared to those represented by recurrent models.
MetaLearning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
by Chelsea Finn, Sergey Levine
https://arxiv.org/pdf/1710.11622v2.pdf
You must be logged in to reply to this topic.