Topic Tag: information theory

home Forums Topic Tag: information theory

 Informational Neurobayesian Approach to Neural Networks Training. Opportunities and Prospects

 

A study of the classification problem in context of information theory is presented in the paper. Current research in that field is focused on optimisation and bayesian approach. Although that gives satisfying results, they require a vast amount of data and computations to train on. Authors propose…


 Emergence of Invariance and Disentangling in Deep Representations

 

Using established principles from Information Theory and Statistics, we show that in a deep neural network invariance to nuisance factors is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network to…


 Algorithmically probable mutations reproduce aspects of evolution such as convergence rate, genetic memory, modularity, diversity explosions, and mass extinction

We show that if evolution is algorithmic in any form and can thus be considered a program in software space, the emergence of a natural algorithmic probability distribution has the potential to become an accelerating mechanism. When the strategy produces unfit organisms massive extinctions occur an…


 Algorithmically probable mutations reproduce aspects of evolution such as genetic memory, modularity, diversity explosions, and mass extinctions

We show that if evolution is algorithmic in any form and can thus be considered a program in software space, the emergence of a natural algorithmic probability distribution has the potential to become an accelerating mechanism. When the strategy produces unfit organisms massive extinctions occur an…