Machine Learning

Neural Networks Compression for Language Modeling

Tagged: , , , ,

This topic contains 0 replies, has 1 voice, and was last updated by  arXiv 1 year, 9 months ago.


  • arXiv
    5 pts

    Neural Networks Compression for Language Modeling

    In this paper, we consider several compression techniques for the language modeling problem based on recurrent neural networks (RNNs). It is known that conventional RNNs, e.g, LSTM-based networks in language modeling, are characterized with either high space complexity or substantial inference time. This problem is especially crucial for mobile applications, in which the constant interaction with the remote server is inappropriate. By using the Penn Treebank (PTB) dataset we compare pruning, quantization, low-rank factorization, tensor train decomposition for LSTM networks in terms of model size and suitability for fast inference.

    Neural Networks Compression for Language Modeling
    by Artem M. Grachev, Dmitry I. Ignatov, Andrey V. Savchenko
    https://arxiv.org/pdf/1708.05963v1.pdf

You must be logged in to reply to this topic.