#### Recent Advances in Convolutional Neural Networks

Convolutional Neural Network DNN image language speech

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging…

#### DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection

audio Convolutional Neural Network DNN

This report presents our audio event detection system submitted for Task 2, “Detection of rare sound events”, of DCASE 2017 challenge. The proposed system is based on convolutional neural networks (CNNs) and deep neural networks (DNNs) coupled with novel weighted and multi-task loss fun…

#### Query-based Attention CNN for Text Similarity Map

Convolutional Neural Network text

In this paper, we introduce Query-based Attention CNN(QACNN) for Text Similarity Map, an end-to-end neural network for question answering. This network is composed of compare mechanism, two-staged CNN architecture with attention mechanism, and a prediction layer. First, the compare mechanism compar…

#### Low-shot learning with large-scale diffusion

Convolutional Neural Network image

This paper considers the problem of inferring image labels for which only a few labelled examples are available at training time. This setup is often referred to as low-shot learning in the literature, where a standard approach is to re-train the last few layers of a convolutional neural network le…

#### Reply With: Proactive Recommendation of Email Attachments

Email responses often contain items-such as a file or a hyperlink to an external document-that are attached to or included inline in the body of the message. Analysis of an enterprise email corpus reveals that 35% of the time when users include these items as part of their response, the attachable …

#### A systematic study of the class imbalance problem in convolutional neural networks

CIFAR Convolutional Neural Network DNN IMAGENET MNIST

In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine l…

#### An Effective Training Method For Deep Convolutional Neural Network

CIFAR Convolutional Neural Network

In this paper, we propose the nonlinearity generation method to speed up and stabilize the training of deep convolutional neural networks. The proposed method modifies a family of activation functions as nonlinearity generators (NGs). NGs make the activation functions linear symmetric for their inp…

#### Checkpoint Ensembles: Ensemble Methods from a Single Training Process

Convolutional Neural Network health image text

We present the checkpoint ensembles method that can learn ensemble models on a single training process. Although checkpoint ensembles can be applied to any parametric iterative learning technique, here we focus on neural networks. Neural networks’ composable and simple neurons make it possibl…

#### Protein identification with deep learning: from abc to xyz

Convolutional Neural Network DNN health image RNN

Proteins are the main workhorses of biological functions in a cell, a tissue, or an organism. Identification and quantification of proteins in a given sample, e.g. a cell type under normal/disease conditions, are fundamental tasks for the understanding of human health and disease. In this paper, we…

#### Projection Based Weight Normalization for Deep Neural Networks

CIFAR Convolutional Neural Network DNN gradient image IMAGENET MNIST

Optimizing deep neural networks (DNNs) often suffers from the ill-conditioned problem. We observe that the scaling-based weight space symmetry property in rectified nonlinear network will cause this negative effect. Therefore, we propose to constrain the incoming weights of each neuron to be unit-n…

#### Deep Convolutional Neural Networks as Generic Feature Extractors

CIFAR Convolutional Neural Network

Recognizing objects in natural images is an intricate problem involving multiple conflicting objectives. Deep convolutional neural networks, trained on large datasets, achieve convincing results and are currently the state-of-the-art approach for this task. However, the long time needed to train su…

#### How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions?

Convolutional Neural Network DNN image

In the last few years, we have seen the rise of deep learning applications in a broad range of chemistry research problems. Recently, we reported on the development of Chemception, a deep convolutional neural network (CNN) architecture for general-purpose small molecule property prediction. In this…

#### Towards lightweight convolutional neural networks for object detection

car Convolutional Neural Network

We propose model with larger spatial size of feature maps and evaluate it on object detection task. With the goal to choose the best feature extraction network for our model we compare several popular lightweight networks. After that we conduct a set of experiments with channels reduction algorithm…

#### Data Augmentation of Spectral Data for Convolutional Neural Network (CNN) Based Deep Chemometrics

Convolutional Neural Network DNN

Deep learning methods are used on spectroscopic data to predict drug content in tablets from near infrared (NIR) spectra. Using convolutional neural networks (CNNs), features are ex- tracted from the spectroscopic data. Extended multiplicative scatter correction (EMSC) and a novel spectral data aug…

#### To prune, or not to prune: exploring the efficacy of pruning for model compression

Convolutional Neural Network DNN LSTM RNN

Model pruning seeks to induce sparsity in a deep neural network’s various connection matrices, thereby reducing the number of nonzero-valued parameters in the model. Recent reports (Han et al., 2015; Narang et al., 2017) prune deep networks at the cost of only a marginal loss in accuracy and …

#### Learning Affinity via Spatial Propagation Networks

Convolutional Neural Network DNN image

In this paper, we propose spatial propagation networks for learning the affinity matrix for vision tasks. We show that by constructing a row/column linear propagation model, the spatially varying transformation matrix exactly constitutes an affinity matrix that models dense, global pairwise relatio…

#### Facial Key Points Detection using Deep Convolutional Neural Network – NaimishNet

Convolutional Neural Network image

Facial Key Points (FKPs) Detection is an important and challenging problem in the fields of computer vision and machine learning. It involves predicting the co-ordinates of the FKPs, e.g. nose tip, center of eyes, etc, for a given face. In this paper, we propose a LeNet adapted Deep CNN model ̵…

#### Backprop KF: Learning Discriminative Deterministic State Estimators

Convolutional Neural Network gradient image RNN

Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the…

#### IQ of Neural Networks

Convolutional Neural Network image

IQ tests are an accepted method for assessing human intelligence. The tests consist of several parts that must be solved under a time constraint. Of all the tested abilities, pattern recognition has been found to have the highest correlation with general intelligence. This is primarily because patt…

#### Improving Efficiency in Convolutional Neural Network with Multilinear Filters

Convolutional Neural Network DNN

The excellent performance of deep neural networks has enabled us to solve several automatization problems, opening an era of autonomous devices. However, current deep net architectures are heavy with millions of parameters and require billions of floating point operations. Several works have been d…

#### Case Study: Explaining Diabetic Retinopathy Detection Deep CNNs via Integrated Gradients

Convolutional Neural Network gradient

In this report, we applied integrated gradients to explaining a neural network for diabetic retinopathy detection. The integrated gradient is an attribution method which measures the contributions of input to the quantity of interest. We explored some new ways for applying this method such as expla…

#### DeepTransport: Learning Spatial-Temporal Dependency for Traffic Condition Forecasting

car Convolutional Neural Network DNN RNN

Predicting traffic conditions has been recently explored as a way to relieve traffic congestion. Several pioneering approaches have been proposed based on traffic observations of the target location as well as its adjacent regions, but they obtain somewhat limited accuracy due to lack of mining roa…

#### AutoEncoder by Forest

Convolutional Neural Network DNN

Auto-encoding is an important task which is typically realized by deep neural networks (DNNs) such as convolutional neural networks (CNN). In this paper, we propose EncoderForest (abbrv. eForest), the first tree ensemble based auto-encoder. We present a procedure for enabling forests to do backward…

#### Adaptive Convolutional Filter Generation for Natural Language Understanding

Convolutional Neural Network language

Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP are not expressive enough, in the sense that all input sentences share the same learned (and static) set of f…

#### Underwater Multi-Robot Convoying using Visual Tracking by Detection

Convolutional Neural Network motion RNN

We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal …