#Bayesian #deepLearning workshop talks from #NeuRIPS2019

Link

🔭 @DeepGravity

A Two-Stage Approach to #FewShotLearning for Image Recognition

This paper proposes a multi-layer neural network structure for few-shot image recognition of novel categories. The proposed multi-layer neural network architecture encodes transferable knowledge extracted from a large annotated dataset of base categories. This architecture is then applied to novel categories containing only a few samples. The transfer of knowledge is carried out at the feature-extraction and the classification levels distributed across the two training stages. In the first-training stage, we introduce the relative feature to capture the structure of the data as well as obtain a low-dimensional discriminative space. Secondly, we account for the variable variance of different categories by using a network to predict the variance of each class. Classification is then performed by computing the Mahalanobis distance to the mean-class representation in contrast to previous approaches that used the Euclidean distance. In the second-training stage, a category-agnostic mapping is learned from the mean-sample representation to its corresponding class-prototype representation. This is because the mean-sample representation may not accurately represent the novel category prototype. Finally, we evaluate the proposed network structure on four standard few-shot image recognition datasets, where our proposed few-shot learning system produces competitive performance compared to previous work. We also extensively studied and analyzed the contribution of each component of our proposed framework.

Paper

🔭 @DeepGravity

Semantic #RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions

Hierarchical Reinforcement Learning algorithms have successfully been applied to temporal credit assignment problems with sparse reward signals. However, state-of-the-art algorithms require manual specification of sub-task structures, a sample inefficient exploration phase or lack semantic interpretability. Humans, on the other hand, efficiently detect hierarchical sub-structures induced by their surroundings. It has been argued that this inference process universally applies to language, logical reasoning as well as motor control. Therefore, we propose a cognitive-inspired Reinforcement Learning architecture which uses grammar induction to identify sub-goal policies. By treating an on-policy trajectory as a sentence sampled from the policy-conditioned language of the environment, we identify hierarchical constituents with the help of unsupervised grammatical inference. The resulting set of temporal abstractions is called action grammar (Pastra & Aloimonos, 2012) and unifies symbolic and connectionist approaches to Reinforcement Learning. It can be used to facilitate efficient imitation, transfer and online learning.

Paper

🔭 @DeepGravity

#Reinforcement Learning as Probabilistic Modelling: A #VariationalInference Formulation

Reinforcement Learning is concerned with an agent attempting to acquire optimal behaviour in unknown environments that typically exhibit stochasticity. Though minimally supervised, reinforcement learning algorithms have shown numerous success ranging from solving ATARI games using Deep Q-Networks, to the triumphant victory against the world champions in the game of GO, and recently in Start Craft.

Article

🔭 @DeepGravity

#NeurIPS2019 Workshop on #MachineLearning for #Creativity and Design

Link

🔭 @DeepGravity

A webinar summarizing lessons learned in #AI implementation from the 2019 Winning With AI Report.

Link

🔭 @DeepGravity

BMW InnovationLab:
#BMW shares #AI algorithms used in production, available on GitHub

Link

🔭 @DeepGravity

#Autograd
Autograd can automatically differentiate native #Python and #Numpy code.

#Google #JAX
JAX is Autograd and XLA, brought together for high-performance machine learning research.

🔭 @DeepGravity

The year in AI: 2019 #ML / #AI advances recap

It has become somewhat of a tradition for me to do an end-of-year retrospective of advances in AI/ML (see last year’s round up for example), so here we go again! This year started with a big recognition to the impact of #DeepLearning when #Hinton, #Bengio, and #Lecun were awarded the #Turing award.

Link

🔭 @DeepGravity

#NeuralNetworks Timeline

🔭 @DeepGravity

A good Telegram channel, managed by an Iranian researcher, covering some new papers in #AI

Link to the channel

🔭 @DeepGravity

#Fun

🔭 @DeepGravity

Our reinforcement learning architect designs have been just published on #NeurIPS2019 AI Art Gallery:

https://lnkd.in/dFZ37BN

Seems it draws like a baby now, but is growing and hopefully would be a skillful #RL artist very soon.

#reinforcementlearning #deeplearning #ai #artificialintelligence #art #deepreinforcementlearning #creativeart #neurips

🔭 @DeepGravity

#NeurIPS2019:

Lectures

Papers

🔭 @DeepGravity

P-CapsNets: a General Form of #ConvolutionalNeuralNetworks

We propose Pure CapsNets (P-CapsNets) which is a generation of normal CNNs structurally. Specifically, we make three modifications to current CapsNets. First, we remove routing procedures from CapsNets based on the observation that the coupling coefficients can be learned implicitly. Second, we replace the convolutional layers in CapsNets to improve efficiency. Third, we package the capsules into rank-3 tensors to further improve efficiency. The experiment shows that P-CapsNets achieve better performance than CapsNets with varied routing procedures by using significantly fewer parameters on MNIST&CIFAR10. The high efficiency of P-CapsNets is even comparable to some deep compressing models. For example, we achieve more than 99% percent accuracy on MNIST by using only 3888 parameters. We visualize the capsules as well as the corresponding correlation matrix to show a possible way of initializing CapsNets in the future. We also explore the adversarial robustness of P-CapsNets compared to #CNNs.

Paper

🔭 @DeepGravity

Seven differences between academia and industry for building machine learning and #deepLearning models

1) Approach to accuracy
2) Training vs serving
3) Emphasis on Engineering
4) Less emphasis on larger models
5) Understanding the baseline
6) Understanding the intricacies of data
7) Focusing on deep learning too early

Link

🔭 @DeepGravity

Neural Design Network: Graphic Layout Generation with Constraints

Graphic design is essential for visual communication with layouts being fundamental to composing attractive designs. Layout generation differs from pixel-level image synthesis and is unique in terms of the requirement of mutual relations among the desired components. We propose a method for design layout generation that can satisfy user-specified constraints. The proposed neural design network (NDN) consists of three modules. The first module predicts a graph with complete relations from a graph with user-specified relations. The second module generates a layout from the predicted graph. Finally, the third module fine-tunes the predicted layout. Quantitative and qualitative experiments demonstrate that the generated layouts are visually similar to real design layouts. We also construct real designs based on predicted layouts for a better understanding of the visual quality. Finally, we demonstrate a practical application on layout recommendation.

Paper

🔭 @DeepGravity

#TensorFlow 2 Tutorial: Get Started in #DeepLearning With tf.keras

After completing this tutorial, you will know:

The difference between Keras and tf.keras and how to install and confirm TensorFlow is working.
The 5-step life-cycle of tf.keras models and how to use the sequential and functional APIs.
How to develop MLP, CNN, and RNN models with tf.keras for regression, classification, and time series forecasting.
How to use the advanced features of the tf.keras API to inspect and diagnose your model.
How to improve the performance of your tf.keras model by reducing overfitting and accelerating training.

#Keras

Link

🔭 @DeepGravity

White Paper: #MachineLearning for #Physics and the Physics of Learning

Paper

🔭 @DeepGravity

Best #Data #Visualization Techniques for small and large data

Data visualization is used in many areas to model complex events and visualize phenomena that cannot be observed directly, such as weather patterns, medical conditions or mathematical relationships. Here we review basic data visualization tools and techniques.

Link

🔭 @DeepGravity

#GradientBased training of #Gaussian Mixture Models in High-Dimensional Spaces

We present an approach for efficiently training Gaussian Mixture Models (GMMs) with Stochastic Gradient Descent (SGD) on large amounts of high-dimensional data (e.g., images). In such a scenario, SGD is strongly superior in terms of execution time and memory usage, although it is conceptually more complex than the traditional Expectation-Maximization (EM) algorithm. For enabling #SGD training, we propose three novel ideas: First, we show that minimizing an upper bound to the GMM log likelihood instead of the full one is feasible and numerically much more stable way in high-dimensional spaces. Secondly, we propose a new annealing procedure that prevents SGD from converging to pathological local minima. We also propose an SGD-compatible simplification to the full #GMM model based on local principal directions, which avoids excessive memory use in high-dimensional spaces due to quadratic growth of covariance matrices. Experiments on several standard image datasets show the validity of our approach, and we provide a publicly available TensorFlow implementation.

Paper

🔭 @DeepGravity