IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data

Learning from offline task demonstrations is a problem of great interest in robotics. For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task. However, leveraging a fixed batch of data can be problematic for larger datasets and longer-horizon tasks with greater variations. The data can exhibit substantial diversity and consist of suboptimal solution approaches. In this paper, we propose Implicit Reinforcement without Interaction at Scale (IRIS), a novel framework for learning from large-scale demonstration datasets. IRIS factorizes the control problem into a goal-conditioned low-level controller that imitates short demonstration sequences and a high-level goal selection mechanism that sets goals for the low-level and selectively combines parts of suboptimal solutions leading to more successful task completions. We evaluate IRIS across three datasets, including the RoboTurk Cans dataset collected by humans via crowdsourcing, and show that performant policies can be learned from purely offline learning. Additional results and videos at this https URL .

Paper

🔭 @DeepGravity

Deep Speech, a good #Persian podcasts about #AI

We will talk about #ArtificialIntelligence, #MachineLearning and DeepLearning news.

Link

🔭 @DeepGravity

Security of #DeepLearning Methodologies: Challenges and Opportunities

Despite the plethora of studies about security vulnerabilities and defenses of deep learning models, security aspects of deep learning methodologies, such as transfer learning, have been rarely studied. In this article, we highlight the security challenges and research opportunities of these methodologies, focusing on vulnerabilities and attacks unique to them.

Paper

🔭 @DeepGravity

An interesting #game :)

#UniversalPaperclips

Link to the game

It's background story

An RL bot for playing it

🔭 @DeepGravity

FootAndBall: Integrated player and ball detector

The paper describes a #deepNeuralNetwork -based detector dedicated for ball and players detection in high resolution, long shot, video recordings of soccer matches. The detector, dubbed FootAndBall, has an efficient fully convolutional architecture and can operate on input video stream with an arbitrary resolution. It produces ball confidence map encoding the position of the detected ball, player confidence map and player bounding boxes tensor encoding players' positions and bounding boxes. The network uses Feature Pyramid Network desing pattern, where lower level features with higher spatial resolution are combined with higher level features with bigger receptive field. This improves discriminability of small objects (the ball) as larger visual context around the object of interest is taken into account for the classification. Due to its specialized design, the network has two orders of magnitude less parameters than a generic deep neural network-based object detector, such as SSD or YOLO. This allows real-time processing of high resolution input video stream.

Paper

🔭 @DeepGravity

#Bayesian #deepLearning workshop talks from #NeuRIPS2019

Link

🔭 @DeepGravity

A Two-Stage Approach to #FewShotLearning for Image Recognition

This paper proposes a multi-layer neural network structure for few-shot image recognition of novel categories. The proposed multi-layer neural network architecture encodes transferable knowledge extracted from a large annotated dataset of base categories. This architecture is then applied to novel categories containing only a few samples. The transfer of knowledge is carried out at the feature-extraction and the classification levels distributed across the two training stages. In the first-training stage, we introduce the relative feature to capture the structure of the data as well as obtain a low-dimensional discriminative space. Secondly, we account for the variable variance of different categories by using a network to predict the variance of each class. Classification is then performed by computing the Mahalanobis distance to the mean-class representation in contrast to previous approaches that used the Euclidean distance. In the second-training stage, a category-agnostic mapping is learned from the mean-sample representation to its corresponding class-prototype representation. This is because the mean-sample representation may not accurately represent the novel category prototype. Finally, we evaluate the proposed network structure on four standard few-shot image recognition datasets, where our proposed few-shot learning system produces competitive performance compared to previous work. We also extensively studied and analyzed the contribution of each component of our proposed framework.

Paper

🔭 @DeepGravity

Semantic #RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions

Hierarchical Reinforcement Learning algorithms have successfully been applied to temporal credit assignment problems with sparse reward signals. However, state-of-the-art algorithms require manual specification of sub-task structures, a sample inefficient exploration phase or lack semantic interpretability. Humans, on the other hand, efficiently detect hierarchical sub-structures induced by their surroundings. It has been argued that this inference process universally applies to language, logical reasoning as well as motor control. Therefore, we propose a cognitive-inspired Reinforcement Learning architecture which uses grammar induction to identify sub-goal policies. By treating an on-policy trajectory as a sentence sampled from the policy-conditioned language of the environment, we identify hierarchical constituents with the help of unsupervised grammatical inference. The resulting set of temporal abstractions is called action grammar (Pastra & Aloimonos, 2012) and unifies symbolic and connectionist approaches to Reinforcement Learning. It can be used to facilitate efficient imitation, transfer and online learning.

Paper

🔭 @DeepGravity

#Reinforcement Learning as Probabilistic Modelling: A #VariationalInference Formulation

Reinforcement Learning is concerned with an agent attempting to acquire optimal behaviour in unknown environments that typically exhibit stochasticity. Though minimally supervised, reinforcement learning algorithms have shown numerous success ranging from solving ATARI games using Deep Q-Networks, to the triumphant victory against the world champions in the game of GO, and recently in Start Craft.

Article

🔭 @DeepGravity

#NeurIPS2019 Workshop on #MachineLearning for #Creativity and Design

Link

🔭 @DeepGravity

A webinar summarizing lessons learned in #AI implementation from the 2019 Winning With AI Report.

Link

🔭 @DeepGravity

BMW InnovationLab:
#BMW shares #AI algorithms used in production, available on GitHub

Link

🔭 @DeepGravity

#Autograd
Autograd can automatically differentiate native #Python and #Numpy code.

#Google #JAX
JAX is Autograd and XLA, brought together for high-performance machine learning research.

🔭 @DeepGravity

The year in AI: 2019 #ML / #AI advances recap

It has become somewhat of a tradition for me to do an end-of-year retrospective of advances in AI/ML (see last year’s round up for example), so here we go again! This year started with a big recognition to the impact of #DeepLearning when #Hinton, #Bengio, and #Lecun were awarded the #Turing award.

Link

🔭 @DeepGravity

#NeuralNetworks Timeline

🔭 @DeepGravity

A good Telegram channel, managed by an Iranian researcher, covering some new papers in #AI

Link to the channel

🔭 @DeepGravity

#Fun

🔭 @DeepGravity

Our reinforcement learning architect designs have been just published on #NeurIPS2019 AI Art Gallery:

https://lnkd.in/dFZ37BN

Seems it draws like a baby now, but is growing and hopefully would be a skillful #RL artist very soon.

#reinforcementlearning #deeplearning #ai #artificialintelligence #art #deepreinforcementlearning #creativeart #neurips

🔭 @DeepGravity

#NeurIPS2019:

Lectures

Papers

🔭 @DeepGravity

P-CapsNets: a General Form of #ConvolutionalNeuralNetworks

We propose Pure CapsNets (P-CapsNets) which is a generation of normal CNNs structurally. Specifically, we make three modifications to current CapsNets. First, we remove routing procedures from CapsNets based on the observation that the coupling coefficients can be learned implicitly. Second, we replace the convolutional layers in CapsNets to improve efficiency. Third, we package the capsules into rank-3 tensors to further improve efficiency. The experiment shows that P-CapsNets achieve better performance than CapsNets with varied routing procedures by using significantly fewer parameters on MNIST&CIFAR10. The high efficiency of P-CapsNets is even comparable to some deep compressing models. For example, we achieve more than 99% percent accuracy on MNIST by using only 3888 parameters. We visualize the capsules as well as the corresponding correlation matrix to show a possible way of initializing CapsNets in the future. We also explore the adversarial robustness of P-CapsNets compared to #CNNs.

Paper

🔭 @DeepGravity

Seven differences between academia and industry for building machine learning and #deepLearning models

1) Approach to accuracy
2) Training vs serving
3) Emphasis on Engineering
4) Less emphasis on larger models
5) Understanding the baseline
6) Understanding the intricacies of data
7) Focusing on deep learning too early

Link

🔭 @DeepGravity