Empirically, XLNet outperforms BERT on 20 tasks, often by a large margin, and achieves state-of-the-art results on 18 tasks including question answering, natural language inference, sentiment analysis, and document ranking.
https://arxiv.org/abs/1906.08237#
#NLP
https://arxiv.org/abs/1906.08237#
#NLP
arXiv.org
XLNet: Generalized Autoregressive Pretraining for Language Understanding
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language...
A great introduction to Multivariate Gaussian Distributations:
https://www.youtube.com/watch?v=eho8xH3E6mE
#statistics
https://www.youtube.com/watch?v=eho8xH3E6mE
#statistics
YouTube
Multivariate Gaussian distributions
Properties of the multivariate Gaussian probability distribution
A great introduction to Multivariate Gaussian Distributations:
https://www.youtube.com/watch?v=JNlEIEwe-Cg
#statistics
https://www.youtube.com/watch?v=JNlEIEwe-Cg
#statistics
YouTube
Gaussian Mixture Models - The Math of Intelligence (Week 7)
We're going to predict customer churn using a clustering technique called the Gaussian Mixture Model! This is a probability distribution that consists of multiple Gaussian distributions, very cool. I also have something important but unrelated to say in the…
Forwarded from Machine learning books and papers (Ramin Mousa)
Adapters: A Compact and Extensible Transfer Learning Method for NLP
____
@Machine_learn
___
https://medium.com/dair-ai/adapters-a-compact-and-extensible-transfer-learning-method-for-nlp-6d18c2399f62
____
@Machine_learn
___
https://medium.com/dair-ai/adapters-a-compact-and-extensible-transfer-learning-method-for-nlp-6d18c2399f62
Medium
Adapters: A Compact and Extensible Transfer Learning Method for NLP
Adapters obtain comparable results to BERT on several NLP tasks while achieving parameter efficiency.
Forwarded from School of AI
Neural Code Search: ML-based code search using natural language queries:
https://ai.facebook.com/blog/neural-code-search-ml-based-code-search-using-natural-language-queries/
https://ai.facebook.com/blog/neural-code-search-ml-based-code-search-using-natural-language-queries/
Facebook
Neural Code Search: ML-based code search using natural language queries
We’ve developed an internal tool that applies natural language processing and information retrieval techniques directly to source code text, in order to produce an machine learning-based code search system.
A great tutorial which shows you how you can implement algorithms above, in Python:
https://github.com/MorvanZhou/Evolutionary-Algorithm
https://github.com/MorvanZhou/Evolutionary-Algorithm
Deep unsupervised Learning Course by Google:
https://sites.google.com/view/berkeley-cs294-158-sp19/home
https://sites.google.com/view/berkeley-cs294-158-sp19/home
Google
CS294-158-SP19 Deep Unsupervised Learning Spring 2019
About: This course will cover two areas of deep learning in which labeled data is not required: Deep Generative Models and Self-supervised Learning. Recent advances in generative models have made it possible to realistically model high-dimensional raw data…
Forwarded from Tensorflow(@CVision) (Vahid Reza Khazaie)
New Google Brain Optimizer Reduces BERT Pre-Training Time From Days to Minutes
کاهش مدت زمان pre-training مدل زبانی BERT از سه روز به 76 دقیقه با ارائه یک تابع بهینه ساز جدید!
Google Brain researchers have proposed LAMB (Layer-wise Adaptive Moments optimizer for Batch training), a new optimizer which reduces training time for its NLP training model BERT (Bidirectional Encoder Representations from Transformers) from three days to just 76 minutes.
لینک مقاله: https://arxiv.org/abs/1904.00962
لینک بلاگ پست: https://medium.com/syncedreview/new-google-brain-optimizer-reduces-bert-pre-training-time-from-days-to-minutes-b454e54eda1d
#BERT #language_model #optimizer
کاهش مدت زمان pre-training مدل زبانی BERT از سه روز به 76 دقیقه با ارائه یک تابع بهینه ساز جدید!
Google Brain researchers have proposed LAMB (Layer-wise Adaptive Moments optimizer for Batch training), a new optimizer which reduces training time for its NLP training model BERT (Bidirectional Encoder Representations from Transformers) from three days to just 76 minutes.
لینک مقاله: https://arxiv.org/abs/1904.00962
لینک بلاگ پست: https://medium.com/syncedreview/new-google-brain-optimizer-reduces-bert-pre-training-time-from-days-to-minutes-b454e54eda1d
#BERT #language_model #optimizer
arXiv.org
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization methods to tackle...
Forwarded from Hidden layer
Alternatives to Kaggle/other sites for machine learning & competitions.
Kaggle is the best, but there are a few alternatives:
https://challenger.ai
https://www.crowdai.org
https://www.drivendata.org
https://www.kdd.org
http://codalab.org/
https://www.crowdanalytix.co
https://www.datasciencechallenge.org
https://www.analyticsvidhya.com
https://www.topcoder.com
https://www.kesci.com
https://www.datafountain.cn
https://grand-challenge.org
https://signate.jp
http://www.dcjingsai.com
https://www.innocentive.com
#engineeringML
Kaggle is the best, but there are a few alternatives:
https://challenger.ai
https://www.crowdai.org
https://www.drivendata.org
https://www.kdd.org
http://codalab.org/
https://www.crowdanalytix.co
https://www.datasciencechallenge.org
https://www.analyticsvidhya.com
https://www.topcoder.com
https://www.kesci.com
https://www.datafountain.cn
https://grand-challenge.org
https://signate.jp
http://www.dcjingsai.com
https://www.innocentive.com
#engineeringML
The Thousand Brains Theory: A Framework for Understanding the Neocortex and Building Intelligent Machines
https://www.youtube.com/watch?v=5LFo36g4Lug
#neuroscience #AI #numenta
https://www.youtube.com/watch?v=5LFo36g4Lug
#neuroscience #AI #numenta
YouTube
The Thousand Brains Theory
The Thousand Brains Theory: A Framework for Understanding the Neocortex and Building Intelligent Machines
Recent advances in reverse engineering the neocortex reveal that it is a highly-distributed sensory-motor modeling system. Each cortical column learns…
Recent advances in reverse engineering the neocortex reveal that it is a highly-distributed sensory-motor modeling system. Each cortical column learns…
Forwarded from Tensorflow(@CVision) (Vahid Reza Khazaie)
طبقه بندی متن با استفاده از Flair
Flair delivers state-of-the-art performance in solving NLP problems such as named entity recognition (NER), part-of-speech tagging (PoS), sense disambiguation and text classification. It’s an NLP framework built on top of PyTorch.
لینک پست: https://towardsdatascience.com/text-classification-with-state-of-the-art-nlp-library-flair-b541d7add21f
کد: https://github.com/zalandoresearch/flair
#text_classification #nlp #flair
Flair delivers state-of-the-art performance in solving NLP problems such as named entity recognition (NER), part-of-speech tagging (PoS), sense disambiguation and text classification. It’s an NLP framework built on top of PyTorch.
لینک پست: https://towardsdatascience.com/text-classification-with-state-of-the-art-nlp-library-flair-b541d7add21f
کد: https://github.com/zalandoresearch/flair
#text_classification #nlp #flair
Medium
Text Classification with State of the Art NLP Library — Flair
Exciting news! A new version of Flair - state-of-the-art NLP library has just been released. Learn how to use it for text classification
Forwarded from AI in Science & Technology