🔹 Something that bothers me about deep neural nets
John D. Cook, PhD
https://www.johndcook.com/blog/2017/10/09/something-that-bothers-me-about-deep-neural-nets/
Overfitting happens when a model does too good a job of matching a particular data set and so does a poor job on new data. The way traditional statistical models address the danger of overfitting is to limit the number of parameters. For example, you might fit a straight line (two parameters) to 100 data points, rather than using a 99-degree polynomial that could match the input data exactly and probably do a terrible job on new data. You find the best fit you can to a model that doesn’t have enough flexibility to match the data too closely.
Deep neural networks have enough parameters to overfit the data, but there are various strategies to keep this from happening. A common way to avoid overfitting is to deliberately do a mediocre job of fitting the model.
When it works well, the shortcomings of the optimization procedure yield a solution that differs from the optimal solution in a beneficial way. But the solution could fail to be useful in several ways. It might be too far from optimal, or deviate from the optimal solution in an unhelpful way, or the optimization method might accidentally do too good a job.
It a nutshell, the disturbing thing is that you have a negative criteria for what constitutes a good solution: one that’s not too close to optimal. But there are a lot of ways to not be too close to optimal. In practice, you experiment until you find an optimally suboptimal solution, i.e. the intentionally suboptimal fit that performs the best in validation.
John D. Cook, PhD
https://www.johndcook.com/blog/2017/10/09/something-that-bothers-me-about-deep-neural-nets/
Overfitting happens when a model does too good a job of matching a particular data set and so does a poor job on new data. The way traditional statistical models address the danger of overfitting is to limit the number of parameters. For example, you might fit a straight line (two parameters) to 100 data points, rather than using a 99-degree polynomial that could match the input data exactly and probably do a terrible job on new data. You find the best fit you can to a model that doesn’t have enough flexibility to match the data too closely.
Deep neural networks have enough parameters to overfit the data, but there are various strategies to keep this from happening. A common way to avoid overfitting is to deliberately do a mediocre job of fitting the model.
When it works well, the shortcomings of the optimization procedure yield a solution that differs from the optimal solution in a beneficial way. But the solution could fail to be useful in several ways. It might be too far from optimal, or deviate from the optimal solution in an unhelpful way, or the optimization method might accidentally do too good a job.
It a nutshell, the disturbing thing is that you have a negative criteria for what constitutes a good solution: one that’s not too close to optimal. But there are a lot of ways to not be too close to optimal. In practice, you experiment until you find an optimally suboptimal solution, i.e. the intentionally suboptimal fit that performs the best in validation.
Johndcook
Something that bothers me about deep neural networks
Deep learning depends on not solving an optimization problem too well.
Forwarded from رادیوفیزیک 📣
Audio
Abbas Karimi
صوت سخنرانی عباس کریمی با موضوع:
🎧 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
🎧 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
💲 PhD studentships in Statistical Applied Mathematics at the University of Bath
www.bath.ac.uk/samba
https://www.findaphd.com/search/PhDDetails.aspx?CAID=2444
www.bath.ac.uk/samba
https://www.findaphd.com/search/PhDDetails.aspx?CAID=2444
⌨️ Starts October 10th
Advanced MATLAB for Scientific Computing.
Sign up/learn more about advanced graphics and tools.
http://online.stanford.edu/course/advanced-matlab-scientific-computing
Advanced MATLAB for Scientific Computing.
Sign up/learn more about advanced graphics and tools.
http://online.stanford.edu/course/advanced-matlab-scientific-computing
📄 Novel and topical business news and their impact on stock market activity
Takayuki Mizuno, Takaaki Ohnishi and Tsutomu Watanabe
🔗 https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-017-0123-7
📌 ABSTRACT
We propose an indicator to measure the degree to which a particular news article is novel, as well as an indicator to measure the degree to which a particular news item attracts attention from investors. The novelty measure is obtained by comparing the extent to which a particular news article is similar to earlier news articles, and an article is regarded as novel if there was no similar article before it. On the other hand, we say a news item receives a lot of attention and thus is highly topical if it is simultaneously reported by many news agencies and read by many investors who receive news from those agencies. The topicality measure for a news item is obtained by counting the number of news articles whose content is similar to an original news article but which are delivered by other news agencies. To check the performance of the indicators, we empirically examine how these indicators are correlated with intraday financial market indicators such as the number of transactions and price volatility. Specifically, we use a dataset consisting of over 90 million business news articles reported in English and a dataset consisting of minute-by-minute stock prices on the New York Stock Exchange and the NASDAQ Stock Market from 2003 to 2014, and show that price volatility, transaction volumes, and the number of transactions exhibited a significant response to a news article when it was novel and topical.
Keywords
#novelty #topicality #exogenous #shocks #financial_markets #business_news
Takayuki Mizuno, Takaaki Ohnishi and Tsutomu Watanabe
🔗 https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-017-0123-7
📌 ABSTRACT
We propose an indicator to measure the degree to which a particular news article is novel, as well as an indicator to measure the degree to which a particular news item attracts attention from investors. The novelty measure is obtained by comparing the extent to which a particular news article is similar to earlier news articles, and an article is regarded as novel if there was no similar article before it. On the other hand, we say a news item receives a lot of attention and thus is highly topical if it is simultaneously reported by many news agencies and read by many investors who receive news from those agencies. The topicality measure for a news item is obtained by counting the number of news articles whose content is similar to an original news article but which are delivered by other news agencies. To check the performance of the indicators, we empirically examine how these indicators are correlated with intraday financial market indicators such as the number of transactions and price volatility. Specifically, we use a dataset consisting of over 90 million business news articles reported in English and a dataset consisting of minute-by-minute stock prices on the New York Stock Exchange and the NASDAQ Stock Market from 2003 to 2014, and show that price volatility, transaction volumes, and the number of transactions exhibited a significant response to a news article when it was novel and topical.
Keywords
#novelty #topicality #exogenous #shocks #financial_markets #business_news
SpringerOpen
Novel and topical business news and their impact on stock market activity - EPJ Data Science
We propose an indicator to measure the degree to which a particular news article is novel, as well as an indicator to measure the degree to which a particular news item attracts attention from investors. The novelty measure is obtained by comparing the extent…
#سمینارهای_هفتگی گروه سیستمهای پیچیده و علم شبکه دانشگاه شهید بهشتی
🔹دوشنبه، ۲۴ مهرماه، ساعت ۴:۰۰ - کلاس۱ دانشکده فیزیک دانشگاه شهید بهشتی.
@carimi
🔹دوشنبه، ۲۴ مهرماه، ساعت ۴:۰۰ - کلاس۱ دانشکده فیزیک دانشگاه شهید بهشتی.
@carimi
Forwarded from رادیوفیزیک 📣
Media is too big
VIEW IN TELEGRAM
فیلم ارائه عباس کریمی (قسمت اول):
🎞 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
🎙 رادیوفیزیک
@radiophysicsir
🎞 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
🎙 رادیوفیزیک
@radiophysicsir
Forwarded from رادیوفیزیک 📣
Media is too big
VIEW IN TELEGRAM
فیلم ارائه عباس کریمی (قسمت دوم):
🎞 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
🎙 رادیوفیزیک
@radiophysicsir
🎞 Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models
🎙 رادیوفیزیک
@radiophysicsir
New python book for #networkanalysis case studies (Panama papers, Wiki, trauma networks, products, etc)
https://pragprog.com/book/dzcnapy/complex-network-analysis-in-python
https://pragprog.com/book/dzcnapy/complex-network-analysis-in-python
This media is not supported in your browser
VIEW IN TELEGRAM
Complex natural systems defy standard mathematical analysis, so one ecologist is throwing out the equations.
https://www.quantamagazine.org/chaos-theory-in-ecology-predicts-future-populations-20151013
https://www.quantamagazine.org/chaos-theory-in-ecology-predicts-future-populations-20151013
🛒 Preliminary Steps Toward a Universal Economic Dynamics for Monetary and Fiscal Policy
http://necsi.edu/research/economics/econuniversal
📌 ABSTRACT
We consider the relationship between economic activity and intervention, including monetary and fiscal policy, using a universal monetary and response dynamics framework. Central bank policies are designed for economic growth without excess inflation. However, unemployment, investment, consumption, and inflation are interlinked. Understanding dynamics is crucial to assessing the effects of policy, especially in the aftermath of the recent financial crisis. Here we lay out a program of research into monetary and economic dynamics and preliminary steps toward its execution. We use general principles of response theory to derive specific implications for policy. We find that the current approach, which considers the overall supply of money to the economy, is insufficient to effectively regulate economic growth. While it can achieve some degree of control, optimizing growth also requires a fiscal policy balancing monetary injection between two dominant loop flows, the consumption and wages loop, and investment and returns loop. The balance arises from a composite of government tax, ennoscriptment, subsidy policies, corporate policies, as well as monetary policy. We further show that empirical evidence is consistent with a transition in 1980 between two regimes—from an oversupply to the consumption and wages loop, to an oversupply of the investment and returns loop. The imbalance is manifest in savings and borrowing by consumers and investors, and in inflation. The latter followed an increasing trend until 1980, and a decreasing one since then, resulting in a zero interest rate largely unrelated to the financial crisis. Three recessions and the financial crisis are part of this dynamic. Optimizing growth now requires shifting the balance. Our analysis supports advocates of greater income and / or government support for the poor who use a larger fraction of income for consumption. This promotes investment due to the growth in expenditures. Otherwise, investment has limited opportunities to gain returns above inflation so capital remains uninvested, and does not contribute to the growth of economic activity.
http://necsi.edu/research/economics/econuniversal
📌 ABSTRACT
We consider the relationship between economic activity and intervention, including monetary and fiscal policy, using a universal monetary and response dynamics framework. Central bank policies are designed for economic growth without excess inflation. However, unemployment, investment, consumption, and inflation are interlinked. Understanding dynamics is crucial to assessing the effects of policy, especially in the aftermath of the recent financial crisis. Here we lay out a program of research into monetary and economic dynamics and preliminary steps toward its execution. We use general principles of response theory to derive specific implications for policy. We find that the current approach, which considers the overall supply of money to the economy, is insufficient to effectively regulate economic growth. While it can achieve some degree of control, optimizing growth also requires a fiscal policy balancing monetary injection between two dominant loop flows, the consumption and wages loop, and investment and returns loop. The balance arises from a composite of government tax, ennoscriptment, subsidy policies, corporate policies, as well as monetary policy. We further show that empirical evidence is consistent with a transition in 1980 between two regimes—from an oversupply to the consumption and wages loop, to an oversupply of the investment and returns loop. The imbalance is manifest in savings and borrowing by consumers and investors, and in inflation. The latter followed an increasing trend until 1980, and a decreasing one since then, resulting in a zero interest rate largely unrelated to the financial crisis. Three recessions and the financial crisis are part of this dynamic. Optimizing growth now requires shifting the balance. Our analysis supports advocates of greater income and / or government support for the poor who use a larger fraction of income for consumption. This promotes investment due to the growth in expenditures. Otherwise, investment has limited opportunities to gain returns above inflation so capital remains uninvested, and does not contribute to the growth of economic activity.
💡For the first time, researchers have experimentally probed topological order and its breakdown. The work could open the way for new approaches to quantum computation.
https://insidetheperimeter.ca/experiment-sneaks-peek-quantum-world/?utm_content=61782672&utm_medium=social&utm_source=twitter
https://insidetheperimeter.ca/experiment-sneaks-peek-quantum-world/?utm_content=61782672&utm_medium=social&utm_source=twitter
Inside The Perimeter
Experiment sneaks a peek at quantum world -- Inside the Perimeter
Researchers have experimentally probed topological order and its breakdown. The work could open the way for new approaches to quantum computation.
🎯 There’s a #law_of_large numbers, a #law_of_small_numbers, and a #law_of_medium_numbers in between.
1️⃣ The law of large numbers is a mathematical theorem. It describes what happens as you average more and more random variables. (goo.gl/9BFtxV)
2️⃣ The law of small numbers is a semi-serious statement about about how people underestimate the variability of the average of a small number of random variables. ( In general, people grossly underestimate the variability in small samples. This phenomenon comes up all the time)
3️⃣ The law of medium numbers is a term coined by Gerald Weinberg in his book An Introduction to General Systems Thinking. He states the law as follows.
"For medium number systems, we can expect that large fluctuations, irregularities, and discrepancy with any theory will occur more or less regularly."
The law of medium numbers applies to systems too large to study exactly and too small to study statistically. For example, it may be easier to understand the behavior of an individual or a nation than the dynamics of a small community. Atoms are simple, and so are stars, but medium-sized things like birds are complicated. Medium-sized systems are where you see chaos.
Weinberg warns that medium-sized systems challenge science because scientific disciplines define their boundaries by the set of problems they can handle. He says, for example, that
"Mechanics, then, is the study of those systems for which the approximations of mechanics work successfully."
He warns that we should not be mislead by a discipline’s “success with systems of its own choosing.”
Weinberg’s book was written in 1975. Since that time there has been much more interest in the emergent properties of medium-sized systems that are not explained by more basic sciences. We may not understand these systems well, but we may appreciate the limits of our understanding better than we did a few decades ago.
1️⃣ The law of large numbers is a mathematical theorem. It describes what happens as you average more and more random variables. (goo.gl/9BFtxV)
2️⃣ The law of small numbers is a semi-serious statement about about how people underestimate the variability of the average of a small number of random variables. ( In general, people grossly underestimate the variability in small samples. This phenomenon comes up all the time)
3️⃣ The law of medium numbers is a term coined by Gerald Weinberg in his book An Introduction to General Systems Thinking. He states the law as follows.
"For medium number systems, we can expect that large fluctuations, irregularities, and discrepancy with any theory will occur more or less regularly."
The law of medium numbers applies to systems too large to study exactly and too small to study statistically. For example, it may be easier to understand the behavior of an individual or a nation than the dynamics of a small community. Atoms are simple, and so are stars, but medium-sized things like birds are complicated. Medium-sized systems are where you see chaos.
Weinberg warns that medium-sized systems challenge science because scientific disciplines define their boundaries by the set of problems they can handle. He says, for example, that
"Mechanics, then, is the study of those systems for which the approximations of mechanics work successfully."
He warns that we should not be mislead by a discipline’s “success with systems of its own choosing.”
Weinberg’s book was written in 1975. Since that time there has been much more interest in the emergent properties of medium-sized systems that are not explained by more basic sciences. We may not understand these systems well, but we may appreciate the limits of our understanding better than we did a few decades ago.
Wikipedia
Law of large numbers
In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close…