AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges (Salesforce AI)
A review of the AIOps vision, trends challenges and opportunities, specifically focusing on the underlying AI techniques.
1. INTRODUCTION
2. CONTRIBUTION OF THIS SURVEY
3. DATA FOR AIOPS
A. Metrics
B. Logs
C. Traces
D. Other data
4. INCIDENT DETECTION
A. Metrics based Incident Detection
B. Logs based Incident Detection
C. Traces and Multimodal Incident Detection
5. FAILURE PREDICTION
A. Metrics based Failure Prediction
B. Logs based Incident Detection
6. ROOT CAUSE ANALYSIS
A. Metric-based RCA
B. Log-based RCA
C. Trace-based and Multimodal RCA
7. AUTOMATED ACTIONS
A. Automated Remediation
B. Auto-scaling
C. Resource Management
8. FUTURE OF AIOPS
A. Common AI Challenges for AIOps
B. Opportunities and Future Trends
9. CONCLUSION
A review of the AIOps vision, trends challenges and opportunities, specifically focusing on the underlying AI techniques.
1. INTRODUCTION
2. CONTRIBUTION OF THIS SURVEY
3. DATA FOR AIOPS
A. Metrics
B. Logs
C. Traces
D. Other data
4. INCIDENT DETECTION
A. Metrics based Incident Detection
B. Logs based Incident Detection
C. Traces and Multimodal Incident Detection
5. FAILURE PREDICTION
A. Metrics based Failure Prediction
B. Logs based Incident Detection
6. ROOT CAUSE ANALYSIS
A. Metric-based RCA
B. Log-based RCA
C. Trace-based and Multimodal RCA
7. AUTOMATED ACTIONS
A. Automated Remediation
B. Auto-scaling
C. Resource Management
8. FUTURE OF AIOPS
A. Common AI Challenges for AIOps
B. Opportunities and Future Trends
9. CONCLUSION
👍1
Technical Report: Evaluation of ChatGPT Model for Vulnerability Detection
Authors found that current GPT-3 and ChatGPT capabilities for effectively detecting vulnerabilities in code are limited. While natural language processing models have demonstrated impressive results in numerous areas, their application in vulnerability detection tasks requires further refinement and investigation.
Authors found that current GPT-3 and ChatGPT capabilities for effectively detecting vulnerabilities in code are limited. While natural language processing models have demonstrated impressive results in numerous areas, their application in vulnerability detection tasks requires further refinement and investigation.
Forwarded from Consciousnesses
Superintelligence
Discussion with philosopher David Chalmers and his fellow experts on the concepts of consciousness, intelligence, and the possibility that we are living in a simulated universe. They delve into the works of Douglas Hofstadter, the idea of an intelligence explosion, and the challenge of aligning artificial general intelligence with human goals. The conversation also touches on the limitations of intelligence, the relationship between complexity and consciousness, and the potential motivations behind simulating a universe.
Table of Contents:
- Introduction to David Chalmers and his work
- The influence of Douglas Hofstadter on AI and philosophy
- The concept of the intelligence explosion
- Aligning artificial general intelligence with human goals
- Consciousness, introspection, and the meta problem
- The relationship between complexity and consciousness
- What makes a simulation interesting?
Discussion with philosopher David Chalmers and his fellow experts on the concepts of consciousness, intelligence, and the possibility that we are living in a simulated universe. They delve into the works of Douglas Hofstadter, the idea of an intelligence explosion, and the challenge of aligning artificial general intelligence with human goals. The conversation also touches on the limitations of intelligence, the relationship between complexity and consciousness, and the potential motivations behind simulating a universe.
Table of Contents:
- Introduction to David Chalmers and his work
- The influence of Douglas Hofstadter on AI and philosophy
- The concept of the intelligence explosion
- Aligning artificial general intelligence with human goals
- Consciousness, introspection, and the meta problem
- The relationship between complexity and consciousness
- What makes a simulation interesting?
Forwarded from Consciousnesses
OpenAI’s CEO Says the Age of Giant AI Models Is Already Over
OpenAI’s CEO warned that the research strategy that birthed ChatGPT is played out. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.”
Altman’s statement suggests that GPT-4 could be the last major advance to emerge from OpenAI’s strategy of making the models bigger and feeding them more data.
Nick Frosst, a cofounder at Cohere who previously worked on AI at Google, says Altman’s feeling that going bigger will not work indefinitely rings true. He, too, believes that progress on transformers, the type of machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling. “There are lots of ways of making transformers way, way better and more useful, and lots of them don’t involve adding parameters to the model.”
OpenAI’s CEO warned that the research strategy that birthed ChatGPT is played out. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.”
Altman’s statement suggests that GPT-4 could be the last major advance to emerge from OpenAI’s strategy of making the models bigger and feeding them more data.
Nick Frosst, a cofounder at Cohere who previously worked on AI at Google, says Altman’s feeling that going bigger will not work indefinitely rings true. He, too, believes that progress on transformers, the type of machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling. “There are lots of ways of making transformers way, way better and more useful, and lots of them don’t involve adding parameters to the model.”
AI / ML / LLM / Transformer Models Timeline
This is a collection of important papers in the area of LLMs and Transformer models.
PDF file.
This is a collection of important papers in the area of LLMs and Transformer models.
PDF file.
Forwarded from Consciousnesses
The Anatomy of Autonomy: Why Agents are the next AI Killer App after ChatGPT
GPTs are General Purpose Technologies, but every GPT needs a killer app. The fifth killer app is here, and it is Autonomous Agents.
GPTs are General Purpose Technologies, but every GPT needs a killer app. The fifth killer app is here, and it is Autonomous Agents.
Bard now helps you code
Bard can help with programming and software development tasks, including code generation, debugging and code explanation in more than 20 programming languages including C++, Go, Java, Javanoscript, Python and Typenoscript. And you can easily export Python code to Google Colab — no copy and paste required. Bard can also assist with writing functions for Google Sheets.
Bard can help with programming and software development tasks, including code generation, debugging and code explanation in more than 20 programming languages including C++, Go, Java, Javanoscript, Python and Typenoscript. And you can easily export Python code to Google Colab — no copy and paste required. Bard can also assist with writing functions for Google Sheets.
👍3
Systemization of Knowledge: Machine Learning for Continuous Integration
This paper reports an systemization of knowledge of different aspects of the use of ML for CI. The systematic analysis highlights the deficiencies of the existing ML-based solutions that can be improved for advancing the state-of-the-art.
- RQ1: What are the CI phases that have been addressed by ML-based solutions?
- RQ2: What are the techniques employed in the development of state-of-the-art ML solutions for each CI phase?
This paper reports an systemization of knowledge of different aspects of the use of ML for CI. The systematic analysis highlights the deficiencies of the existing ML-based solutions that can be improved for advancing the state-of-the-art.
- RQ1: What are the CI phases that have been addressed by ML-based solutions?
- RQ2: What are the techniques employed in the development of state-of-the-art ML solutions for each CI phase?
ICAT'23: 11th International Conference on Advanced Technologies
11th International Conference on Advanced Technologies (ICAT'23) will be organized in Istanbul Ticaret University, Istanbul, Turkey on August 17-19, 2023.
The main objective of ICAT'23 is to present the latest research and results of scientists related to Computer Sciences, Electrical and Electronics, Energy Technologies, Manufacturing Technologies, Mechatronics and Biomedical Technologies. This conference provides opportunities for the different areas delegates to exchange new ideas and application experiences face to face, to establish business or research relations and to find global partners for future collaboration.
All paper submissions will be double-blind and peer-reviewed and evaluated based on originality, technical and/or research content/depth, correctness, relevance to conference, contributions, and readability.
DEADLINES :
- Submission Abstract: 2023 June 1
- Notification of acceptance: Within 7-10 days after submission
11th International Conference on Advanced Technologies (ICAT'23) will be organized in Istanbul Ticaret University, Istanbul, Turkey on August 17-19, 2023.
The main objective of ICAT'23 is to present the latest research and results of scientists related to Computer Sciences, Electrical and Electronics, Energy Technologies, Manufacturing Technologies, Mechatronics and Biomedical Technologies. This conference provides opportunities for the different areas delegates to exchange new ideas and application experiences face to face, to establish business or research relations and to find global partners for future collaboration.
All paper submissions will be double-blind and peer-reviewed and evaluated based on originality, technical and/or research content/depth, correctness, relevance to conference, contributions, and readability.
DEADLINES :
- Submission Abstract: 2023 June 1
- Notification of acceptance: Within 7-10 days after submission
Visual Blocks for ML: Accelerating machine learning prototyping with interactive tools
Paper describes a visual programming platform for rapid and iterative development of end-to-end ML-based multimedia applications. Visual Blocks for ML, formerly called Rapsai, provides a no-code graph building experience through its node-graph editor. Users can create and connect different components (nodes) to rapidly build an ML pipeline, and see the results in real-time without writing any code.
Paper describes a visual programming platform for rapid and iterative development of end-to-end ML-based multimedia applications. Visual Blocks for ML, formerly called Rapsai, provides a no-code graph building experience through its node-graph editor. Users can create and connect different components (nodes) to rapidly build an ML pipeline, and see the results in real-time without writing any code.
ChatGPT could cost over $700,000 per day to operate
- ChatGPT could cost OpenAI up to $700,000 a day to run due to "expensive servers".
- ChatGPT requires massive amounts of computing power on expensive servers to answer queries.
- Microsoft is secretly building an AI chip to reduce the cost.
- ChatGPT could cost OpenAI up to $700,000 a day to run due to "expensive servers".
- ChatGPT requires massive amounts of computing power on expensive servers to answer queries.
- Microsoft is secretly building an AI chip to reduce the cost.
Finetuning Large Language Models
Fine-tuning all layers of a pretrained LLM remains the gold standard for adapting to new target tasks, but there are several efficient alternatives for using pretrained transformers. Methods such as feature-based approaches, in-context learning, and parameter-efficient finetuning techniques enable effective application of LLMs to new tasks while minimizing computational costs and resources.
- In-Context Learning and Indexing
- The 3 Conventional Feature-Based and Finetuning Approaches
- Feature-Based Approach
- Finetuning I – Updating The Output Layers
- Finetuning II – Updating All Layers
- Parameter-Efficient Finetuning
- Reinforcement Learning with Human Feedback
- Conclusion
Fine-tuning all layers of a pretrained LLM remains the gold standard for adapting to new target tasks, but there are several efficient alternatives for using pretrained transformers. Methods such as feature-based approaches, in-context learning, and parameter-efficient finetuning techniques enable effective application of LLMs to new tasks while minimizing computational costs and resources.
- In-Context Learning and Indexing
- The 3 Conventional Feature-Based and Finetuning Approaches
- Feature-Based Approach
- Finetuning I – Updating The Output Layers
- Finetuning II – Updating All Layers
- Parameter-Efficient Finetuning
- Reinforcement Learning with Human Feedback
- Conclusion
👍2
ENASE 2024
The mission of ENASE (Evaluation of Novel Approaches to Software Engineering) is to be a prime international forum to discuss and publish research findings and IT industry experiences with relation to novel approaches to software engineering.
DEADLINES:
Regular Paper Submission: December 13, 2023
Position Paper Submission: January 25, 2024
Doctoral Consortium Paper Submission: March 6, 2024
The mission of ENASE (Evaluation of Novel Approaches to Software Engineering) is to be a prime international forum to discuss and publish research findings and IT industry experiences with relation to novel approaches to software engineering.
DEADLINES:
Regular Paper Submission: December 13, 2023
Position Paper Submission: January 25, 2024
Doctoral Consortium Paper Submission: March 6, 2024
Long-term Forecasting with TiDE: Time-series Dense Encoder
Authors propose a MLP-based encoder-decoder model, Time-series Dense Encoder (TiDE), for long-term time-series forecasting that enjoys the simplicity and speed of linear models while also being able to handle covariates and non-linear dependencies. They prove that the simplest linear analogue of the model can achieve near optimal error rate for linear dynamical systems under some assumptions. The method can match or outperform prior approaches on popular long-term time-series forecasting benchmarks while being 5-10x faster than the best Transformer based.
Google AI Blog
Authors propose a MLP-based encoder-decoder model, Time-series Dense Encoder (TiDE), for long-term time-series forecasting that enjoys the simplicity and speed of linear models while also being able to handle covariates and non-linear dependencies. They prove that the simplest linear analogue of the model can achieve near optimal error rate for linear dynamical systems under some assumptions. The method can match or outperform prior approaches on popular long-term time-series forecasting benchmarks while being 5-10x faster than the best Transformer based.
Google AI Blog
ITER: Iterative Neural Repair for Multi-Location Patches
In this paper, an iterative program repair paradigm called ITER is proposed. It is founded on the concept of improving partial patches until they become plausible and correct:
- ITER iteratively improves partial single-location patches by fixing compilation errors and further refining the previously generated code.
- ITER iteratively improves partial patches to construct multi-location patches, with fault localization re-execution.
ITER is implemented for Java based on battle-proven deep neural networks and code representation and is evaluated on 476 bugs from 10 open-source projects in Defects4J 2.0.
In this paper, an iterative program repair paradigm called ITER is proposed. It is founded on the concept of improving partial patches until they become plausible and correct:
- ITER iteratively improves partial single-location patches by fixing compilation errors and further refining the previously generated code.
- ITER iteratively improves partial patches to construct multi-location patches, with fault localization re-execution.
ITER is implemented for Java based on battle-proven deep neural networks and code representation and is evaluated on 476 bugs from 10 open-source projects in Defects4J 2.0.
👍1
A Survey of Trojans in Neural Models of Source Code: Taxonomy and Techniques
With the growing prevalence of neural models in modern software development ecosystem, the security issues in these models have also become widely apparent. Models are susceptible to poisoning by “Trojans", which can lead them to output harmful, insecure code whenever a special “sign” is present in the input; even worse, such capabilities might evade detection. Given these models’ widespread use, it is thus important to study potential Trojan attacks on them. In order to do so it is important to understand how models of code interpret input, and how they can be attacked.
In this work, the authors study literature in Explainable AI and Safe AI to understand poisoning of neural models of code. In order to do so, they establish a novel taxonomy for Trojan AI for code, and present a new aspect-based classification of triggers in neural models of code.
With the growing prevalence of neural models in modern software development ecosystem, the security issues in these models have also become widely apparent. Models are susceptible to poisoning by “Trojans", which can lead them to output harmful, insecure code whenever a special “sign” is present in the input; even worse, such capabilities might evade detection. Given these models’ widespread use, it is thus important to study potential Trojan attacks on them. In order to do so it is important to understand how models of code interpret input, and how they can be attacked.
In this work, the authors study literature in Explainable AI and Safe AI to understand poisoning of neural models of code. In order to do so, they establish a novel taxonomy for Trojan AI for code, and present a new aspect-based classification of triggers in neural models of code.
🔥3
COSCO: On Contrastive Learning of Semantic Similarity for Code to Code Search
The paper introduces a novel code-to-code search technique that enhances the performance of LLMs by including both static and dynamic features as well as utilizing both similar and dissimilar examples during training. The authors present the code search method that encodes dynamic runtime information during training without the need to execute either the corpus under search or the search query at inference time. The proposed approach outperforms the state-of-the-art cross-language search tool by up to 44.7%.
COSCO (github)
RQ1. How does COSCO’s performance compare to the performance of other cross-language code search techniques?
RQ2. Does COSCO’s methodology and performance generalize across different models?
RQ3. Does including semantic similarity scores during training improve code search?
RQ4. How does changing the number of positive and negative comparison samples available for training effect COSCO’s performance?
The paper introduces a novel code-to-code search technique that enhances the performance of LLMs by including both static and dynamic features as well as utilizing both similar and dissimilar examples during training. The authors present the code search method that encodes dynamic runtime information during training without the need to execute either the corpus under search or the search query at inference time. The proposed approach outperforms the state-of-the-art cross-language search tool by up to 44.7%.
COSCO (github)
RQ1. How does COSCO’s performance compare to the performance of other cross-language code search techniques?
RQ2. Does COSCO’s methodology and performance generalize across different models?
RQ3. Does including semantic similarity scores during training improve code search?
RQ4. How does changing the number of positive and negative comparison samples available for training effect COSCO’s performance?