WaveCoder Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation
CodeOcean is a dataset comprising 20,000 instruction instances across 4 universal code-related tasks, which is aimed at augmenting the effectiveness of instruction tuning and improving the generalization ability of fine-tuned model.
WaveCoder is a fine-tuned Code LLM with Widespread And Versatile Enhanced instruction tuning. Wavecoder models outperform other open-source models in terms of generalization ability across different code-related tasks at the same level of fine-tuning scale.
CodeOcean is a dataset comprising 20,000 instruction instances across 4 universal code-related tasks, which is aimed at augmenting the effectiveness of instruction tuning and improving the generalization ability of fine-tuned model.
WaveCoder is a fine-tuned Code LLM with Widespread And Versatile Enhanced instruction tuning. Wavecoder models outperform other open-source models in terms of generalization ability across different code-related tasks at the same level of fine-tuning scale.
Binary Code Summarization: Benchmarking ChatGPT/GPT-4 and Other Large Language Models
The authors have presented a large-scale and comprehensive study of how LLMs can understand binary code semantics. They built BinSum, a comprehensive benchmark with an expansive dataset with over 557K binary functions, spanning various code representations, computer architectures, and optimization levels.
RQs:
- To what extent can LLMs comprehend binary code? What input of binary code impacts LLM’s output more?
- Which LLM performs the best on binary code comprehension? Which LLM is more efficient than others?
- How do the different computer architectures and optimization levels affect LLMs’ performance?
- What are additional factors of binary code input influencing LLMs’ comprehension capabilities?
The authors have presented a large-scale and comprehensive study of how LLMs can understand binary code semantics. They built BinSum, a comprehensive benchmark with an expansive dataset with over 557K binary functions, spanning various code representations, computer architectures, and optimization levels.
RQs:
- To what extent can LLMs comprehend binary code? What input of binary code impacts LLM’s output more?
- Which LLM performs the best on binary code comprehension? Which LLM is more efficient than others?
- How do the different computer architectures and optimization levels affect LLMs’ performance?
- What are additional factors of binary code input influencing LLMs’ comprehension capabilities?
🤔2
TypeEvalPy: A Micro-benchmarking Framework for Python Type Inference Tools
The paper introduces TypeEvalPy, a comprehensive micro benchmarking framework for evaluating type inference tools. TypeEvalPy contains 154 code snippets with 845 type annotations across 18 categories that target various Python features.
GitHub: https://github.com/secure-software-engineering/TypeEvalPy
The paper introduces TypeEvalPy, a comprehensive micro benchmarking framework for evaluating type inference tools. TypeEvalPy contains 154 code snippets with 845 type annotations across 18 categories that target various Python features.
GitHub: https://github.com/secure-software-engineering/TypeEvalPy
Generative AI for Math: Part I -- MathPile: A Billion-Token-Scale Pretraining Corpus for Math
MathPile is a specialized corpus centered around mathematics, characterized by its diversity and high quality. The authors plan to open-source different versions of MathPile with the noscripts used for processing, to facilitate future developments in this field.
GitHub: https://github.com/GAIR-NLP/MathPile/
MathPile is a specialized corpus centered around mathematics, characterized by its diversity and high quality. The authors plan to open-source different versions of MathPile with the noscripts used for processing, to facilitate future developments in this field.
GitHub: https://github.com/GAIR-NLP/MathPile/
Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon
The work introduces Activation Beacon for the extension of LLM’s context length. Activation Beacon condenses the LLM’s raw activations into more compact forms, enabling the LLM to perceive a vast context with a limited context window. As a plug-and-play component for the LLM, it brings in long contextual information while fully preserving the LLM’s existing capabilities on short contexts. The experimental studies show that Activation Beacon is able to extend Llama-2-7B's context length by x100 times (from 4K to 400K), meanwhile achieving a superior result on both long-context generation and understanding tasks.
GitHub: https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon
The work introduces Activation Beacon for the extension of LLM’s context length. Activation Beacon condenses the LLM’s raw activations into more compact forms, enabling the LLM to perceive a vast context with a limited context window. As a plug-and-play component for the LLM, it brings in long contextual information while fully preserving the LLM’s existing capabilities on short contexts. The experimental studies show that Activation Beacon is able to extend Llama-2-7B's context length by x100 times (from 4K to 400K), meanwhile achieving a superior result on both long-context generation and understanding tasks.
GitHub: https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon
😱4👍1
Committing without git
How to create a branch with two commits (add file and change file) without running git.
Source: https://matheustavares.gitlab.io/assets/committing-without-git/commit.py
How to create a branch with two commits (add file and change file) without running git.
Source: https://matheustavares.gitlab.io/assets/committing-without-git/commit.py
Synergy of Reinforcement Learning and Large Language Models (RL+LLMs) @ AAAI 2024
The goal of the workshop is to bring together RL and LLM communities to facilitate cross-pollination.
Workshop: February 26th 2024
Accepted papers:
- Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
- Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts
- CriticGPT: Multimodal LLM as a Critic for Robot Manipulation
- Decision Transformer With Tokenized Actions
- Reinforcement Learning for Optimizing RAG for Domain Chatbots
- Software Security Vulnerability Repair Using Reinforcement Learning with Large Language Models
- Exploring Reinforcement Learning with Large Language Models for Enhancing Badminton Players' Strategies
- DeLF: Designing Learning Environments with Foundation Models
The goal of the workshop is to bring together RL and LLM communities to facilitate cross-pollination.
Workshop: February 26th 2024
Accepted papers:
- Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
- Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts
- CriticGPT: Multimodal LLM as a Critic for Robot Manipulation
- Decision Transformer With Tokenized Actions
- Reinforcement Learning for Optimizing RAG for Domain Chatbots
- Software Security Vulnerability Repair Using Reinforcement Learning with Large Language Models
- Exploring Reinforcement Learning with Large Language Models for Enhancing Badminton Players' Strategies
- DeLF: Designing Learning Environments with Foundation Models
The behavior of a Tesla vehicle will be determined by artificial intelligence, rather than coded by engineers
FSD Beta v12 upgrades the city-streets driving stack to a single end-to-end neural network trained on millions of video clips, replacing over 300k lines of explicit C++ code.
FSD Beta v12 upgrades the city-streets driving stack to a single end-to-end neural network trained on millions of video clips, replacing over 300k lines of explicit C++ code.
Electrek
Tesla finally releases FSD v12, its last hope for self-driving
Tesla has finally started releasing its FSD Beta v12 update to customers, which is sort of its last hope to...
👍1
Investigating the Efficacy of Large Language Models for Code Clone Detection (CCD)
The authors investigated the applicability of LLMs for CCD (Type-4 code clones).
RQs:
- What is the effect of different prompts to encourage Chat-GPT to identify Code Clones?
- What is the performance of ChatGPT for code clone detection compared to the baselines (CodeBERT, RoBERTa, GraphCodeBERT)?
ChatGPT (GPT-3.5-turbo) surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878.
The authors investigated the applicability of LLMs for CCD (Type-4 code clones).
RQs:
- What is the effect of different prompts to encourage Chat-GPT to identify Code Clones?
- What is the performance of ChatGPT for code clone detection compared to the baselines (CodeBERT, RoBERTa, GraphCodeBERT)?
ChatGPT (GPT-3.5-turbo) surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878.
Meta AI releases Code Llama 70B
The model is designed for general code synthesis and understanding.
Weights: https://huggingface.co/codellama/CodeLlama-70b-hf
The model is designed for general code synthesis and understanding.
Weights: https://huggingface.co/codellama/CodeLlama-70b-hf
X (formerly Twitter)
AI at Meta (@AIatMeta) on X
Today we’re releasing Code Llama 70B: a new, more performant version of our LLM for code generation — available under the same license as previous Code Llama models.
Download the models ➡️ https://t.co/fa7Su5XWDC
• CodeLlama-70B
• CodeLlama-70B-Python
•…
Download the models ➡️ https://t.co/fa7Su5XWDC
• CodeLlama-70B
• CodeLlama-70B-Python
•…
👍1
JetBrains' unremovable AI assistant meets irresistible outcry
Some JetBrains customers feel strongly about AI Assistant and really don't want the plugin to be present in their JetBrains applications at all, whether that's due to corporate policies that are incompatible with AI Assistant or other concerns. But because the plugin code has been "deeply integrated," removal has proven complicated.
More than dozen threads on JetBrains' YouTrack issue board have been posted seeking a way to delete, uninstall, or otherwise excise the AI Assistant plugin since it debuted.
Some JetBrains customers feel strongly about AI Assistant and really don't want the plugin to be present in their JetBrains applications at all, whether that's due to corporate policies that are incompatible with AI Assistant or other concerns. But because the plugin code has been "deeply integrated," removal has proven complicated.
More than dozen threads on JetBrains' YouTrack issue board have been posted seeking a way to delete, uninstall, or otherwise excise the AI Assistant plugin since it debuted.
😁2😱2👎1
GitBug-Java: A Reproducible Benchmark of Recent Java Bugs
The authors introduce GitBug-Java, a reproducible benchmark of recent Java bugs featuring 199 bug-fixes sourced from 55 relevant open-source repositories. To ensure the relevance of the bug-fixes to current development practices, the authors only collected bug-fixes from 2023. This may be useful in LLM evaluations.
GitBug-Java also provides offline reproduction environments for each collected bug-fix. To guarantee the validity and quality of the bug-fixes included in GitBug-Java, the authors manually curated the included bug-fixes.
site: https://www.nuno.saavedra.pt/gitbug-java/#!/
github: https://github.com/gitbugactions/gitbug-java
The authors introduce GitBug-Java, a reproducible benchmark of recent Java bugs featuring 199 bug-fixes sourced from 55 relevant open-source repositories. To ensure the relevance of the bug-fixes to current development practices, the authors only collected bug-fixes from 2023. This may be useful in LLM evaluations.
GitBug-Java also provides offline reproduction environments for each collected bug-fix. To guarantee the validity and quality of the bug-fixes included in GitBug-Java, the authors manually curated the included bug-fixes.
site: https://www.nuno.saavedra.pt/gitbug-java/#!/
github: https://github.com/gitbugactions/gitbug-java
StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
StepCoder is a novelty training framework for code generation via RL. It breaks down complicated exploration problems to reduce the difficulty of exploring environments with sparse rewards while providing fine-grained optimization. In addition, the authors constructed a high-quality dataset APPS+, specifically for code generation.
Dataset: https://github.com/Ablustrund/APPS_Plus
StepCoder is a novelty training framework for code generation via RL. It breaks down complicated exploration problems to reduce the difficulty of exploring environments with sparse rewards while providing fine-grained optimization. In addition, the authors constructed a high-quality dataset APPS+, specifically for code generation.
Dataset: https://github.com/Ablustrund/APPS_Plus
👍3