Мы уже писали про варианты JEPA, например, JEPA для time series (https://news.1rj.ru/str/gonzo_ML_podcasts/513) или для видео, типа V-JEPA (https://news.1rj.ru/str/gonzo_ML/3501) и V-JEPA 2 (https://news.1rj.ru/str/gonzo_ML/3953). Теперь JEPA доехала до LLM и есть LLM-JEPA!
https://news.1rj.ru/str/gonzo_ML_podcasts/880
Результат интересный. Главный челлендж, как для языковых данных создавать различные view.
https://news.1rj.ru/str/gonzo_ML_podcasts/880
Результат интересный. Главный челлендж, как для языковых данных создавать различные view.
Telegram
gonzo_ML_podcasts
LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures
Authors: Hai Huang, Yann LeCun, Randall Balestriero
Paper: https://arxiv.org/abs/2509.14252
Code: https://github.com/rbalestr-lab/llm-jepa
Review: https://arxiviq.substack.com/p/llm…
Authors: Hai Huang, Yann LeCun, Randall Balestriero
Paper: https://arxiv.org/abs/2509.14252
Code: https://github.com/rbalestr-lab/llm-jepa
Review: https://arxiviq.substack.com/p/llm…
😁11❤🔥6🔥3❤1
Когда же уже R2 наконец?!
DeepSeek-V3.1 → DeepSeek-V3.1-Terminus
✨ What’s improved?
🌐 Language consistency: fewer CN/EN mix-ups & no more random chars.
🤖 Agent upgrades: stronger Code Agent & Search Agent performance.
https://x.com/deepseek_ai/status/1970117808035074215?t=zuXvRjUBudH5diKElMnijg&s=19
DeepSeek-V3.1 → DeepSeek-V3.1-Terminus
✨ What’s improved?
🌐 Language consistency: fewer CN/EN mix-ups & no more random chars.
🤖 Agent upgrades: stronger Code Agent & Search Agent performance.
https://x.com/deepseek_ai/status/1970117808035074215?t=zuXvRjUBudH5diKElMnijg&s=19
X (formerly Twitter)
DeepSeek (@deepseek_ai) on X
🚀 DeepSeek-V3.1 → DeepSeek-V3.1-Terminus
The latest update builds on V3.1’s strengths while addressing key user feedback.
✨ What’s improved?
🌐 Language consistency: fewer CN/EN mix-ups & no more random chars.
🤖 Agent upgrades: stronger Code Agent & Search…
The latest update builds on V3.1’s strengths while addressing key user feedback.
✨ What’s improved?
🌐 Language consistency: fewer CN/EN mix-ups & no more random chars.
🤖 Agent upgrades: stronger Code Agent & Search…
👍5😁3❤1
Что-то интересное:
Happy to release Meta Code World Model (CWM), a 32-billion-parameter dense LLM that enables novel research on improving code generation through agentic reasoning and planning with world models.
https://ai.meta.com/research/publications/cwm
When humans plan, we imagine the possible outcomes of different actions. When we reason about code we simulate part of its execution in our head. The current generation of LLMs struggles to do this. What kind of research will an explicitly trained code world model enable? CWM allows us to study this question. Our model is trained on large amounts of coding data & bespoke Python + Bash world modeling data, allowing it to simulate Python function execution and agentic interactions in Bash environments.
The team and I can’t wait to see what new research will be enabled with a world model.
📊 Tech Report https://ai.meta.com/research/publications/cwm/
⚖️ Models weights https://ai.meta.com/resources/models-and-libraries/cwm-downloads/
🤗 On Huggingface https://huggingface.co/facebook/cwm
https://huggingface.co/facebook/cwm-sft
https://huggingface.co/facebook/cwm-pretrain
🧑💻 Inference Code https://github.com/facebookresearch/cwm
We believe CWM provides a strong testbed for research on improving code generation with world models. We performed multi-task RL, and CWM has competitive perfor mance for its size with 68.6% on LiveCodeBench v5, 76% on AIME24, and 65.8% on SweBench Verified with test time scaling.
I'm immensely proud of the work done by my cracked CodeGen team at Meta, with PhD students and veterans, for which nothing is someone else's problem.
The broader Meta AI community all pulled together for this.
I'm very thankful for the unwavering support of our whole leadership.
https://www.facebook.com/share/p/1DEqPXYp1g/
Happy to release Meta Code World Model (CWM), a 32-billion-parameter dense LLM that enables novel research on improving code generation through agentic reasoning and planning with world models.
https://ai.meta.com/research/publications/cwm
When humans plan, we imagine the possible outcomes of different actions. When we reason about code we simulate part of its execution in our head. The current generation of LLMs struggles to do this. What kind of research will an explicitly trained code world model enable? CWM allows us to study this question. Our model is trained on large amounts of coding data & bespoke Python + Bash world modeling data, allowing it to simulate Python function execution and agentic interactions in Bash environments.
The team and I can’t wait to see what new research will be enabled with a world model.
📊 Tech Report https://ai.meta.com/research/publications/cwm/
⚖️ Models weights https://ai.meta.com/resources/models-and-libraries/cwm-downloads/
🤗 On Huggingface https://huggingface.co/facebook/cwm
https://huggingface.co/facebook/cwm-sft
https://huggingface.co/facebook/cwm-pretrain
🧑💻 Inference Code https://github.com/facebookresearch/cwm
We believe CWM provides a strong testbed for research on improving code generation with world models. We performed multi-task RL, and CWM has competitive perfor mance for its size with 68.6% on LiveCodeBench v5, 76% on AIME24, and 65.8% on SweBench Verified with test time scaling.
I'm immensely proud of the work done by my cracked CodeGen team at Meta, with PhD students and veterans, for which nothing is someone else's problem.
The broader Meta AI community all pulled together for this.
I'm very thankful for the unwavering support of our whole leadership.
https://www.facebook.com/share/p/1DEqPXYp1g/
Meta AI
CodeGen Computational World Model access request form - Meta AI
Request access to CodeGen Computational World Model.
👍22❤12
Sakana опять что-то прикольное сделала.
We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency.
Blog: https://sakana.ai/shinka-evolve/
Code: https://github.com/SakanaAI/ShinkaEvolve
Paper: https://arxiv.org/abs/2509.19349
Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources!
Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature.‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful.
On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a massive leap in efficiency compared to previous methods that required thousands of evaluations.
We applied ShinkaEvolve to a diverse set of hard problems with real-world applications:
1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency.
2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition.
3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity.
ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job.
By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it!
We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency.
Blog: https://sakana.ai/shinka-evolve/
Code: https://github.com/SakanaAI/ShinkaEvolve
Paper: https://arxiv.org/abs/2509.19349
Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources!
Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature.‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful.
On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a massive leap in efficiency compared to previous methods that required thousands of evaluations.
We applied ShinkaEvolve to a diverse set of hard problems with real-world applications:
1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency.
2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition.
3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity.
ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job.
By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it!
sakana.ai
Sakana AI
ShinkaEvolve: Evolving New Algorithms with LLMs, Orders of Magnitude More Efficiently
🔥22👏10❤5👍1
Прикольная работа Parallel-R1 про параллелизацию и исследование разных независимых путей во время ризонинга:
https://news.1rj.ru/str/gonzo_ML_podcasts/894
(параллелизация пока скорее только логическая, не техническая, но это логичный следующий шаг)
Ещё забавно, что это напоминает parallel scaling из “One extinction scenario” свежей книги Юдковского и Соареса ;)
https://news.1rj.ru/str/gonzo_ML_podcasts/894
(параллелизация пока скорее только логическая, не техническая, но это логичный следующий шаг)
Ещё забавно, что это напоминает parallel scaling из “One extinction scenario” свежей книги Юдковского и Соареса ;)
Telegram
gonzo_ML_podcasts
Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
Authors: Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, Dong Yu
Paper: https://arxiv.org/abs/2509.07980
Code: https://…
Authors: Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, Dong Yu
Paper: https://arxiv.org/abs/2509.07980
Code: https://…
❤7🔥4
Ещё из интересного, что антропик теперь свой агентский SDK выпустил
https://www.anthropic.com/engineering/building-agents-with-the-claude-agent-sdk
https://www.anthropic.com/engineering/building-agents-with-the-claude-agent-sdk
Anthropic
Building agents with the Claude Agent SDK
How to get started with the Claude Agent SDK and best practices for using it.
1🤯9👍4👀4
😥
Philosopher John Searle, well-known for his work on philosophy of mind and philosophy of language, has died.
https://dailynous.com/2025/09/28/john-searle-1932-2025/
Там вообще какая-то грустная история...
https://www.colinmcginn.net/john-searle/
Philosopher John Searle, well-known for his work on philosophy of mind and philosophy of language, has died.
https://dailynous.com/2025/09/28/john-searle-1932-2025/
Там вообще какая-то грустная история...
https://www.colinmcginn.net/john-searle/
🕊24💔10❤2👍1
gonzo-обзоры ML статей
Стартап Миры Мурати разродился на этой неделе первым постом в блоге. Тема: воспроизводимость ответов LLM. https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ Где там остаётся недетерминизм, когда все сиды уже зафиксированы. Разбирают…
Кстати, там продолжения в блоге пошли:
LoRA Without Regret
https://thinkingmachines.ai/blog/lora/
Modular Manifolds
https://thinkingmachines.ai/blog/modular-manifolds/
LoRA Without Regret
https://thinkingmachines.ai/blog/lora/
Modular Manifolds
https://thinkingmachines.ai/blog/modular-manifolds/
Thinking Machines Lab
LoRA Without Regret
How LoRA matches full training performance more broadly than expected.
🔥10