Teaching Transformers Modular Arithmetic at Scale
The work introduces novel techniques to help ML models learn modular addition. These techniques—varying the diversity of training data, using an angular embedding for model inputs and outputs, and introducing a regularized loss function—enable ML models to add hundreds of elements mod a large $q$ with high accuracy, a significant improvement over prior work.
Modular addition: given $N$ elements in $Z_q$, compute their sum modulo $q$.
The work introduces novel techniques to help ML models learn modular addition. These techniques—varying the diversity of training data, using an angular embedding for model inputs and outputs, and introducing a regularized loss function—enable ML models to add hundreds of elements mod a large $q$ with high accuracy, a significant improvement over prior work.
Modular addition: given $N$ elements in $Z_q$, compute their sum modulo $q$.
The Artificial Inflation (AI) of Artificial Intelligence (AI)—or AI^2 Bursts its Bubble, Bringing Down the Hype of the AI Threat
While some of the promises of AI have come true, and technology (like ChatGPT and its plugins) will continue to impress with its capabilities, AI-based technologies have largely failed to live up to the mountainous hype. In 2025, the authors expect the industry to pull back on the promises, investment, and hype of new AI capabilities and settle down into what is real versus marketing noise.
While some of the promises of AI have come true, and technology (like ChatGPT and its plugins) will continue to impress with its capabilities, AI-based technologies have largely failed to live up to the mountainous hype. In 2025, the authors expect the industry to pull back on the promises, investment, and hype of new AI capabilities and settle down into what is real versus marketing noise.
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
The paper introduces Tokenformer. The architecture leverages the attention mechanism to facilitate not only inter-token computations but also interactions between tokens and model parameters. The authors replace all linear projection layers in the Transformer with Pattention layers, allowing for efficient incremental scaling without the need for retraining from scratch.
Future work:
- Extending the Mixture-of-Experts Paradigm
- Advancing Parameter-Efficient Tuning
- Integrating Vision and Language Models
- Device-Cloud Collaboration
- Enhancing Model Interpretability
Code: https://github.com/Haiyang-W/TokenFormer
The paper introduces Tokenformer. The architecture leverages the attention mechanism to facilitate not only inter-token computations but also interactions between tokens and model parameters. The authors replace all linear projection layers in the Transformer with Pattention layers, allowing for efficient incremental scaling without the need for retraining from scratch.
Future work:
- Extending the Mixture-of-Experts Paradigm
- Advancing Parameter-Efficient Tuning
- Integrating Vision and Language Models
- Device-Cloud Collaboration
- Enhancing Model Interpretability
Code: https://github.com/Haiyang-W/TokenFormer
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
The authors presented RE-Bench, a suite of environments that measure the ability of AI agents to automate AI R&D tasks. They compare humans to several public frontier models through best-of-k with varying time budgets and agent designs, and find that the best AI agents achieve a score 4x higher than human experts when both are given a total time budget of 2 hours per environment. However, humans currently display better returns to increasing time budgets, narrowly exceeding the top AI agent scores given an 8-hour budget, and achieving 2x the score of the top AI agent when both are given 32 total hours.
The authors presented RE-Bench, a suite of environments that measure the ability of AI agents to automate AI R&D tasks. They compare humans to several public frontier models through best-of-k with varying time budgets and agent designs, and find that the best AI agents achieve a score 4x higher than human experts when both are given a total time budget of 2 hours per environment. However, humans currently display better returns to increasing time budgets, narrowly exceeding the top AI agent scores given an 8-hour budget, and achieving 2x the score of the top AI agent when both are given 32 total hours.
👍2
Mechanistic Interpretability
I have prepared a list of papers on Mechanistical Interpretability. If you have good links on this topic, please share them in the comments.
* 2021: A Mathematical Framework for Transformer Circuits
* 2022.06.27: Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
* 2022.09.14: Toy Models of Superposition
* 2022.09.24: In-context Learning and Induction Heads
* 2023.04.28: Towards Automated Circuit Discovery for Mechanistic Interpretability
* 2023.01.12: Progress measures for grokking via mechanistic interpretability
* 2023.05.24: Interpretability Dreams
* 2023.09: Sparse Autoencoders Find Highly Interpretable Model Directions
* 2023.10.25: Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism
* 2024.01.15: Sparse Autoencoders Work on Attention Layer Outputs
...
I have prepared a list of papers on Mechanistical Interpretability. If you have good links on this topic, please share them in the comments.
* 2021: A Mathematical Framework for Transformer Circuits
* 2022.06.27: Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
* 2022.09.14: Toy Models of Superposition
* 2022.09.24: In-context Learning and Induction Heads
* 2023.04.28: Towards Automated Circuit Discovery for Mechanistic Interpretability
* 2023.01.12: Progress measures for grokking via mechanistic interpretability
* 2023.05.24: Interpretability Dreams
* 2023.09: Sparse Autoencoders Find Highly Interpretable Model Directions
* 2023.10.25: Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism
* 2024.01.15: Sparse Autoencoders Work on Attention Layer Outputs
...
👍4
...
* 2024.02.01: Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small
* 2024.02.06: Challenges in Mechanistically Interpreting Model Representations
* 2024.02.22: Do sparse autoencoders find "true features"?
* 2024.03.14: Sparse autoencoders find composed features in small toy models
* 2024.03.15: Improving SAE's by Sqrt()-ing L1 & Removing Lowest Activating Features
* 2024.03.29: SAE reconstruction errors are (empirically) pathological
* 2024.04.22: Mechanistic Interpretability for AI Safety A Review
* 2024.05.21: Mapping the Mind of a Large Language Mode
* 2024.06.13: The engineering challenges of scaling interpretability
* 2024.07.02: A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
* 2024.07.29: Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability
* 2024.10.10: Bilinear MLPs enable weight-based mechanistic interpretability
* 2024.10.11: Explaining AI through mechanistic interpretability
* 2024.10.15: Mechanistic Permutability: Match Features Across Layers
* 2024.10.17: Using Dictionary Learning Features as Classifiers
* 2024.10.24: Probing Ranking LLMs: Mechanistic Interpretability in Information Retrieval
* 2024.10.25: Evaluating feature steering: A case study in mitigating social biases
* 2024.11.25: Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability
* 2024.02.01: Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small
* 2024.02.06: Challenges in Mechanistically Interpreting Model Representations
* 2024.02.22: Do sparse autoencoders find "true features"?
* 2024.03.14: Sparse autoencoders find composed features in small toy models
* 2024.03.15: Improving SAE's by Sqrt()-ing L1 & Removing Lowest Activating Features
* 2024.03.29: SAE reconstruction errors are (empirically) pathological
* 2024.04.22: Mechanistic Interpretability for AI Safety A Review
* 2024.05.21: Mapping the Mind of a Large Language Mode
* 2024.06.13: The engineering challenges of scaling interpretability
* 2024.07.02: A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
* 2024.07.29: Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability
* 2024.10.10: Bilinear MLPs enable weight-based mechanistic interpretability
* 2024.10.11: Explaining AI through mechanistic interpretability
* 2024.10.15: Mechanistic Permutability: Match Features Across Layers
* 2024.10.17: Using Dictionary Learning Features as Classifiers
* 2024.10.24: Probing Ranking LLMs: Mechanistic Interpretability in Information Retrieval
* 2024.10.25: Evaluating feature steering: A case study in mitigating social biases
* 2024.11.25: Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability
👍3
Salesforce Will Hire No More Software Engineers in 2025
Salesforce will not be hiring any more software engineers in 2025 amid significant productivity boosts from AI, Marc Benioff has revealed.
“We’re not adding any more software engineers next year because we have increased the productivity this year with Agentforce and with other AI technology that we’re using for engineering teams by more than 30% – to the point where our engineering velocity is incredible. I can’t believe what we’re achieving in engineering.
“And then, we will have less support engineers next year because we have an agentic layer. We will have more salespeople next year because we really need to explain to people exactly the value that we can achieve with AI. So, we will probably add another 1,000 to 2,000 salespeople in the short term.”
Salesforce will not be hiring any more software engineers in 2025 amid significant productivity boosts from AI, Marc Benioff has revealed.
“We’re not adding any more software engineers next year because we have increased the productivity this year with Agentforce and with other AI technology that we’re using for engineering teams by more than 30% – to the point where our engineering velocity is incredible. I can’t believe what we’re achieving in engineering.
“And then, we will have less support engineers next year because we have an agentic layer. We will have more salespeople next year because we really need to explain to people exactly the value that we can achieve with AI. So, we will probably add another 1,000 to 2,000 salespeople in the short term.”
👍3❤1🔥1👏1
Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
The paper investigates _subliminal learning_, a phenomenon where language models transmit behavioral traits (e.g., animal preferences or misalignment) through generated data that is semantically unrelated to those traits. Experiments show that training a student model on a teacher's number sequences, code, or reasoning traces can cause the student to adopt the teacher's traits, even after rigorous data filtering.
The findings highlight a potential risk in AI development, where unintended traits could be inadvertently propagated through model distillation. If a model becomes misaligned, then data generated by this model might transmit misalignment to other models, even if developers are careful to remove overt signs of misalignment from the data.
The paper investigates _subliminal learning_, a phenomenon where language models transmit behavioral traits (e.g., animal preferences or misalignment) through generated data that is semantically unrelated to those traits. Experiments show that training a student model on a teacher's number sequences, code, or reasoning traces can cause the student to adopt the teacher's traits, even after rigorous data filtering.
The findings highlight a potential risk in AI development, where unintended traits could be inadvertently propagated through model distillation. If a model becomes misaligned, then data generated by this model might transmit misalignment to other models, even if developers are careful to remove overt signs of misalignment from the data.
🔥5🤯3👍1🤔1
At what maturity level is the use of AI in SE in your company?
Anonymous Poll
17%
Resistant: Inhibits AI/SDLC practices; presence of anti-patterns
33%
Adhoc: Undisciplined practices; reliant on individuals
17%
Exploratory: Basic AI practices exist; no integration
17%
Structured: AI adoption is intentional and supports process improvement
0%
Established: Process-driven, consistent application across teams
13%
Integrated: AI practices are intuitive, evolving, and culturally ingrained
4%
Transformative: AI innovation is systemic; the process adapts fluidly to context
Open-source has continued to trail frontier, closed-source models in performance by nine to 12 months
Open-source models offer clear enterprise advantages: greater customization, potential cost savings, and the ability to deploy within private cloud or on-premises environments. But despite these benefits and recent improvements, open-source has continued to trail frontier, closed-source models in performance by nine to 12 months.
AI is making us work more
The article highlights the paradox that AI tools, designed to increase efficiency, are instead fueling a culture of overwork. With systems available 24/7, a psychological pressure emerges where any moment not spent being "productive" feels like falling behind. This mirrors historical shifts, like artificial lighting, which turned the ability to work longer into an obligation.
Personally, I find that while I constantly use AI and accomplish more tasks, I don't work any less—and might even be working more than before.
The article highlights the paradox that AI tools, designed to increase efficiency, are instead fueling a culture of overwork. With systems available 24/7, a psychological pressure emerges where any moment not spent being "productive" feels like falling behind. This mirrors historical shifts, like artificial lighting, which turned the ability to work longer into an obligation.
Personally, I find that while I constantly use AI and accomplish more tasks, I don't work any less—and might even be working more than before.
👍1😢1