Self Supervised Boy – Telegram
Self Supervised Boy
@selfsupervised
160 subscribers
9 photos
56 links
Posting links to papers I read. Right now I'm mostly interested in things around LLMs, AI agents, and ML4Code. That is subject to change.

@martolod
Download Telegram
About
Blog
Apps
Platform
Join
Self Supervised Boy
160 subscribers
Self Supervised Boy
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
from the ICML2020.

Previously it was noted, that if one swaps contrastive loss with a tighter bound on MI, the downstream quality decreases. The authors propose, therefore, to move from InfoMax intuition to rather simple concepts: alignment and uniformity. The former enforces that positive pairs stay as close as possible and the latter enforces that all samples stay as evenly distributed as possible.

These components are empirically important for downstream performance. Furthermore, their direct optimization may outperform the classical contrastive loss training.

With images and a bit longer: here
Source: here
Telegraph
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
The work from the ICML2020 goes deeper in the understanding of contrastive learning. Authors diverged from a proposal that contrastive loss maximizes the mutual information between the positive views because it was shown that optimizing tighter bound on MI…
1.89K views
Self Supervised Boy
Well, there was more than three years since the last post here. In these three years a lot has changed. I'm done with my PhD in Heidelberg Uni, and moved on to JetBrains to lead a team on AI agents. With all this on my hands, I will have even less time for writing the reviews I'd like to read. But on the other hand, I'd still like to share the papers I read.

So, instead, I will post here links to the papers that I read. You can view this experiment as copycatting the @j_links but with a bias towards LLMs and probably agents.
🔥10👍5
236 views
Self Supervised Boy
https://arxiv.org/abs/2509.18542
arXiv.org
Symphony-MoE: Harmonizing Disparate Pre-trained Models into a...
Mixture-of-Experts (MoE) models enable scalable performance by activating large parameter sets sparsely, minimizing computational overhead. To mitigate the prohibitive cost of training MoEs from...
🔥2
223 views
Self Supervised Boy
https://arxiv.org/abs/2509.19170
arXiv.org
Soft Tokens, Hard Truths
The use of continuous instead of discrete tokens during the Chain-of-Thought (CoT) phase of reasoning LLMs has garnered attention recently, based on the intuition that a continuous mixture of...
224 views
Self Supervised Boy
https://arxiv.org/abs/2509.21013
arXiv.org
Predicting LLM Reasoning Performance with Small Proxy Model
Given the prohibitive cost of pre-training large language models, it is essential to leverage smaller proxy models to optimize datasets before scaling up. However, this approach becomes...
216 views
Self Supervised Boy
https://arxiv.org/abs/2509.26476
arXiv.org
Regression Language Models for Code
We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to...
207 views
Self Supervised Boy
https://arxiv.org/abs/2510.01123
arXiv.org
Rethinking Thinking Tokens: LLMs as Improvement Operators
Reasoning training incentivizes LLMs to produce long chains of thought (long CoT), which among other things, allows them to explore solution strategies with self-checking. This results in higher...
230 views
Self Supervised Boy
https://arxiv.org/abs/2406.18665v4
arXiv.org
RouteLLM: Learning to Route LLMs with Preference Data
Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More...
👍1
257 views
Self Supervised Boy
https://arxiv.org/abs/2403.12031
arXiv.org
RouterBench: A Benchmark for Multi-LLM Routing System
As the range of applications for Large Language Models (LLMs) continues to grow, the demand for effective serving solutions becomes increasingly critical. Despite the versatility of LLMs, no...
👍1
279 views
Self Supervised Boy
https://arxiv.org/abs/2510.02375
arXiv.org
Pretraining with hierarchical memories: separating long-tail and...
The impressive performance gains of modern language models currently rely on scaling parameters: larger models store more world knowledge and reason better. Yet compressing all world knowledge...
257 views
Self Supervised Boy
https://arxiv.org/abs/2510.05445
arXiv.org
AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative...
Large language models (LLMs) and agent-based frameworks have advanced rapidly, enabling diverse applications. Yet, with the proliferation of models and agentic strategies, practitioners face...
267 views
Self Supervised Boy
https://arxiv.org/abs/2510.12773
arXiv.org
Dr.LLM: Dynamic Layer Routing in LLMs
Large Language Models (LLMs) process every token through all layers of a transformer stack, causing wasted computation on simple queries and insufficient flexibility for harder ones that need...
260 views
Self Supervised Boy
https://arxiv.org/abs/2510.18148v1
arXiv.org
Extracting Rule-based Denoscriptions of Attention Features in Transformers
Mechanistic interpretability strives to explain model behavior in terms of bottom-up primitives. The leading paradigm is to express hidden states as a sparse linear combination of basis vectors,...
👍1
262 views
Self Supervised Boy
https://arxiv.org/abs/2510.18147v1
arXiv.org
LLMs Encode How Difficult Problems Are
Large language models exhibit a puzzling inconsistency: they solve complex problems yet frequently fail on seemingly simpler ones. We investigate whether LLMs internally encode problem difficulty...
262 views
Self Supervised Boy
https://arxiv.org/abs/2510.21614v1
arXiv.org
Huxley-Gödel Machine: Human-Level Coding Agent Development by an...
Recent studies operationalize self-improvement through coding agents that edit their own codebases. They grow a tree of self-modifications through expansion strategies that favor higher software...
253 views
Self Supervised Boy
https://arxiv.org/abs/2601.05167
arXiv.org
RelayLLM: Efficient Reasoning via Collaborative Decoding
Large Language Models (LLMs) for complex reasoning is often hindered by high computational costs and latency, while resource-efficient Small Language Models (SLMs) typically lack the necessary...
122 views
Self Supervised Boy
https://arxiv.org/abs/2601.03335v1
arXiv.org
Digital Red Queen: Adversarial Program Evolution in Core War with LLMs
Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most...
👍1
141 views
Self Supervised Boy
https://arxiv.org/abs/2601.04786v1
arXiv.org
AgentOCR: Reimagining Agent History via Optical Self-Compression
Recent advances in large language models (LLMs) enable agentic systems trained with reinforcement learning (RL) over multi-turn interaction trajectories, but practical deployment is bottlenecked...
154 views
Self Supervised Boy
https://arxiv.org/abs/2601.05106
arXiv.org
Token-Level LLM Collaboration via FusionRoute
Large language models (LLMs) exhibit strengths across diverse domains. However, achieving strong performance across these domains with a single general-purpose model typically requires scaling to...
👍1
123 views
Self Supervised Boy
https://arxiv.org/abs/2601.07582v1
arXiv.org
ES-Mem: Event Segmentation-Based Memory for Long-Term Dialogue Agents
Memory is critical for dialogue agents to maintain coherence and enable continuous adaptation in long-term interactions. While existing memory mechanisms offer basic storage and retrieval...
125 views
Self Supervised Boy
https://arxiv.org/abs/2601.09503v1
arXiv.org
What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm...
Large language model (LLM) agents have demonstrated remarkable capabilities in complex decision-making and tool-use tasks, yet their ability to generalize across varying environments remains a...
🔥1
106 views