Quiet Years of Agents

Summary: While 2023 marked a significant era for AI with the rise of advanced monolithic systems, the future is poised to shift towards the development and integration of parallel agents.

Aim:

To provide sense-making during technological shifts that might be outside the overton window.

Motivation:

During this time, there is seemingly an impasse– where both humans and the agents themselves are operating within cognitive and computational limitations but making untenable progress.

2023 marked a watershed moment in AI history, distinguished by:

• Large language models reaching unprecedented capabilities in chained reasoning
• Integration of multimodal abilities into unified systems and tools (such as computer use)
• Widespread adoption of AI in enterprise, national security and consumer applications

Introduction

Agents can be described as systems where large language models (LLMs) dynamically control their own processes and tool usage to accomplish tasks – for instance, an AI system that can use your computer to browse for gifts online is a tool-use(computer). This contrasts with workflows, where LLMs and tools follow predefined, orchestrated code paths. In essence, agents maintain autonomy in how they complete tasks, unlike workflows that adhere to fixed steps ["Building effective agents," 2024]. Additionally, we have also seen LLMs that 'think out loud' at inference time (test-time compute).

Social Agents (Deliberative Agents)

This entails agents that will be part of our platform economy, trading preferences, upvoting views(social media interaction etc) and deliberating with us. Agents in social contexts will reshape social dynamics by facilitating and mediating interactions between humans and machines, influencing concepts of cooperation, trust, and responsibility, as they adapt to the complexities of human communication and behavior.

The platform economy is poised for transformation through deliberative agents that will:

• Participate in social consensus building
• Trade preferences and represent user interests
• Engage in meaningful deliberation with humans
• Create new forms of digital democracy and governance

*[Melody Guan et. al, 2024] introduced the framework of Deliberative Alignment —a method that teaches AI models safety rules directly and trains them to recall and reason through these rules before responding. Deliberative Alignment enhances both robustness against jailbreak attempts– by reasoning through explicitly defined policies, this method offers a more scalable, trustworthy, and interpretable way to align AI systems.

Whereas [Konya, 2023], defined deliberative alignment systems to be any alignment system that uses deliberative technology(such as Remesh, Pol.is etc), and deliberative alignment to broadly refer to the study, creation, or utilization of such systems. While the two definitions emphasize different methods and tools, they share a common goal: fostering alignment through structured reasoning and explicit consideration of principles. Guan et al.'s approach could be seen as a specific implementation of deliberative alignment principles within AI models, whereas Konya's framework encompasses a broader set of techniques, including collaborative technologies, to achieve similar objectives.

Automated Deliberation

The concept of Factored Cognition (FC) explores breaking down complex tasks into smaller, manageable subtasks to improve AI systems' safety and interpretability*[Factored Cognition, Equiano]*. The aim of automated reasoning is to find scalable ways to leverage ML for deliberation.
We can break down the problem of automating deliberation into two main parts:

1. Making deliberation explicit enough for machine learning replication, i.e., generating appropriate training data
2. Automating Deliberation

Assessing agent capabilities requires new frameworks, such as automated deliberation, measuring reasoning capacity, iterated amplification[Christiano, 2018], evaluating social intelligence*[Mossel, 2019],* and testing decision-making in complex social contexts.

Looking ahead – Beyond 2025

Tacit Knowledge for Agents (Implizites Wissen oder stilles Wissen)

Tacit knowledge refers to implicit, non-verbalizable knowledge acquired through experience and context, famously described by Michael Polanyi as "we know more than we can tell." Minsky envisioned this for AI in “The Society of Mind”[Singh, 2023] as Negative Expertise as knowledge about avoiding common errors or unproductive actions in problem-solving. Minsky suggests this knowledge, though often invisible, may constitute a significant portion of what we know and links it to humor as a way of learning about pitfalls in reasoning. Experimentally, this could be an application of censor-suppressor modes of reasoning. David Duetsch also argues that we should find new ways to explain explanations – in practice, this could be explored through ‘Fun Criterion’ and Funny Models. For AI agents, this entails the ability to internalize and act upon knowledge that is difficult to formalize in explicit rules or data structures. This could include intuition, pattern recognition in complex settings, or context-sensitive decision-making. We should gather epistemic approaches to knowledge that could be attained by agents, the knowledge which is difficult to put in words.

Non-Erotetic Agents (Decision Benchmarks – Entscheidungsproblem)

Erotetic refers to the philosophical study of questions and answers. We should inquire into the process by which we evaluate AI systems, currently we rely on question-answer benchmarks, however, agents operate in wider open-world domains that might not be erotetic. We can look into decision problems [Saparov, 2023], where the answer is a simple “yes” or “no” – such deductions should be universally valid according to the Church-Turing thesis [Turing, 1936, Khaliq, 2023]. Decision Benchmarks could improve the scope of agent computable problems, from this, we could gather a sense of agent-complete problems – analogous to Turing-complete problems, but specific to the capabilities of agents problems that are complete for agents. This can improve our understanding and scope agent satisfiability for advanced AI systems.

Tamper-Resistant Agents

The models will use smart tools to improve their decision-making, including methods that help them weigh probabilities(Bayesian), predict outcomesGame Theory), and find the best possible solutions in tricky situations(min-max equilibrium learning) [Costis, 2022, Zang, 2023]. Such systems will require even more robust refusal training – complex dynamics in black box systems will require care probing mechanisms.

XR Agents (Bridging Digital and Physical Realities)

AI agents have demonstrated exceptional capabilities in performing various tasks, such as manipulating robots, interacting with operating systems through shell scripts, querying databases, browsing the web, and playing video games [Yao, Webshop] [Liu, AgentBench]. As these agents evolve, their capabilities are expanding into more immersive environments, such as mixed reality, where XR agents will bridge the digital and physical realms.

The Protocol Years: Building Foundations

While headlines may focus elsewhere, these years will be crucial for:

As agents' interfaces become more accustomed to tools, it will be important to design new communication protocols for the human-tool-agent triad, similar to classical internet [Internet Protocol, 1974] by Vint Cerf, and Bob Khan. Creation of safety frameworks

Evolution of agent coordination mechanisms

“We must find ways to build societies of agents that are tolerant to some degree of vagueness and ambiguity in both their thoughts and communications.” Minsky argued that such communication can be done through K-lines, connection lines and internal languages or even post-symbolic communication.

Looking Forward: 2024-2029

The next five years will likely see:

Conclusion

The transition from monolithic systems to diverse, specialized agents represents not just a technical evolution but a fundamental shift in how we conceptualize and interact with artificial intelligence. This "Decade of Agents" promises to reshape our digital landscape in ways both subtle and profound.