Field note — 2024 → 2026 · Working note
From action to populations
Looking back at the shift from models becoming actors to actors becoming populations.
In early 2024, the Office of the UN Secretary-General’s Envoy on Technology asked me to take part in an AI horizon-scanning exercise for the Secretary-General’s High-Level Advisory Body on Artificial Intelligence. I was asked what emerging trends I thought could have the most surprising or significant impact on AI-related risks over the following eighteen months.
My answer focused on what I called real-world enabled AI. I singled out voice systems, physically situated systems, and “Remote AI”: agentic systems capable of taking autonomous actions with decreasing dependence on direct human prompting. My concern at the time was that these capabilities were arriving faster than the safeguards around them.
Looking back, I think that was one boundary of the transition: models becoming actors. The more interesting boundary now may be actors becoming populations.
An agent now encounters other agents that can send messages, delegate work, exchange resources, and retain a history of previous interactions. Its capabilities depend partly on the population around it.
Project Sid offers an early example: its authors placed roughly 10 to more than 1,000 language-model agents inside Minecraft and reported specialised roles, collective rule formation, and forms of cultural transmission. Other work has begun examining agent populations under formal institutional arrangements. At the infrastructure layer, the Agent2Agent protocol supports communication and delegation between independently built agents.
These projects motivate a change in the scale of inquiry. Repeated encounters make population structure, resource limits, identity, and the persistence of shared rules worth studying directly.
A simulation gives us control over some of those conditions. We can change the cost of communication, remove persistent identity, introduce property, or replace participants while retaining their records. Each intervention tests which parts of an observed pattern depend on a particular rule.
The challenge is to find patterns that recur across different tasks and starting conditions. A thousand agents in Minecraft cannot stand in for a human society. A result that survives several carefully chosen changes would provide a more useful basis for designing another experiment, and eventually for testing an intervention in a larger system.
I keep returning to the idea of the noosphere: a layer of collective thought produced by interconnected minds. The networked circulation of human knowledge already suggests such a layer. Agents add participants that can interpret a request, negotiate, and reorganise activity in response to others.
That possibility motivates this notebook. The proposed Games experiments define a world; Ecology examines interactions within its population; Genus tracks the differences and histories that its participants acquire.
References
UN Office of the Secretary-General’s Envoy on Technology, AI Risk Pulse Check / horizon-scanning exercise (2024); Altera.AL et al., “Project Sid: Many-agent simulations toward AI civilization,” arXiv:2411.00114 (2024); Michael Richards, Danny Cowser, Daniel Nielson, and Jacob W. Crandall, “Toward Simulating Networked Societies with Formal Institutions Using AI Agents,” Proceedings of AAAI 40(46) (2026), 39143–39150; Agent2Agent Protocol v1.0, production-ready open standard for agent-to-agent communication (2026).