One organism,
many theories.

The landscape of artificial organisms in behavior analysis

Every effort to build an artificial organism in behavior analysis can be placed by two questions: what is its algorithmic style (horizontal, from symbolic and heuristic, through stochastic and dynamical, to connectionist, reinforcement, and evolutionary), and what is its primary aim (vertical, from reproducing data, through closed-loop simulation, to engineering control, neural plausibility, and explicit mapping). Each point is a research approach. This lab's work is in accent; hover any point for its description, and click one of ours to jump into it.

Building organisms is an old idea

The ambition to understand behavior by constructing it predates this line of research by decades, and reinforcement learning in artificial intelligence grew up alongside it from the same question. To be clear, we are one branch of a long lineage. The two literatures developed in parallel, asking one question from two directions.

Many ways to build one; three we have tested

Many approaches could animate an artificial organism. These three are the families we have worked with so far, and each buys something at a price.

Those are the approaches we have tried. The rest of this update is this line of research filling them in (i.e., the demonstration engines toward the left, the first-principles organism toward the right), with every engine answering to the same measured behavior.

Pick a theory, then run the organism

One way to study a learning theory is to embody it as an artificial organism and watch what behavior emerges. Below are four such engines, each borrowing an existing theory. Select one and its controls appear; the organism then forages on concurrent variable-interval schedules (i.e., two options, each paying off on its own unpredictable timer). The first three are stochastic choosers, so the panel plots the generalized matching law (i.e., how the log ratio of behavior tracks the log ratio of obtained reinforcement). Brunt–Väisälä, being a continuous dynamical model, instead relaxes its allocation toward the matching equilibrium over time.

All four engines run on the shared harness ao-simulator over the same environments and schedules. Each borrows its theory; the rest of this update builds an organism that borrows none.

The organism we build from first principles

The four engines borrow a theory. This one borrows none. Most models of behavior install the answer in advance, a value function, a matching equation, a policy. This organism installs none of them, and is built to stay objective and non-mentalistic: every term reduces to something measurable (energy expended, distance moved, stimulus intensity, response probability), never a posited inner state.

It is not a reinforcement-learning policy. It is a distributed dynamical system of many coupled, response-capable units, the behavioral atoms. Stimuli act as force fields; learning history changes the forces those stimuli exert; behavior emerges from integrating the coupled dynamics over time; and survival is governed by an energy budget drawn from behavioral ecology. Nothing in the animation is scripted; the path is read out from the atom dynamics.

A trained organism foraging (left): position and recent trail. The energy reserve (right) dips toward starvation on the trek to the patch, then is sustained by foraging it.

Two tiers: the operant is built into the architecture

This is the behavior-analytic definition of an operant, built into the design: a functional class defined by its outcome, not its topography. "Approach food" is the operant; it is expressed through whatever movements the environment makes available.

One step of the organism, repeated for a whole life

Stimuli are sensed as forces, the atom activations are integrated one step, an action is emitted by a softmax over activations, the energy reserve is updated, and the one learning rule changes the weights. Then it repeats. (1 step is 15 minutes; a life runs to ~10,000 steps.)

The physics of the organism

Nothing here is fit to behavioral data. These equations are the mechanics of the organism; the behavioral laws (matching, momentum, generalization, the energy-budget rule) come out the far end. Each is shown with a plain-language reading.

Each row pairs the equation with a live picture of what it does. The math is the bookkeeping; the picture on the right is the intuition.

It also forms associations

Operant theories address what consequences select. They are largely silent on what a stimulus comes to predict. The first-principles organism also carries a Rescorla–Wagner associative process (Rescorla & Wagner, 1972), so the same body shows acquisition, an asymptote, and extinction. Adjust the salience and the phase lengths and watch associative strength build, then decline.

State decides whether risk pays

Give the organism a reserve, a steady metabolic drain, and a hard death boundary, and risk preference stops being a free parameter. It follows from survival. This is the energy-budget rule (Caraco, 1980; Stephens & Krebs, 1986): with two options of equal mean intake, prefer the certain option when reserves sit above the requirement, and prefer the variable option when they fall below it. Move the reserve and the optimal choice changes.

One substrate, operant and respondent

The organism is built toward a single goal: that one substrate (i.e., the atoms, the forces, the Verlet integration, the energy budget, and the Rescorla–Wagner rule) can account for the phenomena of operant and respondent conditioning, each demonstrated on its own. Many emerge from that core substrate directly; others currently use an added mechanism or a companion analysis in the same codebase, and the repository flags which is which. The organizing thesis is that respondent and operant conditioning are one associative process read out under different procedures. Whether behavior appears respondent or operant depends on the procedure arranged (a stimulus paired with an outcome, or a response followed by one); the mechanism inside the organism is the same in both cases.

These emerge from the core substrate, the organism's own behavior. Each is a population of organisms (95% confidence bands), run from the same engine:

From the same engine, the program has also taken these further, using its choice and survival-dynamics modules: risk-sensitive foraging and the energy-budget rule, resurgence as model mimicry, the reinforcement–punishment asymmetry, and higher-moment (skew) effects. Those are collected in the full suite.

Building, then answering to data

The point of this arm is to put the models against real behavior. One case does it directly today: an artificial organism animated by Q-learning, applied to a real behavioral record (Cox & Santos, 2025). On the eight-pigeon data, a flexible machine-learning model defines the prediction benchmark, and human-task platforms supply more data.

Putting our own engines against these data

We ran the actual engines, imported from their repositories, on the eight pigeons' side-entry choices: each is driven teacher-forced as an online predictor, with its own parameters tuned per pigeon (a grid seed plus a Nelder-Mead refine) on training trials and scored on held-out trials. Q-learning is the strongest, averaging about 0.65 AUROC and, on several pigeons, matching or exceeding the GRU (up to 0.75); tuning the learning rate well below the original grid is what closed the gap. MPR ranks choices above chance (about 0.57) but its probabilities hug indifference, so at a 0.5 cut it rarely flips its prediction (MCC near 0), which is what a matching-based engine does once local reinforcement rates equalize. ETBD stays near chance (about 0.53). The two evaluation splits, the last 30% of trials and the last ~30% of sessions held out, give the same picture. So a tuned Q-learning agent rivals the flexible GRU on individual birds, while MPR carries weaker rank-order signal and ETBD adds little here.

What this line of research is writing

These are the manuscripts of this line of research, grouped as we group them internally. Statuses are current; unrelated work from other lines is left out.

Toward one organism with swappable theories