Omo Notes
Omo NoteSpatial AIWorking definition

What is a spatial AI agent?

Not simply an AI shown in 3D. A spatial agent perceives place, remembers relationships, acts from a position, and makes that situated agency understandable.

Conceptual writing based on public sources. This is not an Omo experiment, paper, finding, or product announcement.

Published

Reading time

8 minutes

Type

Definition

Status

Provisional

01 / Definition

Spatial agency begins when place changes what the agent can know and do

Our working definition: a spatial AI agent is a software system whose perception, memory, decisions, or actions are meaningfully organized by relationships in a physical or virtual environment—and whose situated behavior can be understood and redirected by people.

The first half connects spatial agency to embodied AI. AI Habitat describes embodied AI in terms of intelligent machines with a physical or virtual embodiment and studies tasks such as navigation, instruction following, and question answering. The second half adds an interface requirement. A capable system is not automatically a good collaborator. People need cues about where it is, what it can perceive, why it moved, and how to intervene.

“Spatial” therefore means more than rendering an avatar beside a chat box. Position must have consequences. Distance might change what is audible or relevant. Orientation might determine what is perceived. A room might hold persistent context. Movement might be an action, a signal, or both.

02 / Four layers

A practical stack for spatial agency

01

Situated perception

The agent receives observations from somewhere: a camera view, coordinates, nearby objects, sound direction, or a bounded field of attention. It cannot treat all context as equally present.

02

Spatial memory

The agent retains useful relations among places, objects, people, and prior events. It can distinguish “beside,” “behind,” “where we met,” and “the route that failed.”

03

Situated action

The agent can alter its position or the environment: navigate, point, place, retrieve, open, build, or hand off. Action is constrained by reach, access, time, and other occupants.

04

Social legibility

The system communicates attention, intention, uncertainty, and availability in ways a nearby person can interpret. Presence becomes part of the interface contract.

Research platforms make the technical layers concrete. Habitat Lab lets researchers define agents with physical forms, sensors, capabilities, and tasks. Project Malmo exposes sensing and acting inside Minecraft. Neither project implies Omo’s social-legibility layer; that layer is our design proposal for turning spatial intelligence into a human-facing interface.

03 / Boundaries

What does not count on its own

Not sufficient

A 3D avatar

Representation is not agency. If place never changes perception, memory, or action, the space is decoration.

Not sufficient

A map lookup

A system can answer geographic questions without occupying a position or maintaining a situated relationship to a world.

Not sufficient

A remote cursor

Moving a pointer can act on a screen, but it does not by itself create spatial memory, constraints, or presence.

These can still be ingredients. An avatar can signal gaze. A map can support route planning. A cursor can reveal intent. The distinction is whether spatial relationships participate in the loop from observation to memory to choice to action—and whether a human can read that loop well enough to coordinate with it.

04 / Design questions

The questions a spatial interface should answer without a manual

01

Where is the agent’s attention?

A person should be able to tell what the system is attending to, and distinguish looking from knowing.

02

What will happen if I move?

Proximity and orientation need consistent meaning. Crossing a boundary should not silently change permissions or context.

03

What does this place remember?

Persistent context needs visible scope, ownership, and expiry. A room that remembers is also a room that can surprise.

04

How do I stop or correct it?

Intervention must work at the pace of action. A spatial agent should make pausing, redirecting, and undoing easier than watching.

This shifts evaluation from abstract intelligence toward lived coordination. Did the agent reach the destination? Did the person understand its route? Could they predict its next action? Did the space clarify responsibility or obscure it? The strongest spatial system may not be the most autonomous one. It may be the one that makes joint action easiest.

05 / Primary sources

Foundations for the definition

06 / Continue

See the definition inside a world

Related Omo Note

Minecraft makes spatial agency inspectable.

The related note looks at a shared game world as interface material for visible action, collaboration, recovery, and persistent traces.