→lyrics

robotics · position paper

Vision for Model-less Obsolescence

A position paper on hybrid robot intelligence: connecting persistent belief, attention loops, learned transitions, and a map shared across a peer-to-peer fleet.

Abstract

Robots operating over long periods must act on an environment that is only partially visible, changes unevenly, and is often already known to another agent. This paper argues for a hybrid architecture that combines learned reactive control with persistent spatial memory, selective attention, and action-conditioned prediction. The central claim is that these components should share an explicit, uncertainty-aware representation: memory preserves useful evidence, attention determines which evidence needs revision, and a transition model predicts the consequences relevant to the next decision. The same representation can support richer maps derived from archived observations and external sources, then distribute those maps across a peer-to-peer robot fleet. We propose mechanisms for incremental updates, task-conditioned retrieval, short-horizon learned dynamics, and replicated submap storage. We distinguish inferred structure from verified observations and separate spatial knowledge that transfers across robots from dynamics that depend on embodiment. This is a position and research agenda, not a report of measured gains. Its testable hypothesis is that selective reuse of shared world knowledge can improve adaptation and reduce repeated perception and exploration under matched resource budgets.

1. The position: hybrid intelligence

A robot should not need to rediscover a room each time it turns its camera, repeat a journey because its observations were discarded, or learn independently what a nearby robot already knows. Consider a robot returning to collect a cup it saw earlier. The cup is now hidden behind a box, a chair has moved into the route, and another robot has recently inspected the destination. Success depends on preserving identity through occlusion, revising only the changed parts of the scene, and deciding whether another observation is worth its cost. These are connected problems of maintaining and using knowledge.

Here, “model-less” refers to model-free control: learning a policy or value function without an explicit action-conditioned transition model. It does not imply an absence of neural representations or memory. A recurrent policy can retain history, and a policy trained with a world model can execute without online planning. Likewise, a spatial map is not by itself a transition model. The relevant distinctions are what information is retained, whether action consequences are predicted, and when those predictions influence decisions.

The position of this paper is that persistent, changing environments justify a hybrid by default as a research starting point. Learned policies should provide efficient local behavior; explicit memory should preserve reusable evidence; prediction should support decisions whose consequences cannot be resolved reactively. Active Neural SLAM supplies a precedent for combining learned mapping and policies with analytical path planning. Our proposed integration extends that modular approach to attention, object memory, and distributed knowledge. “Obsolescence” in the title expresses a direction for complete robot systems, not a demonstrated claim that model-free learning is universally inferior or should disappear as a component.

2. Object permanence and persistent belief

Object permanence requires more than caching the last image. An object should remain represented when it leaves the field of view, while its location and state become less certain when change is plausible. A useful record would include identity hypotheses, pose estimates, appearance features, relationships, observation times, and provenance. Geometry, semantics, and confidence need separate fields: recognizing a cup confidently does not imply knowing its position precisely. Repeated observations should refine an existing record when the association is credible, while ambiguous matches remain alternatives.

Absence should be interpreted through visibility. Failing to see a cup behind a box provides little evidence that it has moved; failing to see it on a clearly visible, well-observed table is more informative. A wall, an unattended cup, and a walking person also require different persistence assumptions. The map should therefore track a belief about the scene rather than freeze one reconstruction as truth. Confidence may decay with time, but a uniform expiration timer would ignore the differences between occlusion, sensor failure, and actual environmental change.

This representation creates an opportunity to reduce recalculation. Stable geometry and visual features can be reused while changed regions receive new processing. Derived information should retain dependencies: moving a chair invalidates the local clearance estimate and perhaps a route, while a correction to the robot’s pose can affect an entire group of observations. Cached plans must be associated with the map versions that justified them. The benefit is conditional: dependency tracking, association, and updating must cost less than the work they avoid. Persistent memory is valuable when it makes later computation more selective, which leads directly to attention.

3. A common architecture

The proposed system has a fast control path and a slower process for maintaining belief and considering alternatives. The control policy receives current local observations, the immediate goal, and a compact selection of relevant memory. In parallel, perception associates new evidence with map entities, attention selects further processing, and planning requests predictions when a choice warrants them. These components share identifiers, coordinate transforms, timestamps, and uncertainty estimates. A separate local obstacle check can veto an action even when the stored map or planner considers it feasible.

For clarity, let b_t denote the current belief about the world, M_t the persistent map, and g the task. The controller acts from current observations and a task-relevant projection of b_t and M_t. Belief includes transient hypotheses, such as where an occluded person might move; the map stores evidence intended to outlast the immediate decision. They need not contain identical information or share one update frequency. An attention decision determines which observation, memory query, or prediction to acquire next.

Interfaces matter as much as individual model quality. A planner needs to know whether a retrieved passage is measured or inferred; a policy needs to know whether its target pose is stale; an attention module needs to know which uncertainty would change the chosen action. Updates should be asynchronous, but every decision should use a coherent snapshot of the relevant state. The proposed hierarchy allows rapid behavior without rebuilding or simulating the entire world at every control step. It also makes failure attributable to a particular observation, association, retrieval, or prediction.

4. Attention loops

Attention is the mechanism that turns a large memory into a usable decision resource. It should operate as a loop: retrieve what appears relevant, compare it with current evidence, identify consequential uncertainty, acquire more information, and update the decision. Here, attention includes both neural weighting over features and explicit scheduling of sensing and computation. These are different mechanisms, and each should be evaluated separately. Four complementary candidates make the proposal concrete.

4.1. Task-conditioned retrieval

A query formed from the goal, pose, recent history, and candidate action retrieves a bounded set of map entities. A mug search might retrieve likely surfaces, containers, and past mug observations. Spatial filtering can first eliminate unreachable or distant candidates; semantic retrieval can then rank the remainder. A small graph neighborhood around each candidate supplies relationships that isolated embeddings would miss. Cross-attention over this selected set could condition the policy or planner. Retrieval must also preserve recent hazards and route constraints, even when they are semantically unrelated to the named target.

4.2. Foveated visual processing

A low-cost full-field pass can identify motion and coarse structure, while selected crops receive higher-resolution processing. Recurrent Models of Visual Attention establishes a precedent for choosing image regions to inspect in sequence. In the proposed system, crop selection would depend on both the task and map uncertainty: a distant door handle may merit detail during approach, while a well-localized wall does not. This saves computation only if the coarse pass and selection overhead remain inexpensive; masking tokens after an expensive full-image encoder would not automatically deliver that saving.

4.3. Prediction-error updates

Memory can predict the appearance or occupancy expected at a pose. A discrepancy can trigger richer perception and invalidate affected map entries. The trigger must distinguish likely scene changes from lighting variation, viewpoint changes, and localization error. Otherwise the robot may repeatedly remap a stable room. Separate thresholds for escalation and return to ordinary processing can reduce oscillation. Periodic broad observations are still necessary: a detector restricted to already attended regions cannot reliably notice an unexpected change elsewhere.

4.4. Decision-aware information gathering

The highest-uncertainty region is not always the most useful region to inspect. The better criterion is whether additional evidence could improve the current decision. Conceptually, the value of an observation is the expected reduction in decision loss after receiving it, minus acquisition and delay costs. An initial implementation could approximate that value using task relevance, uncertainty, disagreement between predictors, and proximity to a consequential action. Camera motion, a peer query, and a short physical inspection become competing ways to resolve the same question.

The loop stops when further information is unlikely to change the action enough to justify its cost, or when a time budget expires. Budget exhaustion should produce an explicit fallback, such as using a conservative local action or pausing for an essential check. Attention therefore connects memory to both movement and communication. It determines what deserves a new observation, what can be trusted from storage, and what should be requested from another robot.

5. More knowledge from fewer journeys

A richer map does not require a new physical visit for every new annotation. Existing images, depth measurements, trajectories, and interaction records can be processed again to extract object attributes, relations, or task-relevant features. ConceptGraphs provides a relevant example of an open-vocabulary, graph-structured representation of 3D scenes. A proposed map could combine metric geometry for clearance, topology for route selection, and semantic relationships for retrieval. Each layer answers a different question while referring to the same places and objects.

Archived observations could reveal details the original task did not require: which surface supported a tool, which doorway connects two rooms, or where an object was last visible. This is enrichment of recorded evidence, not new observation of the present. A caption and an embedding derived from the same image are correlated products and should not count as independent confirmations. Retaining selected keyframes or links to source measurements makes later reprocessing and correction possible. Storage policies should favor distinctive or uncertain observations instead of preserving every frame indefinitely.

Floor plans, CAD models, human annotations, and peer observations can extend the map beyond one robot’s experience. They must be registered to the map frame, and their age and precision must remain visible. Learned priors may also suggest likely room layouts or object locations. Those suggestions should occupy a hypothesis layer: a predicted opening is a candidate to inspect, not evidence of collision-free passage. Richer representation cannot recover a hidden fact that was never observed or supplied by a reliable source. The aim is fewer redundant journeys and better targeted verification, rather than eliminating exploration.

6. Learning the transition function

The map supports a belief about what exists; a transition function predicts how relevant state changes after an action. The hybrid should learn that function where it improves choices. A useful target is a distribution such as p(z_next | z, a, e), where z summarizes task-relevant state, a is an action, and e describes the robot’s embodiment or operating context. A recurrent estimator can construct this state from observation history. An accompanying observation model or measurement update is needed to compare predictions with what the robot subsequently sees.

The representation should serve control. Predicting object displacement, door state, collision likelihood, or skill success may matter more than reconstructing every future pixel. Known kinematics can supply a baseline, while a learned residual accounts for slip, compliance, uncertain loads, or contact effects. Different scales can use different models: local motion may use continuous dynamics, while a longer plan uses probabilities and costs for actions such as passing through a door. These abstractions must retain the distinctions that change feasibility; a compact representation that discards handle state may be inadequate for opening a door.

Dreamer demonstrates learning behavior through imagined trajectories in a learned latent model. That precedent motivates prediction, but does not establish that the proposed robot should plan through long imagined futures. Initial experiments should use short horizons, compare predicted outcomes with actual outcomes, and replan frequently. Predictor disagreement can help flag unfamiliar situations, though agreement between models does not guarantee correctness. Observation noise and uncertainty from insufficient experience should be evaluated separately, and confidence estimates should be checked against held-out outcomes.

Training can combine recorded transitions with carefully evaluated new experience. A model should first run alongside the controller, predicting without choosing actions, so its errors are visible before planning depends on it. Evaluation must include consequences of action selection: a planner may discover inputs on which the predictor is overoptimistic. A successful design could retain direct policy execution for routine behavior and invoke prediction mainly for unfamiliar contacts, route alternatives, or recovery. Learning a transition function is thus a selective capability, with evidence required for where and when it earns its cost.

7. A peer-to-peer map for the fleet

Once memory has explicit identity, provenance, and versioning, it becomes possible to share more than isolated policy updates. Each robot could host detailed submaps for part of the environment, while a replicated directory records which peers hold which regions and versions. Partitioning by spatial region makes nearby queries efficient; a coarse topological graph helps a robot discover the next region to request. Important chunks need multiple replicas, and robots should cache their current operating area and likely next destinations. Distributing storage provides no benefit if every decision still requires downloading the entire map.

Communication should follow the same attention logic as perception. A robot can request a compact summary first, retrieve geometry or keyframes only when needed, and subscribe to changes affecting its route. Peers can exchange version summaries before sending missing updates. Capacity and available bandwidth should influence placement and replication. A local controller must continue to use current sensing during disconnection, with a policy for slowing or stopping when missing remote information is essential. Peer-to-peer availability depends on replication and caching; it is not guaranteed merely by removing a central server.

Map alignment and evidence reconciliation are harder than transferring bytes. Robots need credible relative poses and place matches before observations can be fused. Kimera-Multi provides a precedent for distributed metric-semantic mapping and inter-robot estimation. The sharded storage and attention-driven querying proposed here would require additional engineering. Submaps could retain local coordinate frames with versioned transforms between them, limiting how much geometry must be rewritten after a global correction. Navigation must account for uncertainty in those transforms.

A robust update protocol should preserve source observation identifiers, causal relationships, and conflicting hypotheses. Receiving the same image-derived claim through three peers must not triple its confidence. Choosing the latest timestamp alone cannot resolve two different objects mistakenly assigned one identity or two observations expressed in incompatible frames. Evidence records can merge before geometric estimates are reconciled; corrections and deletion records must propagate so stale peers do not revive invalid entries. Finally, shared geometry can transfer across robot types more readily than action dynamics. A narrow passage may be traversable for one robot and inaccessible to another, even when both agree perfectly on the map.

8. The complete loop in practice

Return to the cup retrieval task. The robot first retrieves the cup’s last observation, its support surface, and routes to that room. A peer supplies a recent submap indicating that a chair has moved. Provenance shows that this is new evidence rather than a forwarded copy of the robot’s own older map. The route planner revises the affected clearance estimate, while the local policy continues along the unaffected part of the corridor using current sensing.

At the doorway, the cup is occluded. The belief preserves its identity but represents uncertainty about its location. Attention compares a camera shift with a short movement that would reveal the relevant surface. The local transition model predicts the consequences and costs of the movement; an observation prediction estimates what would become visible. The robot chooses an informative view, updates the object record, and then executes its grasp policy. If the cup is absent from a clearly visible surface, the map revises the hypothesis and retrieval expands the search.

This example illustrates the dependency between the proposed components. Memory makes a targeted inspection possible. Attention avoids unnecessary reconstruction. Prediction helps choose the inspection action. Shared mapping avoids repeating another robot’s survey. None can be assumed to improve the task in isolation: a stale peer update, a missed visual change, or an inaccurate transition model could erase the benefit. The architecture should expose those errors and support comparisons with simpler alternatives.

9. Evaluation and limits

The central hypothesis should be tested against strong baselines, including a reactive policy, a recurrent model-free policy with comparable memory capacity, and a conventional mapping-and-planning system. Add explicit persistent memory, selective attention, learned transitions, and fleet sharing in controlled stages, followed by experiments removing each component from the complete system. Match sensor access, task distributions, training data, and resource budgets. A larger architecture given more experience or more compute would not isolate the value of the proposed organization.

Tasks should involve revisiting locations after delays, objects moving during occlusion, changing goals, inaccurate imported maps, and robots with different sizes or sensors. Fleet experiments should vary robot count, disconnection length, bandwidth, and the loss of peers holding useful chunks. Measure task completion, collisions, recovery time, unnecessary travel, and duplicate exploration alongside energy, storage, transferred bytes, and decision latency, including slow outliers. Prediction quality and map confidence should also be checked for calibration, while attention should be judged by consequential missed events as well as processing savings.

The proposal makes several falsifiable expectations. Persistent belief should help most when tasks reuse past observations; attention should help when a small part of the stored world matters at a time; learned transitions should help when alternative actions have consequential, predictable differences; fleet sharing should help when experiences overlap enough to be reusable. Each can fail. Rapid environmental change can make maintenance more expensive than fresh perception, poor associations can spread errors, and communication can cost more than a local observation. A direct policy may remain preferable for a short, stable task with little need for retained knowledge.

The position is therefore a commitment to a particular research question: how much more capable can a robot become when its experience persists, can be selectively revised, and can be used by others? A persuasive answer requires gains under these constraints, not just a more elaborate architecture. The proposed hybrid gives memory, attention, prediction, and communication a common purpose: preserving the evidence that matters and acquiring new evidence when it can improve the next action.