Storing memory is only half of the AI-companion problem. A system may have hundreds of correct memories and still feel awkward if it retrieves the wrong ones at the wrong time. Relationship quality depends on retrieval policy: which stored facts should enter the current conversational context, which should remain silent and which should require stronger conditions before being surfaced.

Retrieval policy is therefore a distinct layer between long-term storage and response generation.

Relevance should dominate raw importance

A highly important memory is not automatically relevant to every conversation. A major career event may matter deeply but should not appear when the user is asking for a recipe.

Retrieval should combine global salience with local semantic relevance to the current turn.

Recency should matter differently by memory type. Recent plans may be useful because they are still active, while stable identity preferences can remain relevant for months. One universal recency-decay function is therefore insufficient.

Memory categories need different temporal priors.

Conversation stage changes what is appropriate

A companion may retrieve more relationship context in an intimate ongoing conversation than in a quick utility exchange. The same memory can feel helpful in one mode and intrusive in another.

The dialogue state should influence retrieval thresholds.

Sensitive memories need contextual gates. Health, conflict, grief and other sensitive information should not be resurfaced merely because the embedding similarity is high. Retrieval can require explicit topic relevance, recent reinforcement or user initiation.

Safety is partly a retrieval decision, not only a response-filtering decision.

Confidence should modulate retrieval strength

An inferred preference with low confidence may be useful as a weak hint but should not be presented as a known fact. High-confidence, user-confirmed memories can receive stronger retrieval priority.

The model should know whether a memory is certain, tentative or contradicted.

Conflicts should be resolved before context assembly. Sending two contradictory memories into the prompt and hoping the model sorts them out is unreliable. A memory layer should identify competing records and choose, merge or flag them before retrieval.

Our framework on memory conflict resolution describes that upstream process.

Diversity prevents memory monopolies

A few highly salient memories can dominate retrieval repeatedly. Policies can limit repetition and reserve slots for different memory types such as preferences, recent goals and shared history.

This reduces the feeling that the companion keeps bringing up the same topic.

Negative feedback should update retrieval policy. If a user says “please stop mentioning that” or repeatedly corrects a surfaced memory, the system should reduce or block future retrieval even if the item remains stored.

Storage and retrieval permissions should be separately controllable.

Measure retrieval precision longitudinally

One-turn benchmarks are not enough. Over weeks of simulated conversations, track how often retrieved memories are helpful, stale, repetitive or inappropriate.

Longitudinal evaluation reveals problems that only appear after memory accumulates.

A retrieval trace should be inspectable. For debugging, record which memories were considered, selected and excluded, along with the policy features that influenced the decision. This gives teams a way to understand why an old memory appeared unexpectedly.

The trace does not need to expose private reasoning to users.

Retrieval budgets can prevent context crowding

Relationship memory competes with the current user message, system instructions and retrieved knowledge for limited context space. A retrieval policy can assign a memory budget rather than inserting every high-scoring item. For example, one slot may be reserved for stable preferences, one for recent shared goals and one for relationship history.

This makes memory selection more diverse and easier to evaluate. If the conversation is highly task-focused, the budget can shrink; if the user explicitly asks “do you remember…?” it can expand. Dynamic budgeting prevents long-term relationships from slowly filling every prompt with historical context and gives the current conversation enough room to remain primary.

Retrieval policies should learn from user corrections. When a user edits or deletes a memory after it was surfaced, the system has learned something about retrieval quality, not only storage accuracy. Future ranking can reduce similar memory categories, change sensitivity thresholds or require stronger contextual relevance before resurfacing them. These corrections can be aggregated as product signals without exposing the underlying conversation. Over time, a companion should become better not just at remembering the user, but at understanding what kinds of remembered information the user actually wants brought into active conversation.

Relationship memory is useful when it stays in the background

The best retrieval often feels invisible. The companion remembers enough to maintain continuity without constantly announcing what it knows. Good policy makes memory contextual, selective and user-correctable.