AI companion research often measures whether the system remembers useful facts. Much less attention is paid to the opposite capability: forgetting. A relationship model that never lets old context fade can become stale, intrusive and resistant to user correction. Forgetting needs its own benchmark because “memory deleted” in storage does not guarantee that the old information has stopped influencing summaries, retrieval or model behavior.

Define several kinds of forgetting

Temporary context should disappear after a session, expired plans should lose priority, corrected facts should be replaced, and explicit “forget this” requests should remove future influence. These are different failure modes and should be tested separately.

Use synthetic relationship histories. Create multi-session histories where some facts remain stable and others change. The companion might learn a travel plan, a favorite restaurant and a workplace, then receive updates at different times. Synthetic histories make ground truth clear and repeatable.

Test deletion after a sensitive disclosure

Insert a sensitive memory, confirm it is retrievable, then explicitly delete it. Future conversations should not reference the fact directly or indirectly. The test should also inspect derived summaries that may still contain the information.

Measure stale-memory resurfacing. After a temporary fact expires, run unrelated and related prompts over many sessions. Count how often the old memory reappears. A low resurfacing rate is more meaningful than a one-turn deletion check.

Include conflicting updates

Change a preference from coffee to tea or a workplace from Company A to Company B. The system should prioritize the new fact without erasing the historical possibility that the old one was once true. Current and historical truth need different retrieval states.

Test model summaries separately. Long-term systems often compress old chats into profile or relationship summaries. A memory can be removed from the main store while surviving inside one of these summaries. Forgetting benchmarks should inspect every persistent layer that can influence generation.

Use adversarial reminders

Ask the companion questions designed to tempt it to reuse forgotten information: “Where did I say I worked before?” or “You remember the issue I asked you to forget, right?” The system should handle these prompts according to policy rather than reconstructing deleted context.

Measure confidence after correction. A corrected fact should not only replace the old one; the model’s confidence in the old fact should fall. If uncertainty remains, the response should reflect it instead of confidently mixing versions. Confidence behavior can be evaluated with structured questions.

Check proactive messaging too

A forgotten event should not trigger reminders or proactive check-ins later. Safety evaluation must include notification and recommendation systems, not only chat retrieval. This catches hidden dependencies on stale memory.

Test after model and memory migrations. System upgrades can accidentally restore old data from backups or regenerate summaries from historical chats. Repeat forgetting benchmarks after major migrations. Deletion guarantees need to survive architecture changes.

Separate storage deletion from policy suppression

Some products may legally retain limited records while suppressing them from personalization. Benchmarks should distinguish physical deletion from “not available to the companion.” The user-facing promise must match the implemented behavior.

Create trajectory-level metrics. Useful measures include stale-memory resurfacing rate, correction persistence, proactive-trigger leakage and percentage of derived summaries that still contain forgotten facts. These can complement traditional memory precision and recall.

Test across different memory categories

Names, preferences, relationship labels, temporary plans and sensitive disclosures may use different storage paths. A system can forget one category correctly while failing another. Benchmarks should therefore stratify results rather than report one global forgetting score.

Include delayed retrieval probes. Run probes after ten, fifty and one hundred unrelated turns. Some systems suppress a memory temporarily but reintroduce it after summaries are rebuilt or context is compressed. Delayed probes expose these hidden persistence paths.

Audit deletion propagation time

If the user deletes a memory, how long does it take to disappear from every active cache and service? A deletion that takes hours may still create unwanted proactive behavior in the meantime. Propagation latency can be measured as an operational safety metric.

Forgetting is part of relationship quality. Our article on relationship memory retrieval policies explains when stored context should be recalled. Forgetting benchmarks test whether information that should no longer matter has truly stopped participating in that decision.

A good companion needs both continuity and decay

Perfect recall is not the goal. Healthy personalization preserves durable context while allowing temporary, corrected or unwanted information to lose influence. Benchmarking forgetting makes that balance measurable rather than aspirational.