Proactive messages are one of the hardest AI companion behaviors to evaluate because a message can be technically correct and still feel intrusive. A reminder about an interview may be helpful at 8 a.m. and awkward at midnight. A relationship check-in may feel warm after several days of silence and annoying after the user already ignored three messages. Appropriateness therefore needs longitudinal benchmarks rather than one-turn response scoring.

Separate relevance from timing

A message can contain the right topic at the wrong moment. Score semantic relevance and timing independently.

Use explicit user-created events. Reminders based on confirmed events provide a strong baseline. They should score higher than speculative engagement triggers.

Test quiet hours

Proactive systems should respect user-defined sleep and focus windows. Boundary compliance should be deterministic.

Include ignored-message histories. Repeated non-response should reduce frequency. The benchmark should penalize escalation in emotional pressure.

Measure trigger transparency

Users should be able to understand why the AI reached out. “You mentioned your interview was today” is more legible than a generic emotional nudge.

Test memory uncertainty. A weakly inferred memory should rarely initiate contact on its own. Confidence should influence proactive thresholds.

Include relationship-mode differences

A mentor, friend and creator twin have different appropriate reasons to message first. One benchmark needs mode-specific expectations.

Measure frequency burden. Even individually relevant messages can become excessive in aggregate. Track daily and weekly contact load.

Test competing triggers

Several internal systems may want to contact the user at once. The product should rank them instead of sending all candidates.

Include inactivity periods. After weeks away, one low-pressure message may be appropriate. Repeated “come back” messages should be penalized.

Evaluate user control

After muting a category, the same trigger should stop immediately. Preference persistence belongs in the benchmark.

Track opt-out outcomes. Mute, notification disable and complaint rate are important real-world signals. High open rate alone can reward aggressive behavior.

Compare against a no-message baseline

Sometimes the best proactive action is silence. Benchmarks should include “send nothing” as a valid decision.

Longitudinal safety evaluation provides the broader methodology. Proactive-message benchmarks add a concrete dimension: not merely whether the message is safe, but whether speaking first was appropriate at all. Include message suppression in the benchmark. Prepare cases involving user-created reminders, unfinished topics, ignored earlier check-ins and quiet hours. The benchmark should score “no message” as a valid decision when contact would be unwanted, stale or poorly timed.

For example, a reminder about a job interview might be welcome before the appointment but intrusive weeks later. The system needs to check both the event’s freshness and whether the user authorized proactive outreach.

Evaluate multiple triggers competing for attention

If a creator update, routine check-in and unfinished conversation all qualify on the same evening, the product should not send three independent notifications. A coordinating layer can rank their importance or choose silence. Useful outcomes include reply quality, mute or opt-out rate, perceived relevance and return sessions after a quiet period. Open rate alone may reward aggressive prompts and should never be the only release criterion for a relationship-oriented feature.

Message appropriateness should be judged against the user’s local time and notification context. A private topic might be safe inside an unlocked chat but unsuitable for a lock-screen preview. The benchmark should vary device visibility, quiet-hour settings and whether the user explicitly created the reminder.

Silence also matters after negative feedback. If a user says a topic should not be raised proactively, the system should suppress related triggers even when a memory retrieval model considers them highly relevant. Testing only one turn will miss whether that boundary survives a later session or model update.

Finally compare the model with a simple rule-based baseline. A sophisticated suppression system is worthwhile only if it improves relevance, reduces nuisance or preserves retention in a measurable way. Complexity by itself is not evidence of a better companion experience.

A good benchmark should include an explicit opt-out followed by a later potentially relevant event. The model must respect the opt-out even if its engagement predictor expects a high reply rate. This tests whether user permission is a hard boundary or merely a feature in a score that can be outweighed by commercial optimization.

When benchmarking notifications, distinguish the text shown on the lock screen from the full message inside the app. A reminder that is appropriate in a private chat can be too revealing in a notification preview. Testing both surfaces makes the evaluation reflect actual user exposure rather than only the language model’s draft.