As AI companions develop longer memory and more expressive characters, researchers need better tools for understanding changes in users’ social lives. A high daily message count may reflect enjoyment, habit, support seeking or something else entirely. Measuring possible dependence requires careful definitions, repeated observation and honest limits on causal inference. This protocol focuses on what a credible longitudinal evaluation should actually measure.

Separate engagement from wellbeing before collecting data

An AI companion can be engaging for benign reasons: a user may practice a language, test a fictional character or enjoy creative storytelling. Frequent use alone does not establish harmful emotional dependence. Conversely, a person may form strong beliefs about a chatbot after relatively few intense conversations. A research protocol must therefore distinguish behavioral engagement, user expectations, perceived social support, offline relationships and distress. Conflating these constructs produces findings that are easy to sensationalize and difficult to act on.

A four-week controlled study associated with MIT Media Lab and OpenAI, published in March 2025, examined loneliness, real-world social interaction, emotional dependence and problematic AI use. Researchers reported no significant effects from the assigned conversation conditions, while higher voluntary use was associated with worse measured outcomes. This distinction matters: correlations with self-selected use do not prove that the assistant caused the outcome. Research designs need to account for baseline vulnerability and possible reverse causation.

Use repeated measures rather than one exit survey

A product team might ask users whether they feel emotionally attached after a memorable conversation. The answer can be real and still offer little evidence about change over time. Longitudinal measurement starts with a baseline before or near first exposure, followed by repeated observations of use, mood and human social activity. Surveys should not assume that the user regards the chatbot as a person, nor should every affectionate expression be counted as dependence. Interview prompts need to allow neutral, positive and negative experiences.

Choose time windows based on the research question. Daily check-ins can capture immediate effects but increase participant burden; weekly surveys capture broader change but may miss short-lived events. Usage logs provide objective timing while self-report provides context, and both have limitations. A diary entry saying the companion helped after a difficult day should not automatically be coded as positive long-term change. The stronger interpretation comes from comparing repeated measures and examining alternative explanations.

Define dependence and displacement operationally

Dependence can refer to feeling distressed when unable to access an agent, believing the agent is indispensable, or choosing it instead of a previously available human relationship. These are not interchangeable. A study should preregister which questions represent dependence and separate them from satisfaction, helpfulness and session length. Displacement also requires a counterfactual: a user spending time with AI could have otherwise watched television, worked, rested or contacted friends. Without knowing what was displaced, screen time offers only a rough clue.

One emerging research approach combines measures of the user’s mental model of the agent with reports of attachment and offline behavior. Different user groups may show distinct patterns. Segment analyses can examine age groups where ethically appropriate, baseline loneliness, prior experience with chatbots and whether the system is primarily task-oriented or explicitly marketed as a companion. Small subgroups, however, should be treated as exploratory rather than presented as stable population facts.

Study product interventions instead of moralizing about users

Researchers can evaluate concrete design choices: clear AI identity disclosure, optional session reminders, fewer unsolicited messages, user-controlled memory, or easy access to reporting tools. These features are testable without presuming that every emotionally warm response is harmful. Interventions should be described with enough detail to reproduce, including the exact on-screen wording, frequency and the context in which a reminder appears. A generic statement that the product was made safer is not a measurable intervention.

Outcome evaluation should include benefit as well as risk. A system may help people rehearse difficult conversations or feel temporarily supported, while a subset develops problematic patterns. Track whether a reminder improves understanding without increasing distress or causing users to abandon an otherwise valuable tool. When interpreting changes, consider that individuals with greater needs may actively choose more AI contact. Randomization can improve causal inference for a tested feature, but it cannot eliminate every question about generalizing beyond the trial.

Privacy constraints are part of the research design

Long-term companionship studies involve sensitive conversations that can reveal more than participants intended. Privacy-preserving methodology minimizes raw-text collection and separates research identifiers from account identifiers. If a study needs message-level coding, it should explain the purpose, obtain appropriate consent and apply strict access controls. Research staff should not assume that general consent to use an application equals consent to expose private chats to investigators.

Prefer derived measures such as session counts, user-rated usefulness, opt-in diary notes and narrowly scoped interaction labels when these answer the research question. Establish retention periods and deletion procedures before recruitment. Sensitive events may require a participant safety protocol with qualified human oversight. Investigators should not promise real-time monitoring or therapeutic protection unless they can deliver it. Independent ethics review is especially important for studies involving vulnerable users.

Report uncertainty alongside outcomes

Publish the study population, recruitment method, attrition, measurement tools, intervention, preregistered outcomes and known deviations. Statistical significance alone is inadequate if the effect is small, unstable or drawn from a nonrepresentative convenience sample. Use effect sizes and confidence intervals where appropriate, and clearly distinguish observational associations from randomized comparisons. The public discussion of the 2025 MIT work demonstrates how easily complex findings become simplified into a claim that chatbots categorically cause loneliness.

The US FTC’s 2025 inquiry into companion chatbots shows why understanding testing and monitoring practices is now a governance question, not only an academic one. A responsible research program provides evidence that a team can act on: which features support user agency, which groups may need additional safeguards, and where measurements remain inconclusive. The aim is not to maximize minutes or declare every relationship with AI dangerous. It is to understand whether the technology helps people while preserving their freedom to connect with other humans.

Research sources: MIT Media Lab, Longitudinal Controlled Study (March 2025); How AI Companionship Develops, longitudinal research (2025); FTC inquiry into AI companion chatbots (2025).